Was this Helpful?

RegularExpressionProtection policy icon Regular Expression Protection policy

About | Samples | Element reference | Error codes | Schemas | Usage notes | Related topics

What

Extracts information from a message (for example, URI Path, Query Param, Header, Form Param, Variable, XML Payload, or JSON Payload) and evaluates that content against predefined regular expressions. If any specified regular expressions evaluate to true, the message is considered a threat and is rejected.

Where

This policy can be attached in the following locations.

ProxyEndpoint TargetEndpoint
    PreFlow Flow PostFlow PreFlow Flow PostFlow    
Request    
    Response
    PostFlow Flow PreFlow PostFlow Flow PreFlow    

Samples

<RegularExpressionProtection name="JsonPathRegExProtection">
    <DisplayName>Regular Expression Protection 1</DisplayName>
    <Source>request</Source>
    <JSONPayload escapeSlashCharacter="true">
       <JSONPath>
          <Expression>$.</Expression>
          <Pattern>&lt;\s*script\b[^&gt;]*&gt;[^&lt;]+&lt;\s*\/\s*script\s*&gt;
          </Pattern>
          <Pattern>n\s*\\\\\s*slash</Pattern>
          <Pattern>n\s*\/\s*slash</Pattern>
          <Pattern>n\s*\\"\s*quotes</Pattern>
          <Pattern>n\s*\\b\s*space</Pattern>
          <Pattern>n\s*\\f\s*forwardfeed</Pattern>
          <Pattern>n\s*\\n\s*newline</Pattern>
          <Pattern>n\s*\\r\s*carria</Pattern>
          <Pattern>n\s*\\t\s*tab</Pattern>
          <Pattern>n\s*\\uFFFF\s*hex</Pattern>
       </JSONPath>
    </JSONPayload>
 </RegularExpressionProtection>

The sample above illustrates how to use the RegularExpressionProtection policy to evaluate JSON payloads for SQL injection attacks. Specifically, the content extracted by <JSONPath>/<Expression> is evaluated against the regular expression in <JSONPath>/<Pattern>.

If the regular expression in your <JSONPath>/<Pattern> includes XML-reserved characters (", &, ', <, or .), you must XML-encode it before you include it in the policy XML configuration file. For example, in the sample above, the regular expression <\s*script\b[^>]*>[^<]+<\s*\/\s*script\s*> has been XML-encoded as &lt;\s*script\b[^&gt;]*&gt;[^&lt;]+&lt;\s*\/\s*script\s*&gt;.

Additionally, if your regular expression includes forward slashes (/), you must escape them by setting the <JSONPayload> escapeSlashCharacter attribute to true.

Element reference

The element reference describes the elements and attributes of the RegularExpressionProtection policy.

<RegularExpressionProtection async="false" continueOnError="false" enabled="true" name="Regular-Expression-Protection-1">
     <DisplayName>Regular Expression Protection 1</DisplayName>
     <Source>response</Source>
     <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
     <URIPath>
         <Pattern>[tT]rue</Pattern>
         <Pattern>.*true.*</Pattern>
     </URIPath>
     <QueryParam name="greeting">
         <Pattern>[tT]rue</Pattern>
         <Pattern>.*true.*</Pattern>
     </QueryParam>
     <Header name="greeting">
         <Pattern>[tT]rue</Pattern>
         <Pattern>.*true.*</Pattern>
     </Header>
     <FormParam name="greeting">
         <Pattern>[tT]rue</Pattern>
         <Pattern>.*true.*</Pattern>
     </FormParam>
     <Variable name="request.content">
         <Pattern>[tT]rue</Pattern>
         <Pattern>.*true.*</Pattern>
     </Variable>
     <XMLPayload>
         <Namespaces>
             <Namespace prefix="apigee">http://www.apigee.com</Namespace>
         </Namespaces>
         <XPath>
             <Expression>/apigee:Greeting/apigee:User</Expression>
             <Type>string</Type>
             <Pattern>[tT]rue</Pattern>
             <Pattern>.*true.*</Pattern>
         </XPath>
     </XMLPayload>
     <JSONPayload>
         <JSONPath>
             <Expression>$.store.book[*].author</Expression>
             <Pattern>[tT]rue</Pattern>
             <Pattern>.*true.*</Pattern>
         </JSONPath>
	</JSONPayload>
</RegularExpressionProtection>

<RegularExpressionProtection> attributes

<RegularExpressionProtection async="false" continueOnError="false" enabled="true" name="Regular-Expression-Protection-1"> 
Attribute Description Default Presence
async

Set to true to specify that the policy should be run in a thread pool different than the pool servicing the request/response flow. Default is false.

Note: This setting is only used for for internal optimization. Contact Apigee support at the Support Portal for more information.

false Optional
continueOnError

Most policies are expected to return an error when a failure occurs (for example, when a Quota is exceeded). By setting this attribute to true, Flow execution continues on failure.

false Optional
enabled Determines whether a policy is enforced or not. If set to false, a policy is 'turned off', and not enforced (even though the policy remains attached to a Flow). true Optional
name

The internal name of the policy. Characters you can use in the name are restricted to: A-Z0-9._\-$ %. However, the Edge management UI enforces additional restrictions, such as automatically removing characters that are not alphanumeric.

Optionally, use the <DisplayName> element to label the policy in the management UI proxy editor with a different, natural-language name.

N/A Required

<DisplayName> element

A natural-language name that labels the policy in the management UI proxy editor. If omitted, the policy name attribute is used.

<DisplayName>Custom label used in UI</DisplayName>
Default: Policy name attribute value.
Presence: Optional
Type: String

<Source> element

Indicates the message from which information needs to be extracted.

If the <Source> element is omitted, the value defaults to message. For example, <Source>message</Source>. When set to message, the policy uses the request message as source when attached to a request flow. Likewise, the policy uses the response message when attached to a response flow.

If the source message cannot be resolved or if it resolves to a non-message type, the policy returns an error.

<Source>response</Source>
Default: N/A
Presence: Optional
Type: String

<IgnoreUnresolvedVariables> element

If set to true and any variable is unresolvable, the policy returns an error. If set to true and any variable is unresolvable, the unresolved variable is treated as empty string (Null).

<IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
Default: false
Presence: Optional
Type: Boolean

<URIPath> element

Specifies that information needs to be extracted from the request URI path and evaluated against the regular expressions provided.

<URIPath>
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>
</URIPath>
Default: N/A
Presence: Optional
Type: N/A

<URIPath>/<Pattern> element

Provides the regular expressions against which information extracted from the URI path needs to be evaluated.

<URIPath>
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>
</URIPath>
Default: N/A
Presence: Required
Type: String

<QueryParam> element

Specifies that information needs to be extracted from the request query parameter and evaluated against the regular expressions provided.

<QueryParam name="greeting">
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>     
</QueryParam>
Default: N/A
Presence: Optional
Type: N/A

Attributes

Attribute Description Default Presence
name

Name of the request query parameter from which information needs to be extracted for evaluation against the regular expressions provided.

N/A Required

<QueryParam>/<Pattern> element

Provides the regular expressions against which information extracted from the request query parameter needs to be evaluated.

<QueryParam name="greeting">
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>     
</QueryParam>
Default: N/A
Presence: Required
Type: String

<Header> element

Specifies that information needs to be extracted from the request and response headers and evaluated against the regular expressions provided.

<Header name="greeting">
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>     
</Header>
Default: N/A
Presence: Optional
Type: N/A

Attributes

Attribute Description Default Presence
name

Name of the request and response header from which information needs to be extracted for evaluation against the regular expressions provided.

N/A Required

<Header>/<Pattern> element

Provides the regular expressions against which information extracted from the request and response headers needs to be evaluated.

<Header name="greeting">
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>     
</Header>
Default: N/A
Presence: Required
Type: String

<FormParam> element

Specifies that information needs to be extracted from the request form parameter and evaluated against the regular expressions provided.

<FormParam name="greeting">
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>     
</FormParam>
Default: N/A
Presence: Optional
Type: N/A

Attributes

Attribute Description Default Presence
name

Name of the request form parameter from which information needs to be extracted for evaluation against the regular expressions provided.

N/A Required

<FormParam>/<Pattern> element

Provides the regular expressions against which information extracted from the request form parameter needs to be evaluated.

<FormParam name="greeting">
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>     
</FormParam>
Default: N/A
Presence: Required
Type: String

<Variable> element

Specifies that information needs to be extracted from the given variable and evaluated against the regular expressions provided.

<Variable name="request.content">
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>
</Variable>
Default: N/A
Presence: Optional
Type: N/A

Attributes

Attribute Description Default Presence
name

Name of the variable from which information needs to be extracted for evaluation against the regular expressions provided.

N/A Required

<Variable>/<Pattern> element

Provides the regular expressions against which information extracted from the given variable needs to be evaluated.

<Variable name="request.content">
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>
</Variable>
Default: N/A
Presence: Required
Type: String

<XMLPayload> element

Specifies that information needs to be extracted from an XML payload and evaluated against the regular expressions provided.

<XMLPayload>
   <Namespaces>
      <Namespace prefix="apigee">http://www.apigee.com</Namespace>
   </Namespaces>
   <XPath>
      <Expression>/apigee:Greeting/apigee:User</Expression>
      <Type>string</Type>
      <Pattern>[tT]rue</Pattern>
      <Pattern>.*true.*</Pattern>
   </XPath>
</XMLPayload>
Default: N/A
Presence: Optional
Type: N/A

<XMLPayload>/<Namespaces> element

Specifies the namespaces to be used in the XPath evaluation.

<XMLPayload>
   <Namespaces>
      <Namespace prefix="apigee">http://www.apigee.com</Namespace>
   </Namespaces>
   <XPath>
      <Expression>/apigee:Greeting/apigee:User</Expression>
      <Type>string</Type>
      <Pattern>[tT]rue</Pattern>
      <Pattern>.*true.*</Pattern>
   </XPath>
</XMLPayload>
Default: N/A
Presence: Optional
Type: String

<XMLPayload>/<Namespaces>/<Namespace> element

Specifies each namespace to be used in the XPath evaluation.
<Namespaces>
   <Namespace prefix="apigee">http://www.apigee.com</Namespace>
</Namespaces>
Default: N/A
Presence: Optional
Type: String

Attributes

Attribute Description Default Presence
prefix

Provides a prefix to help qualify a given namespace.

N/A Required

<XMLPayload>/<XPath> element

Specifies the XPath to be evaluated.
<XPath>
   <Expression>/apigee:Greeting/apigee:User</Expression>
   <Type>string</Type>
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>
</XPath>
Default: N/A
Presence: Optional
Type: N/A

<XMLPayload>/<XPath>/<Expression> element

Specifies the XPath expression defined for the variable. Only XPath 1.0 expressions are supported. For example, <Expression>/company/employee[@age>=$request.header.age]</Expression> extracts details for employees whose age is greater than or equal to the value specified in request.header.age.
<XPath>
   <Expression>/apigee:Greeting/apigee:User</Expression>
   <Type>string</Type>
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>
</XPath>
Default: N/A
Presence: Optional
Type: String

<XMLPayload>/<XPath>/<Type> element

Specifies the datatype.
<XPath>
   <Expression>/apigee:Greeting/apigee:User</Expression>
   <Type>string</Type>
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>
</XPath>
Default: string
Presence: Optional
Type: String
Valid values:

String.

Valid values include string, boolean, int, long, float, double, and nodeset.

<XMLPayload>/<XPath>/<Pattern> element

Defines the regular expression pattern. If a regular expression in your<Pattern> element includes XML-reserved characters (", &, ', <, or .), you must XML-encode it before you include it.

<XPath>
   <Expression>/apigee:Greeting/apigee:User</Expression>
   <Type>string</Type>
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>
</XPath>
Default: N/A
Presence: Required
Type: String

<JSONPayload> element

Specifies that information needs to be extracted from a JSON payload and evaluated against the regular expressions provided.

<JSONPayload>
   <JSONPath>
      <Expression>$.store.book[*].author</Expression>
      <Pattern>[tT]rue</Pattern>
      <Pattern>.*true.*</Pattern>
   </JSONPath>
</JSONPayload>
Default: N/A
Presence: Optional
Type: N/A

Attributes

Attribute Description Default Presence
escapeSlashCharacter

Set to true to escape any forward slash (/) characters in the regular expressions in <JSONPath>/<Pattern>.

true Optional

<JSONPayload>/<JSONPath>/<Expression> element

Specifies the JSONPath expression defined for the variable.

<JSONPath>
   <Expression>$.store.book[*].author</Expression>
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>
</JSONPath>
Default: N/A
Presence: Optional
Type: String

<JSONPayload>/<JSONPath>/<Pattern> element

Defines the regular expression pattern. If a regular expression in your <Pattern> element includes XML-reserved characters (", &, ', <, or .), you must XML-encode it before you include it.

<JSONPath>
   <Expression>$.store.book[*].author</Expression>
   <Pattern>[tT]rue</Pattern>
   <Pattern>.*true.*</Pattern>
</JSONPath>
Default: N/A
Presence: Required
Type: String

Error codes

The default format for error codes returned by policies is:

{
  "code" : " {ErrorCode} ",
  "message" : " {Error message} ",
  "contexts" : [ ]
}

The RegularExpressionProtection policy type defines the following error codes:

Error Code Message
NothingToEnforce RegularExpressionProtection {0}: at least one of URIPath, QueryParam, Header, FormParam, XMLPayload, JSONPayload is mandatory
NoPatternsToEnforce RegularExpressionProtection {0}: No patterns to enforce in {1}
EmptyXPathExpression RegularExpressionProtection {0}: Empty XPath expression
EmptyJSONPathExpression RegularExpressionProtection {0}: Empty JSONPath expression
DuplicatePrefix RegularExpressionProtection {0}: Duplicate prefix {1}
NONEmptyPrefixMappedToEmptyURI RegularExpressionProtection {0}: Non-empty prefix {1} cannot be mapped to empty uri
ThreatDetected Regular Expression Threat Detected in {0}: regex: {1} input: {2}
ExecutionFailed Failed to execute the RegularExpressionProtection StepDefinition {0}. Reason: {1}
VariableResolutionFailed Failed to resolve variable {0}
NonMessageVariable Variable {0} does not resolve to a Message
SourceMessageNotAvailable {0} message is not available for RegularExpressionProtection StepDefinition {1}
InvalidRegularExpression RegularExpressionProtection {0}: Invalid Regular Expression {1}, Context {2}
InstantiationFailed Failed to instantiate the RegularExpressionProtection StepDefinition {0}
CannotBeConvertedToNodeset RegularExpressionProtection {0}: Result of xpath {1} cannot be converted to nodeset. Context {2}
XPathCompilationFailed RegularExpressionProtection {0}: Failed to compile xpath {1}. Context {2}
JSONPathCompilationFailed RegularExpressionProtection {0}: Failed to compile jsonpath {1}. Context {2}

Schemas

See our GitHub repository samples for the most recent schemas.

Usage notes

Apigee Edge enables you to configure regular expressions that can be evaluated against API traffic at runtime to identify common content-level threats that follow certain patterns.

A regular expression, or regex for short, is a set of strings that specify a pattern in a string. Regular expressions enable content to be programmatically evaluated for patterns. Regular expressions can be used, for example, to evaluate entered a properly structured email address. For more information, see Regular Expressions in the Java Tutorials.

The most common usage of RegularExpressionProtection is the evaluation of JSON and XML payloads for malicious content.

No regular expression can eliminate all content-based attacks, and multiple mechanisms should be combined to enable defense-in-depth. With this in mind, some recommended patterns for blacklisting content:

Blacklist patterns

Regular expressions must be XML-encoded in the policy's XML configuration file.

Name Regular Expression
SQL Injection
[\s]*((delete)|(exec)|(drop\s*table)|(insert)|(shutdown)|(update)|(\bor\b))
Server-Side Include Injection
<!--\s*<!--(include|exec|echo|config|printenv)\s+.*

XML encoded: &lt;!--\s*&lt;!--(include|exec|echo|config|printenv)\s+.*

XPath Abbreviated Syntax Injection
(/(@?[\w_?\w:\*]+(\+\])*)?)+
XPath Expanded Syntax Injection
/?(ancestor(-or-self)?|descendant(-or-self)?|following(-sibling))
JavaScript Injection
<\s*script\b[^>]*>[^<]+<\s*/\s*script\s*>

XML encoded: &lt;\s*script\b[^&gt;]*&gt;[^&lt;]+&lt;\s*/\s*script\s*&gt;

Java Exception Injection
.*Exception in thread.*

Related topics

JSON Threat Protection policy

XML Threat Protection policy


Help or comments?

  • Something's not working: See Apigee Support
  • Something's wrong with the docs: Click Send Feedback in the lower right.
    (Incorrect? Unclear? Broken link? Typo?)