Was this helpful?

A regular expression, or regex for short, is a set of strings that specify a pattern in a string. Regular expressions enable content to be programmatically evaluated for patterns. Regular expressions can be used, for example, to evaluate entered a properly structured email address. For more information, see Regular Expressions in the Java Tutorials.

Apigee Edge enables you to configure regular expressions that can be evaluated at runtime against API traffic to identify common content-level threats that follow certain patterns. The policy extracts information from a message (for example, URI Path, Query Param, Header, Form Param, Variable, XML Payload, or JSON Payload) and evaluates that content against pre-defined regular expressions. If any specified regular expressions evaluate to true, the message is considered a threat and is rejected.

The most common usage of RegularExpressionProtection is the evaluation of JSON and XML payloads for malicious content.

For example, to evaluate JSON payloads for SQL injection attacks, evaluate the JSON payload using the following JSONPath expression combined with a regular expression. The regular expression pattern is evaluated against the content extracted by the JSONPath expression.

Note that you need to URL encode regular expressions in the XML configuration file. In the example below, the following regular expression:

[\s]*((delete)|(exec)|(drop\s*table)|(insert)|(shutdown)|(update)|(\bor\b))

is encoded as:

%5B%5Cs%5D%2A%28%28delete%29%7C%28exec%29%7C%28drop%5Cs%2Atable%29%7C%28insert%29%7C%28shutdown%29%7C%28update%29%7C%28%5Cbor%5Cb%29%29

For example:

<RegularExpressionProtection name="JsonSQLInjectionEvaluation">
  <Source>request</Source>
  <JSONPayload>
    <JSONPath>
      <Expression>$.</Expression>
      <--The regular expression below is URL encoded --> 
      <Pattern>%5B%5Cs%5D%2A%28%28delete%29%7C%28exec%29%7C%28drop%5Cs%2Atable%29%7C%28insert%29%7C%28shutdown%29%7C%28update%29%7C%28%5Cbor%5Cb%29%29</Pattern>
    </JSONPath >
  </JSONPayload>
</RegularExpressionProtection>

No regular expression can eliminate all content-based attacks, and multiple mechanisms should be combined to enable defense-in-depth. With this in mind, some recommended patterns for blacklisting content:

Blacklist patterns

Regular expressions must be URL encoded in the policy's XML configuration file.

Name Regular Expression
SQL Injection
[\s]*((delete)|(exec)|(drop\s*table)|(insert)|(shutdown)|(update)|(\bor\b))
Server-Side Include Injection
<!--\s*<!--(include|exec|echo|config|printenv)\s+.*
XPath Abbreviated Syntax Injection
(/(@?[\w_?\w:\*]+(\+\])*)?)+
XPath Expanded Syntax Injection
/?(ancestor(-or-self)?|descendant(-or-self)?|following(-sibling))
JavaScript Injection
<\s*script\b[^>]*>[^<]+<\s*/\s*script\s*>
Java Exception Injection
.*Exception in thread.*

Configuring RegularExpressionProtection policy

Configure the RegularExpressionProtection policy using the following elements.

Field Name Description
Source (Optional) Contains the message from which information needs to be extracted.
  • If the Source element is omitted, the value defaults to message. For example, <Source>message</Source>. When set to message, the policy uses the request message as source when attached to a request Flow. Likewise, the policy uses the response message when attached to a response Flow.
  • If Source cannot be resolved, or if it resolves to a non-message type, the policy returns an error.
IgnoreUnresolvedVariables (Optional) Valid values: true/false
Default value: false
If set to true and any variable is unresolvable, the policy returns an error.
If set to true and any variable is unresolvable, the unresolved variable is treated as empty string (Null).
Patterns (Optional) URIPath Extracts information from the request URI path and matches it with the specified regular expressions.
QueryParam Extracts information from the request query parameter and matches it with the specified regular expressions.
Header Extracts information from the request and response header and matches it with the specified regular expressions.
FormParam Extracts information from the request form parameter and matches it with the specified regular expressions.
Variable Extracts information from the given variable and matches it with the specified regular expressions.
XMLPayload (Optional) Namespaces Specifies the namespace to be used in the XPath evaluation.
XPath Expression Specifies the XPath expression defined for the variable. Only XPath 1.0 expressions are supported. For example, <Expression>/company/employee[@age>=$request.header.age]</Expression> extracts details for employees whose age is greater than or equal to the value specified in request.header.age.
Type Specifies the datatype; default is string.
Pattern Defines the regular expression pattern.
JSONPayload (Optional) JSONPath Expression Specifies the JSONPath expression defined for the variable.
Pattern Defines the regular expression pattern.

Example - RegularExpression Protection policy

<RegularExpressionProtection name="mypolicy">
    <Source>response</Source>
    <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
    <URIPath>
        <Pattern>[tT]rue</Pattern>
        <Pattern>.*true.*</Pattern>
    </URIPath>
    <QueryParam name="greeting">
        <Pattern>[tT]rue</Pattern>
        <Pattern>.*true.*</Pattern>
    </QueryParam>
    <Header name="greeting">
        <Pattern>[tT]rue</Pattern>
        <Pattern>.*true.*</Pattern>
    </Header>
    <FormParam name="greeting">
        <Pattern>[tT]rue</Pattern>
        <Pattern>.*true.*</Pattern>
    </FormParam>
    <Variable name="request.content">
        <Pattern>[tT]rue</Pattern>
        <Pattern>.*true.*</Pattern>
    </Variable>
    <XMLPayload>
        <Namespaces>
            <Namespace prefix="apigee">http://www.apigee.com</Namespace>
        </Namespaces>
        <XPath>
            <Expression>/apigee:Greeting/apigee:User</Expression>
            <Type>string</Type>
            <Pattern>[tT]rue</Pattern>
            <Pattern>.*true.*</Pattern>
        </XPath>
    </XMLPayload>
    <JSONPayload>
        <JSONPath>
            <Expression>$.store.book[*].author</Expression>
            <Pattern>[tT]rue</Pattern>
            <Pattern>.*true.*</Pattern>
        </JSONPath >
    </JSONPayload>
</RegularExpressionProtection>

Policy-specific error codes

The default format for error codes returned by Policies is:

{
  "code" : " {ErrorCode} ",
  "message" : " {Error message} ",
  "contexts" : [ ]
}

The RegularExpressionProtection Policy type defines the following error codes:

Error Code Message
NothingToEnforce RegularExpressionProtection {0}: at least one of URIPath, QueryParam, Header, FormParam, XMLPayload, JSONPayload is mandatory
NoPatternsToEnforce RegularExpressionProtection {0}: No patterns to enforce in {1}
EmptyXPathExpression RegularExpressionProtection {0}: Empty XPath expression
EmptyJSONPathExpression RegularExpressionProtection {0}: Empty JSONPath expression
DuplicatePrefix RegularExpressionProtection {0}: Duplicate prefix {1}
NONEmptyPrefixMappedToEmptyURI RegularExpressionProtection {0}: Non-empty prefix {1} cannot be mapped to empty uri
ThreatDetected Regular Expression Threat Detected in {0}: regex: {1} input: {2}
ExecutionFailed Failed to execute the RegularExpressionProtection StepDefinition {0}. Reason: {1}
VariableResolutionFailed Failed to resolve variable {0}
NonMessageVariable Variable {0} does not resolve to a Message
SourceMessageNotAvailable {0} message is not available for RegularExpressionProtection StepDefinition {1}
InvalidRegularExpression RegularExpressionProtection {0}: Invalid Regular Expression {1}, Context {2}
InstantiationFailed Failed to instantiate the RegularExpressionProtection StepDefinition {0}
CannotBeConvertedToNodeset RegularExpressionProtection {0}: Result of xpath {1} cannot be converted to nodeset. Context {2}
XPathCompilationFailed RegularExpressionProtection {0}: Failed to compile xpath {1}. Context {2}
JSONPathCompilationFailed RegularExpressionProtection {0}: Failed to compile jsonpath {1}. Context {2}

Policy schema

Each policy type is defined by an XML schema (.xsd). For reference, policy schemas are available on GitHub.

Add new comment

Provide your email address if you wish to be contacted offline about your comment.
We will not display your email address as part of your comment.

We'd love your feedback and perspective! Please be as specific as possible.
Type the characters you see in this picture. (verify using audio)

Type the characters you see in the picture above; if you can't read them, submit the form and a new image will be generated. Not case sensitive.