Domain Specific Language Reference

In Continuous Detection and Response (CDR), you can create your own fine-grained custom rules by creating search queries with the available Cloud Email Protection domain specific language (DSL). These search queries are to find and match messages with the characteristics you define with the DSL. This topic defines the components of the DSL available in Fortra Cloud Email Protection and provides some examples of queries using the DSL.

You enter search queries in the Rule Description field on the Local CDR Rule page in Cloud Email Protection See Create a Continuous Detection and Response Rule for more information.

Syntax

A search query can contain one or more search expressions of field operator operand, with each additional search expression delineated by an AND/OR conjunction.

CDR does not support multiple uses of the same field/operator combination within an expression if they are joined by AND, but only if they are joined by OR.

For example, the following search_dsl is invalid:

from != "foo" and from != "bar"

Trying to preview a CDR Rule with this search_dsl would generate the error message:

The field 'from' cannot be used multiple times within the same statement.

This is true for any combination of a field + operator. The operators = and EQ are considered to be the same, as are!= and NOT EQ .

In the case of a field that supports list operators, using AND within an expression generates a different error message. For example, look at this search_dsl expression:

domain_tags != 'internal' and domain_tags != 'marketing'

This would give the error message:

The field 'domain_tags' supports list operators. Use IN or NOT IN when specifying multiple values.

To make this search_dsl valid you can rewrite it:

domain_tags NOT IN ['internal', 'marketing']

As noted above, using a field/operator combination multiple times is valid if each use is separate by OR. For example, these are valid:

domain_tags = 'internal' OR domain_tags = 'marketing'

 

(subject = "foobar" AND domain_tags != 'internal') OR (subject = "barbaz" AND domain_tags != 'marketing')

The default order of precedence is left to right for evaluating multiple search expressions in a query. You can group search expressions in parentheses to control precedence of evaluation. Wildcards are not allowed.

You should use double quotes around all string operands for clarity and consistency. Technically quotes are not required for operands that do not contain spaces or reserved characters, such as the operator characters = and >, but to avoid the possibility of errors always use them to enclose string operands. You must not use quotes around a number or date operand. Only double quotes are allowed, not single quotes.

Fields

A field represents a message characteristic that you can search for. You can use the following fields as the field component of a search expression. The Type indicates which operands you can use. For example, a field of type float can be used with any operand that can evaluate numbers.

NOTE: Fields below described as Exact matches only do not support the any operator, but do support the = operator.

Fields below described as Exact matches only do not support the any operator, but do support the = operator.

Field Type Description

 

attachment_extension

 

string

 

An exact match of an attachment extension as a string. For example, doc will not match docx. Exact matches only.

 

NOTE: Do not include the "." (period, or dot) that typically precedes extensions in filenames.

 

 

attachment_filename

 

 

string

 

An exact match of an attachment filename. Exact matches only.

 

attachment_sha256

 

 

string

 

An exact match of an attachment SHA256 hash. Exact matches only.

 

attack_types

 

string

 

An attack type classification as either a string or a list. The following table shows the values you can use in a search expression with the attack_types field and the associated attack type:

 

Search Expression Field Attack Type
low_trust Spam or Graymail
spoof Domain Spoof
dni Brand Display Name Imposter
address_group Individual Display Name Imposter
compromised_webmail Compromised Account
likely_malicious_uri Likely Malicious URI
malicious_attachments Likely Malicious Attachment
covid_19 COVID-19

 

authenticity

 

float

 

Authenticity score represented as a floating point value between 0.0 and 10.0.

Contains Active Content string True or False indicates whether CEP detected executable code in the body or attachment(s) of the message.

 

dkim_result

 

string

 

List of possible message DKIM results. The following table shows the values you can use in a search expression with the dkim_result field and the associated meaning for each (note that the values are all single characters):

 

Search Expression Value DKIM Result
+ DKIM pass
n Not signed

 

 

dmarc_result

 

string

 

List of possible message DMARC results. The following table shows the values you can use in a search expression with the dmarc_result field and the associated meaning for each (note that the values are all single characters):

 

Search Expression Value DMARC Result
+ DMARC pass
- DMARC fail
n Not available

 

 

domain_reputation

 

 

float

 

Reputation of sender domain, represented as a floating point value between 0.0 and 10.0.

 

domain_tags

 

string

 

An exact match of a domain tag, as either a string or a list.

 

 

from

 

string

 

The full From: in a message header.

 

 

from_domain

 

string

 

The domain portion within the full From: message header only.

 

 

from_to_match

 

boolean

 

Whether or not the From email address portion of the full header matches the To email address.

 

 

has_attachment

 

boolean

 

Whether or not message has an attachment.

 

has_malicious_attachment

 

 

boolean

Whether or not a message has an attachment that has been determined to be malicious.

 

ip

 

string

 

IP address or CIDR (see https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing for more information about CIDRs).

 

 

mail_from_domain

 

string

 

Mail from domain. The domain of the envelope sender. Matches only the entire mail from domain. (No partial matches.)

 

 

message_id

 

string

 

Unique message identifier. Matches only the entire message identifier. (No partial matches.)

 

message_trust_score

 

 

float

Message trust score, represented as a floating point value between 0.0 and 10.0.

 

reply_to

 

string

 

The full Reply-To in a message header.

 

 

reply_to_no_match

 

boolean

 

Whether the Reply-To address does not match the From address. (Use true in the operand to find when they do not match, and use false in the operand to find when they do match.) Exact matches only.

 

 

IP Reputation

 

float

IP Reputation score, represented as a floating point value between -10.0 and 10.0.

 

spf_result

 

string

 

List of possible message SPF results. The following table shows the values you can use in a search expression with the spf_result field and the associated meaning for each (note that the values are all single characters):

 

Search Expression Value SPF Result
+ SPF pass
- SPF fail
~ SPF soft fail
n No SPF record
p Permanent Error

 

 

start_date

 

date

 

A date in ISO 8601 format that defines the earliest message time and date to match (not applicable for matching incoming messages).

 

This cannot be the only expression in a rule, and it can include the operator :

=

Example:

 

Add the following condition to limit your search to messages after July 14th 2020 at 14:44 and 25 seconds UTC:

 

start_date = 2020-07-14T14:44:25Z

 

 

 

subject

 

string

 

Message subject. For example, if you enter a search expression of subject ANY "login" it would match the subject "Change your login password." If you enter a search expression of subject eq "login" it would be an exact match on the subject "login"."

 

 

to

 

string

 

The full To in a message header.

 

 

uri

 

string

 

A URL anywhere in a message. Exact matches only.

 

Operators and Operands

An operator defines what you can look for in a search expression and how you can look for it. Each operator in a search expression has specific legal operand types. Operators are divided into two types:

  • Numeric: For numeric comparisons.
  • String. For finding and matching text.

The following tables list the available operators you can use in search expressions.

Numeric Operators

Operator Description
= field is equal to all or part of the operand value
!= field is not equal to all or part of the operand value
>= field is greater than or equal to the operand value
> field is greater than the operand value
< field is less than the operand value
<= field is less than or equal to the operand value

String Operators

Operator Description
in field is in the operand list, where the operand list is one or more comma-separated values, enclosed in brackets
not in field is not in the operand list, where the operand list is one or more comma-separated values, enclosed in brackets
any any part of the field matches the entire operand
not any no part of the field matches the entire operand
eq, = field is equal to all or part of the operand value
not eq, != field is not equal to all or part of the operand value

An operand defines the value(s) you're looking in the field.

Types

Types refer to data types of the values that you can use in search expressions operands.

Type Description
string

One or more characters. In search expressions, strings must be enclosed in quote marks.

Strings that contains numeric characters are not the same as numeric values.

float A number that includes a decimal point.
integer A number that does not include a decimal point.
date A number in ISO 8601 date format. Note that this format contains non-mumeric characters, such as hyphens, but is still interpreted as a numeric type. Note: Do not put dates in quotes.
boolean A binary value that is represented by true or false. Note that these are not considered strings and do not get enclosed in quote marks.
list

A complex type that allows you to combine multiple strings in a search expression operand using the following format: [”item 1”, ”item 2”].

Conjunctions

Conjunctions connect simple search expressions to create complex search expressions.

Conjunction Description
AND, & Both must be true
OR, | Either may be true

Errors

Cloud Email Protection checks for syntax errors in the search expressions before it tries to process them. There are also other errors that could occur. This section lists possible errors and describes what to look for in your search expression to resolve the error.

There was an error in your DSL syntax. Premature end of input near [snippet].

 

This means something was not closed, such a parenthesis or quote marks. Any grouping marks must be used in pairs, and this error signifies that the second instance of a grouping pair was not found. [snippet] is a section of the search expression as close as possible to the error.

 

There was an error in your DSL syntax. Unexpected input near [snippet].

 

This means an incorrect operand, a mismatched operand, or an extra closing parenthesis. "Incorrect" could mean a mis-typed operand, so check carefully for spelling errors using the information in this topic. [snippet] is a section of the search expression as close as possible to the error. A mismatched operand would be something such as to >= username@gmail.com because the to field takes a string value, which cannot be evaluated as more than or equal to.

 

There was an error in your DSL syntax. Don't know what to do with [snippet].

 

This is the generic error message if an error is found and cannot be definitively determined to be one of the above errors. One reason this could occur is if you do not use the field operator operand pattern in your search expression. This could also occur for mis-typed fields or operators.

Wildcards not allowed

 

This means a wildcard character was found in the operand. If you want to include the characters * and ? in search expressions as string literals they must be escaped with ^, for example: ^* and ^?

 

OR statements not allowed in parentheses. Please rewrite your query.

 

Parentheses cannot enclose multiple-term search expressions that contain an OR conjunction.

Incorrect:

from_domain = foo.com and (subject = "x" OR subject = "y")

 

Correct:

(from_domain = foo.com and subject= "x") OR (from_domain = foo.com and subject = "y")

 

 

Organizations with MS Graph API enforcement can now create custom CDR rules that cannot exceed 10000 existing messages. Also, Organizations with Sensor enforcement cannot create custom CDR rules that exceed 2000 existing messages. Please refine your search criteria to return a smaller set of messages.

 

The search expression returned more than 10000 results for MS Graph API or more than 2000 with sensor enforcement . Add additional field operator operand constructions to limit the total number of search results. See the examples below.

 

NOTE: It is not necessarily an error when a search expression returns zero results. While these errors catch syntax errors and system limitations, it is entirely possible that either other errors in the search expression, such as a mistyped email address in an operand when you're using a to or from field, or there are no matches currently in the data store.
If you expect to see results and do not, double check spelling, date formats, and other possibilities, such as similar characters in your operands. For example, I (upper case i) vs l (lower case L) or O (upper case o) vs. 0 (zero). You also might expect zero results when you first design a search expression when you create a local rule to catch message threats that you know exist but have not yet been seen entering your organization.

 

[field name] is not a supported field.

 

This means that a field was entered as part of a search expression that is not one supported for CDR rules. Only the fields listed above can be used in CDR search expressions.

 

Lists are not a supported value for field: [field name].

 

In the search expression that uses the field, lists of values are not allowed as an operand. The search expression can have only a single value.

As an alternative, you can use multiple search expressions connected with an OR conjunction. So instead of this:

field = ["value1", "value2", "value3"]

You can rewrite the above like this:

field = "value1" OR field = "value2" OR field = "value3"

 

The value of [field name] must be numeric.

 

Only a numeric value is valid as an operand in search expressions with this field. Make sure that the operand is a number and that it is not enclosed in quotes. Values enclosed in quotes are considered strings.

 

The value of [field name] must be a string or valid list.

 

Only a string or a list of strings is valid as an operand in search expressions with this field. Make sure that any search expression with this field does not contain a numeric or boolean value and that the operand is enclosed in quotes.

 

The value of [field name] must be true or false.

 

Only a boolean value is valid as an operand in search expressions with this field. Make sure than any search expression with this field contains an operand value of either true or false only, and that the value is not enclosed in quotes.

 

The value of [field name] cannot be an empty string.

 

An empty string, or "", is not allowed in CDR rule search expressions.

 

search_dsl must contain at least one valid field for ongoing monitoring.

 

This is an error that something unexpected occurred after initially validating and saving the CDR rule. Contact Fortra Cloud Email Protection support for assistance.

Examples

This section provides examples of search expressions.

Internal Domains

You want to identify domains that are internal, that is, domains that you own, have control over, and trust. These are domains that you would have given the internal tag to. (Note that this is a search expression that can also be accomplished in message search. Anything that can be done with message search can also be done with DSL search expressions, although the latter can also do much more. This example demonstrates a very simple search expression.)

domain_tags = "internal"

This search expression will match any message that contains a domain that has been given the internal tag.

IP Address and Evaluation Results

You want to identify messages that include specific IP addresses.

(ip = "1.2.3.4") OR (ip = "2.3.4.5")

This search expressions will match any messages that contain either the IP address 1.2.3.4 or the IP address 2.3.4.5 and also contain an IP Reputation value.

User Receiving Attachments

You're concerned about attachments in messages sent to a specific user.

to = "jsmith@mycompany.com" AND has_attachment = "true"

This search expression will match any message sent to jsmith@mycompany.com that has an attachment of any kind.

You may also want to find only specific types of attachments.

(to="jsmith@mycompany.com" & has_attachment="true"& attachment_extensions="doc") OR

(to="jsmith@mycompany.com" & has_attachment="true"& attachment_extensions="docx") OR

(to="jsmith@mycompany.com" & has_attachment="true" & attachment_extensions="rtf")

This search expression will match any message sent to jsmith@mycompany.com that has an attachment and that attachment has an extension of doc, docx, or rtf, which are typical extensions of Microsoft Word documents.

Specific Terms With Low Reputation

You're concerned with messages you see that seem to be part of a specific campaign that you want to specifically block.

 

(subject="important" AND domain_reputation <= 2 AND domain_tags NOT IN ["service", "customer"]) OR

(subject="urgent" AND domain_reputation <= 2 AND domain_tags NOT IN ["service", "customer"])

This search expression will match any messages with "urgent" or "important" in the subject line (note that this is not case-sensitive) that also has a low domain reputation and does not come from a domain that you have tagged as one of your trusted types.