Regular expressions and the description field
What am I doing wrong. I'm looking for Event 4022 with a source of "MSExchange Availability" and the EventDescription matching a regular expression. I would like to use "CrossSite,\surl\s=\shttps://.*(\.atr...\.|\.md\.)domain\.com/EWS/Exchange\.asmx"

HOWEVER, I tried to simplify it down to ".*" and I still get an error logged in the operations manager log file with eventid 10303 which says "the microsoft Operations manager expression filter module failed to process a data item and dropped it." ERROR 0x80004005

you cannot get any simpler than ".*". I am generating the event log entry manually (.net code) to test. What is it with MOM/SCOM and regular expressions !!!!!!

I test the expression in a tool "expresso" which is excellent for testing. "Real" regular expressions NEVER seem to work with MOM/SCOM.

that being said... I tried looking for a literal string "contained" in the description and it works fine but I want to use the regular expression to be more flexible. I may end up having to use multiple OR'd contains statements but it really defeats having the option for regular expressions.

Does anyone have a Regular Expression tester that "thinks" the way the MOM and SCOM engines "think"?

sorry for grumbling... I fought through regExpressions with MOM 2005 expecting SCOM to work with standard regular expressions. Error 80004005 must be the classic "Syntax Error" of the 80's

Craig
November 19th, 2009 12:09am

That form of regular expression is not supported by that module. Please try not to cast a legal definition around the term "regular expression" as there are many different regular expression syntax that are perfectly fine. In this case, think xpath expressions.

There are some good examples of this on the MVP site as well as on authormps.com.

Thanks for not grumbling.

Free Windows Admin Tool Kit Click here and download it now
November 19th, 2009 4:38pm

sorry for the grumble. Xpath helps get the text you want to compare against and even use as part of the comparison. However I'm still left in a quandry since an Xpath is not a regular expression and I don't see why ".*" would not work. ".*" means any character 0 or more repititions. that should have worked but instead it logged an event (10303) in the event log "failed to process data item".

So with a regular expression against the EventDescription, does that mean I needed to specify an Xpath for the Event description and then the regular expression would have worked?

Alternatively, if regular expressions are not valid for that module, is there documentation on what comparisons are allowed for modules since "matches regular expression" is listed in the dropdown for comparison?

Craig
November 19th, 2009 7:06pm

Ok, this forum doesn't take long documents. Here comes a few parts of a document....

Regular expression support in SCOM 2007

Many teams that are authoring management packs may need to include regular expression matching in their discoveries and groups, as well as for pattern matching in expression criteria in monitors and rules.

There are two different types of regular expression support in the SCOM product, and you have to know which element you are working in to choose the correct one. Specifically, Group membership calculation and expression filters use distinctly different syntaxes for pattern matching.

Group Calculation matching criteria

Group calculation uses PERL regular expression syntax. By default, the matching is case insensitive, but in the XML you can specify that an expression needs to be case sensitive by way of a special attribute dedicated to specifying that the expression content should be evaluated in a case sensitive way.

Group Calculation is found in your MP whenever you are using the Group Calc module.

The GroupCalc expression has an operator called MatchesRegularExpression that is used to create dynamic group membership based on pattern matching expressions. The implementation of this operator passes the expression found in the MP XML to the SQL call name dbo.fn_MatchesRegularExpression. If this call returns 0, the match is false. If the expression returns 1, the match is true.

GroupCalc also supports two special sub-elements that abstract away a couple of common regex style queries.

GroupCalc sub element

Regex Equivalent

ContainsSubstring

^*{O}.*$ ({O} is replaced by the substring)

MatchesWildcard

MP expression

Regex Equivalent

?

.

*

.*

#

[0-9]

Table 1: GroupCalc special functions

Note: If either of these two special operators are used, the evaluation will always be case sensitive.

Expression Filter matching criteria

Expression filters used in management packs use .NET Regex expression syntax. A summary of the .NET regular expression syntax elements appears below. Expression filters are present in your management pack whenever you are using the Expression Eval module.

Construct

SCOM Regex

Any Character

.

Character in Range

[ ]

Character not in range

[^ ]

Beginning of Line

^

End of Line

$

Or

|

Group

( )

0 or 1 matches

?

0 or more matches

*

1 or more matches

+

Exactly N matches

{n}

Atleast N matches

{n, }

Atmost N matches

{ , n}

N to M Matches

{n, m}

New line character

\n

Tab character

\t

Regular expressions via SDK

The SCOM SDK has a Matches criteria operator for filtering objects. This operator use the same functionality as MatchesCriteria in the GroupCalc case explained above.

When using the SDK to construct a criteria expression to find objects in the Ops Manager database, the following syntax elements are valid (see below). This syntax is useful when creating a criteria expression that includes any of the following elements:

Comparison operators

Wildcard characters

DateTime values

Integer to XML Enumeration comparisons

Comparison operators

You can use comparison operators when constructing a criteria expression. The valid operators are described in the following table:

Operator

Description

Example(s)

=, ==

Evaluates to true if the left and right operand are equal.

Name = 'mymachine.mydomain.com'

!=, <>

Evaluates to true if the left and right operand are unequal.

Name != 'mymachine.mydomain.com'

>

Evaluates to true if the left operand is greater than the right operand.

Severity > 0

<

Evaluates to true if the left operand is less than the right operand.

Severity < 2

>=

Evaluates to true if the left operand is greater than or equal to the right operand.

Severity >= 1

<=

Evaluates to true if the left operand is less than or equal to the right operand.

Severity <= 3

LIKE

Evaluates to true if the left operand matches the pattern that is defined by the right operand. Use the characters in the wildcard table later in this topic to define the pattern.

Name 'LIKE SQL%'

Evaluates to true if the Name value is "SQLEngine."

Name LIKE '%SQL%'

Evaluates to true if the Name value is "MySQLEngine."

MATCHES

Evaluates to true if the left operand matches the regular expression defined by the right operand.

Name MATCHES 'SQL*05'

Evaluates to true if the Name value is "SQL2005."

IS NULL

Evaluates to true if the value of the left operand is null.

ConnectorId IS NULL

Evaluates to true if the ConnectorId property does not contain a value.

IS NOT NULL

Evaluates to true if the value of the left operand is not null.

ConnectorId IS NOT NULL

Evaluates to true if the ConnectorId property contains a value.

IN

Evaluates to true if the value of the left operand is in the list of values defined by the right operand.

Note

The IN operator is valid for use only with properties of type Guid.

Id IN ('080F192C-52D2-423D-8953-B3EC8C3CD001', '080F192C-53B2-403D-8753-B3EC8C3CD002')

Evaluates to true if the value of the Id property is one of the two globally unique identifiers provided in the expression.

AND

Evaluates to true if the left and right operands are both true.

Name = 'SQL%' AND Description LIKE 'MyData%'

OR

Evaluates to true if either the left or right operand is true.

Name = 'SQL%' OR Description LIKE 'MyData%'

NOT

Evaluates to true if the right operand is not true.

NOT (Name = 'IIS' OR Name = 'SQL')

Table 3: SDK comparison operators

Free Windows Admin Tool Kit Click here and download it now
November 20th, 2009 2:35am

Regular expression support in SCOM 2007 - Part 2 of 2

Wildcards

The following table defines the wildcard characters you can use to construct a pattern when using the LIKE operator:

Wildcard

Description

Example

%

A wildcard that matches any number of characters.

Name LIKE 'SQL%'

Evaluates to true if the Name value is "SQLEngine."

Name LIKE '%SQL%'

Evaluates to true if the Name value is "MySQLEngine."

_

A wildcard that matches a single character.

Name LIKE 'SQL200_'

Evaluates to true for the following Name values:

"SQL2000"

"SQL2005"

Note

The expression evaluates to false for "SQL200" because the symbol _ must match exactly one character in the Name value.

[]

A wildcard that matches any one character that is enclosed in the character set.

Note

Brackets are also used when qualifying references to MonitoringObject properties. For more information, see Defining Queries for Monitoring Objects.

Name LIKE 'SQL200[05]'

Evaluates to true for the following Name values:

"SQL2000"

"SQL2005"

The expression evaluates to false for

"SQL2003."

[^]

A wildcard that matches any one character that is not enclosed in the character set.

Name LIKE 'SQL200[^05]'

Evaluates to true for

"SQL2003."

The expression evaluates to false for

"SQL2000" and

"SQL2005."

Table 4: Wildcard operators used with LIKE operator

DateTime comparisons

When you use a DateTime value in a query expression, use the general DateTime format ("G") to convert the DateTime value to a string value. For example,

C#

string qStr = "TimeCreated <= '" + myInstant.ToString("G") + "'";

ManagementPackCriteria mpCriteria = new ManagementPackCriteria(qStr);

All date values need to be converted to the G format (GMT) so that valid string comparisons can be made.

Integer value comparison to enumerations

When you use an integer enumeration value in a query expression, cast the enumeration value to an integer. For example,

C#

string qStr = "Severity > " + (int)ManagementPackAlertSeverity.Warning;

MonitoringAlertCriteria alertCriteria = new MonitoringAlertCriteria(qStr);

November 20th, 2009 2:36am

No activity for 30 days, will mark this thread as answered now. Feel free to open it again.
Free Windows Admin Tool Kit Click here and download it now
December 18th, 2009 7:40am

This is an old thread but still an issue.  This is a possible solution.

https://support.microsoft.com/en-us/kb/3004791

This problem may occur if the length of the line for the match is longer than the Expression Filter can handle.

To resolve this problem, follow these steps:

    Create the following registry subkey:

    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\v3\Modules\Global\ExpressionFilter
    Underneath this subkey, create a DWORD value.
    Type the following name for the DWord value:

    MaxExpressionDepth
    Assign a data value that is between 500 and 100000. The default value is 2000.

May 26th, 2015 5:32pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics