A Guide to Generic Log Sources in Sigma

In a previous blog article I gave a short introduction to the idea of generic log sources in Sigma. The change is already available for testing in the project-1 branch of the Sigma repository and will be merged to the master branch within the next week and released on PyPI.

Mainly, the change consists of the following components:

  • Sigma converter sigmac now support multiple configurations that can be stacked.
  • Configurations can now rewrite the log source definition after conducting transformations.
  • Generic configurations for Sysmon and Windows auditing events for process creations were provided in tools/config/generic.
  • Most Windows process creation rules were converted into generic process creation rules and are now located in a process_creation directory instead of the sysmon, builtin or possibly other directories.
  • A conversion tool sigma2genericsigma was added, that converts Sysmon/1 or Security/4688 Sigma rules into generic rules.
  • Support for generic process creation rules in the Windows Defender ATP backend.

This blog post gives a practical guide to using generic Sigma rules.

Conversion of Sigma Rules

Until now, a typical rule conversion command line would looks as follows:

tools/sigmac -I -t es-qs -c tools/config/elk-winlogbeat.yml -r rules/windows/sysmon

The parameters explained:

  • With -I the converter continues on errors. This is useful to convert as many rules as possible, even when the baclend doesn't supports all features of Sigma, like aggregations in simple Elasticsearch queries.
  • -t es-qs selects the ElasticSearch query string backend. Use -l to get a list of all supported backends.
  • -c tools/config/elk-winlogbeat.yml uses the given configuration file (additional conditions for particular log sources, field mappings) for the conversion.
  • -r traverses recursively into the directory that is given as last parameter.

This converts all Sysmon rules into Elasticsearch query strings. With generic Sigma rules, the command line changes to:

tools/sigmac -I -t es-qs -c tools/config/generic/sysmon.yml -c tools/config/elk-winlogbeat.yml -r rules/windows/process_creation

This example converts the generic process creation rules into Elasticsearch query strings for Sysmon EventID 1 events. Most notable, two configurations were provided. The first one, sysmon.yml, looks as follows:

logsources:
    process_creation:
        category: process_creation
        product: windows
        conditions:
            EventID: 1
        rewrite:
            product: windows
            service: sysmon

Let's go through it. First, it matches on all rules with the logsource product set to windows and category to process_creation. If this is the case for a rule, the following transformations are applied:

  1. A condition that matches EventID to 1 is added to the generated query
  2. The log source is changed to the product windows and service sysmon for further processing by following configurations.

The second configuration provided to the command line is exactly the same as in the first example and does the remaining transformations required for a valid query like field mappings, additional log source conditions etc. As you can imagine, the order matters. The conversion result of the generic Sigma rules above would be wrong or not optimal if the configurations would be swapped.

For better distinction, we call the first configuration log source configuration (e.g. Sysmon) and the second one target system configuration (e.g. ELK, Splunk). Technically, there is nothing special about the configurations. The whole configuration could also be consolidated into one configuration file or three or more configurations may be provided. The transformations contained in the configurations are executed in the given order.

Exchange the Sysmon log source configuration from the example above with the one for Windows Audit events in the same directory to convert the generic process creation Sigma rules into Windows Security EventID 4688 events:

tools/sigmac -I -t es-qs -c tools/config/generic/windows-audit.yml -c tools/config/elk-winlogbeat.yml -r rules/windows/process_creation

A special case is the Windows Defender ATP (wdatp) backend, which does the transformations without configurations. There the command line would be as usual:

tools/sigmac -I -t wdatp -r rules/windows/process_creation

Conversion of Open Source Rules

All process creation rules from the Sigma GitHub repository were already converted into generic process creation rules which are contained in the rules/windows/process_creation directory. In the future, process creation rules will be written directly as generic rules.

Conversion of Private Rule Repositories

If you have an internal or private repository of Sigma rules and want to convert them into generic rules, you can use the new tool sigma2genericsigma that is available in the tools directory of the GitHub repository or in the sigmatools PyPI release.

A single Sigma rule can be converted simply by running the following command:

tools/sigma2genericsigma sigma-rule.yml

Bulk conversion can be done with:

tools/sigma2genericsigma -r rules -o generic_rules -c converted.txt

This converts all process creation rules from the directory rules into generic Sigma rules and puts the resulting files into the directory generic_rules. The rule file names that were converted are listed in converted.txt. Rules that mix events eligible for conversion with other events aren't converted and warnings are printed to the console. The conversion tool tries to reduce the Sigma rules as far as possible. One common case are Sigma rule collections for Sysmon/1 and Security/4688 events that result in the same rules after conversion. Duplicate rules are removed after conversion. Another common case is a rule collection with a global and another rule that inherits from the global. Such collections are merged into one Sigma rule.

All rules converted by the script should be reviewed manually. Common cases that can be optimized are:

  • Multiple rules of a collection that are converted into semantically equivalent rules but differ in attributes, e.g. descriptions in log sources.
  • Rules that only differ in field names, e.g. CommandLine from Sysmon and ProcessCommandLine from Windows Security 4688 events.

Future Development

In the future, further generic rule types may be added if this makes sense. Process creation events have by far the largest share in the open source repository, followed by registry manipulation events, which may be the next generic rule type.