Up
Previous Next

Sheriff CSM™

Event Collection, Processing, and Correlation Workflow

All Sheriff CSM's security monitoring and management capabilities stem from its overall ability to collect data from devices, transform the data into a common set of data fields that define events, and then process, filter, and correlate those events to identify potential threats and vulnerabilities, or real occurrences of attacks. Sheriff CSM also assesses the importance and priority of events by assigning risk values based on the value of the underlying assets, the source and nature of the identified threat, and the likelihood of successful attack. More detail on this overall workflow is provided in this section for the following topics:

• Log Data Collection, Parsing, and Normalization
• Event Processing and Filtering
• Event Correlation, Alarms, and Notification
• Event Visualization and Analysis

Log Data Collection, Parsing, and Normalization

Log collection is at the root of Sheriff security management. Sheriff CSM collects logs from various sources: network devices, such as firewalls and routers, host servers and systems, and software applications running on servers. Some devices, for example, those that support the Syslog protocol, are configured to send their logs directly to the Sheriff CSM Sensor (Deputy). For other devices, Sheriff CSM goes out and retrieves the logs. In both cases, data in the logs is normalized to extract and store information in common data fields that define an event: IP addresses, host names, user names, interface names, and so on. These are the events that a security analyst can analyze in Sheriff CSM to uncover threats and vulnerabilities, and assess an organization's risk.

Log Parsing Using Plugins

Running on a Sheriff CSM Deputy, a Sheriff CSM agent is configured with a collection of different log-parsing plugins, which define how to collect logs from specific devices, systems, or applications, and how to transform that log data into standardized event data fields before sending the events to the Sheriff CSM Server. The plugins also control other event-gathering functions on the Deputy, such as intrusion detection. Sheriff CSM comes equipped with plugins for many commonly encountered data sources. Contact Sheriff to request a new plugin for any data source or product for which a plugin does not already exist. You can also create your own custom plugins, or customize Sheriff CSM’s existing plugins.

Normalization of Security Events

No matter the format of a log message, certain pieces of data (such as user names or IP and MAC addresses) are common in all of the device logs. Extracting these values out of the log message text and storing them into matching common fields is called normalization. Normalization is what allows you to perform queries across events collected from varied sources (for example, “Show all events where the source IP is 192.168.1.3”.) Although the format of the original data collected from devices may be different, similar information across devices is stored in the same field for events sent to the Sheriff CSM Server.

The logs are broken down into their message type, and the information from them is used to populate a standard set of fields that define an event (for example, date, Deputy, plugin_id, priority, src_ip, src_port, dst_ ip, dst_port, username, userdata1).

Note: For a complete list of normalized event fields, see Review Event Details.

Event Processing and Filtering

After normalizing the data obtained from log files and other sources, the Sheriff CSM Deputy transmits security events to the Sheriff CSM Server. The Sheriff CSM Server also performs several additional operations on incoming events, including:

• Parsing the event priority and reliability — Each event type is assigned a priority, which indicates how urgently the event should be investigated, and a reliability score, which assesses the chance the event is a false positive.

• Checking asset values to calculate a risk score — The Sheriff CSM Server maintains an inventory of known devices on the network, with an associated asset value for each device, defining their importance to the organization. This asset value is then weighed against the event’s priority and reliability score to produce a risk value. Higher risk scores help analysts know what is most important to examine first.
  
  For more information on how Sheriff CSM calculates risk, see Sheriff CSM Network Security Concepts and Terminology.

• Application of the event taxonomy — There are system and network events common across many system types, no matter the source of the event or its original data format. Sheriff maintains a hierarchical categorization of event types (referred to as a taxonomy) to which Sheriff CSM can match events in policies and correlation directives.

• Cross-checking reputation data — The Sheriff CSM Server checks the IP addresses specific to each event against a reputation database of Internet addresses. IP addresses that match are flagged for future reference and follow-up.

After performing these operations, and based on specified user policy and filter conditions, the Sheriff CSM Server will save selected or qualified events in a SIEM events database for further analysis and correlation. The events database commonly resides on the same host as the Sheriff CSM Server, but in large deployments, the database can be installed on a separate host for increased performance and capacity.

Event Correlation, Alarms, and Notification

Following the basic processing, analysis, and filtering that the Sheriff CSM Server performs, selected or qualified events are fed into the Sheriff CSM correlation engine. Using Sheriff CSM correlation, analysts can look for patterns and sequences of events across multiple devices and system types. Events may actually be processed by the correlation engine several times, as different correlation rules may take the same events as input.

Correlation directives create alarms

As events continue to feed into the correlation engine, Sheriff CSM generates alarms based on event conditions specified in correlation directives or rules:

• Alarm processing starts when the conditions of a correlation directive are met.

• Alarms may trigger on a single event matching certain conditions, or may require a specific sequence of events to trigger.

• Alarm processing may continue over a matter of hours. Alarms that appear in the system may indicate they are still processing additional incoming events to further corroborate detection.

• Alarms are themselves events (directive events), that can feed into other correlation directives once they are triggered, so you can create cascading levels of alarms.

In addition, when you sign up for the Open Threat Exchange® (OTX™), Sheriff CSM is configured to receive raw “pulse” data and indicators of compromise (IoCs), from OTX. Sheriff CSM correlates that data and alerts you to any related OTX pulse and IP reputation-related security events and alarms when it detects those same IoCs interacting with assets in your environment.

As soon as you log into Sheriff CSM, you can see from the Sheriff CSM dashboard which OTX indicators are active in your environment. You will receive immediate notification in the form of an event or an alarm when a malicious IP address identified in OTX communicates with any of your system assets, or when Sheriff CSM identifies any other IoCs seen in OTX are active in your network.

Note: For more information about how Sheriff CSM alarms are processed and correlated, see Alarm Management.

Event Visualization and Analysis

Events obtained from device logs, as well as those generated by the correlation engine itself, can all be searched, viewed, and reported on from the Sheriff CSM web UI.

• View of security events with options to search, filter, and group events based on specific event field values. To use this option, select Analysis > Security Events (SIEM) from the web UI.

For more information on viewing events and performing other security management operations from the Sheriff CSM web UI, see Review Security Events.