Customize Existing Plugins Yourself

After enabling a plugin, the Sheriff CSM Sensor processes the plugin's log data and sends events it collects to the Sheriff CSM Server, where they are stored in the SIEM database. You can then view the events in the Sheriff CSM web UI by selecting the Analysis > Security Events (SIEM) option.

After using the plugin for a while, you may find a number of events that you don't find useful to display or track, and just create noise and take up space in the SIEM database. Rather than creating Sheriff CSM policies to filter out these events, which incurs some processing overhead, you can include the exclude_sids parameter in a .local copy of the plugin's configuration file to achieve the same result.

To exclude collection of specific event type IDs from a plugin

1. Connect to Sheriff CSM through SSH.

2. On the Sheriff Setup menu select Jailbreak System.

3. If the file does not already exist, create the file

   /etc/vigilante/agent/plugins/<plugin_name>.cfg.local.

4. In the <plugin_name>.cfg.local file, add the exclude_sids parameter in the [CONFIG] section and specify one or more event type IDs (separated by commas) that you don't want the Sensor to process. For example:

   [CONFIG]
   exclude_sids=200,302,404,403

5. Save the file and restart vigilante-agent:

   /etc/init.d/vigilante-agent restart

The SIDs you specify with the exclude_sids parameter are the Event Type IDs that Sheriff CSM assigns to each event that matches a specific rule in the plugin configuration file. The quickest way to locate the SID number of a specific event type is to go to the Analytics > Security Events (SIEM) page, and click on the row of a particular event you want to exclude. From the Event Detail display that is shown, you can locate the Event Type ID associated with the event. Use that value for the exclude_sids parameter in the plugin's configuration file.

Another way, in which you can view all the event type IDs associated with every plugin, is to go to the Configuration > Threat Intelligence page and select the Data Source tab. From there, each row of the display lists information about a plugin, and you can double-click on a specific row to view all the associated event type IDs for the selected plugin.

If you have a plugin that processes logs from an asset located in a different timezone than your sensor, sometimes due to the source device using an incorrect timezone or a timezone that cannot be changed, you can modify the timezone in the plugin file instead. The following task shows how to do this using the SSH plugin as an example.

To change the timezone used by a plugin

1. Connect to Sheriff CSM through SSH.

2. On the Sheriff Setup menu select Jailbreak System.

3. Check the default timezone that vigilante-agent uses.

   ~# grep tzone /etc/vigilante/agent/config.cfg
   tzone=Europe/Dublin

   This should match the timezone of where your Sheriff CSM Sensor resides.

4. Create the file /etc/vigilante/agent/plugins/ssh.cfg.local.

5. In ssh.cfg.local, add the following parameter:

   [DEFAULT]
   tzone=Australia/Melbourne

   where

   Australia/Melbourne represents the timezone of the asset you are collecting logs.

   Note: Sheriff CSM validates timezones against this list.

6. Save the file and then restart vigilante-agent.

   /etc/init.d/vigilante-agent restart

7. Check the agent log to make sure the new timezone has been applied:

   ~# grep -A1 "Starting detector" /var/log/sheriff/agent/agent.log
   2016-01-18 10:39:38,948 Sheriff-Agent [INFO]: Starting detector ssh (4003)..Plugin tzone: Australia/Melbourne
   2016-01-18 10:39:38,950 Sheriff-Agent [WARNING]: Using custom plugin tzone data: Australia/Melbourne

Different devices often use different date and time formats in their logs. Sheriff CSM solves this issue by providing a built-in function, normalize_date(), which converts different date formats to ISO 8601, the format accepted by the Sheriff CSM Server. For a list of formats that the function recognizes, see Supported Formats by the normalize_date() Function.

If the date and time format is not supported by the normalize_date() function, for example, DD/MM/YYYY HH:MM:SS, you can create a custom function to handle the specific date and time format that you need.

Note: Existing date formats are defined in the date_formats.json file. Modifying this file is NOT recommended, since Sheriff CSM updates will overwrite any changes you make to the file.

The following example provides a template for a custom function named normalize_date_not_american(), in which you can define the patterns of date and time you want to format.

#
# Description:
# This function should only be called when a date
# is in this format:
# 12/08/2016 21:44:47
# dd/mm/yyyy hh:mm:ss
#
# Usage:
# date={:normalize_date_not_american( $date_log )}
#
Start Function normalize_date_not_american
from re import compile
from datetime import datetime
def normalize_date_not_american( self, string = "" ):
pattern = compile( r'(?P<day>\d{1,2})/(?P<month>\d+)/
(?P<year>\d{4})\s(?P<hour>\d+):(?P<minute>\d+)
:(?P<second>\d+)' )
result = pattern.search(string)
groups = result.groupdict()
date = datetime(year=int(groups['year']),
month=int(groups['month']), day=int(groups['day']),
hour=int(groups['hour']), minute=int(groups['minute']),
second=int(groups['second'])).isoformat(' ')
return date
End Function

After creating and saving the custom function, you can use it in the plugin configuration file. For more information on using custom functions in plugins, see Define and Use Custom Functions.

In addition to the built-in functions that Sheriff CSM supports, you may also create your own custom functions to convert extracted data to the format required by specific event field types.

To create a new custom function

1. In the Config section, specify the location of a new file containing one or more new function definitions.

   [config]
   custom_functions_file=/etc/vigilante/agent/plugin/custom.cfg

2. Create a text file that matches the custom function's file name. In the new file, define the operation of each new custom function you want to create. Each new function must start with Start Function <function_name> and end with End function.

   The following example shows the definition of two new functions, log_hello and log_hello_data.

   Start Function log_hello
   def log_hello(self):
   return "Hello log!"
   End Function
    
   Start Function og_hello_data
   def log_hello_data(self,data):
   return "Hello log: %s" % data
   End Function

   In this example, the log_hello() function returns the string "Hello log!" and the value assigned to a normalized event field. The second function returns the "Hello log!" string concatenated with the extracted user name value.

3. Save and close the custom function file.

   Note: Be careful if you add a custom function to a plugin or if you access a proprietary database. This can degrade performance if not well designed. Custom plugins might take up to five minutes to appear in the Sheriff CSM web UI after you add them.

4. To use the new custom functions, you can simply include as part of any event field assignment corresponding to extracted data returned for a specific rule or log event. For example

   [ssh - Failed password]
   # Feb 8 10:09:06 server1 sshd[24472]: Failed password for dgil from
   192.168.6.69 port 33992 ssh2
   event_type=event
   regexp="(\w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d)\s+(?P<sensor>\S*).*ssh.*Failed
   password for
   (?P<user>\S+)\s+from\s+.*?(?P<src>\IPV4).*port\s+(?P<sport>\d{1,5})"
   plugin_sid=1
   Sensor={resolv($sensor)}
   date={normalize_date($1)}
   src_ip={$src}
   dst_ip={resolv($sensor)}
   src_port={$sport}
   username={$user}
   userdata1={:log_hello()}
   userdata2={:log_hello_data($user)}

Important: You are not allowed to use a custom function in a built-in function, for example, translate(:log_hello()), as custom functions are the last functions to be executed in a rule.

The following task shows how to change an existing rule and add a new rule to a plugin, in this case, for Cisco ASA.

To change or add a rule to an existing plugin

1. Connect to Sheriff CSM through SSH.

2. On the Sheriff Setup menu select Jailbreak System.

3. Create the file /etc/vigilante/agent/plugins/cisco-asa.cfg.local.

4. (As needed) Change any existing regex mappings or rules needed for the re-purposed plugin.

   The following example shows the changed rule, in bold, as the value of the regexp parameter. The commented section shows the mapping target of the regexp parameter.

   #Sep 17 06:25:31 5.5.5.5 : Sep 17 06:26:10 EDT: %ASA-ids-4-401004: #Shunned packet: 1.1.1.1 ==> 1.1.1.2 on interface inside
   [0001 - cisco-asa - Shunned packet]
   regexp="(?P<date>\w{3}\s+\d{1,2}\s(\d{4}\s)?\d\d:\d\d:\d\d)\s*:?((?P<device>\S+)\s*:?)?\s+[\w\:\s]*%?(?P<asa_short_msg>(asa|ASA)-[\w\-]*-?(\d+)-(?P<sid>\d+)):\s+(?P<userdata>[^:]+:\s+(?P<src_ip>\S+)\s+==>\s+(?P<dst_ip>\S+).*)"

5. (As needed) Add any new rule needed for the revised plugin, as shown in the following example. The commented section shows the mapping target of the regexp parameter.

   #Mar 28 12:03:04 testbox %ASA-7-609002: Teardown local-host inside:1.1.1.1 duration 0:02:12
   [9000 - cisco-asa - Teardown local-host]
   precheck="Teardown"
   regexp="(?P<date>\w{3}\s+\d{1,2}\s(\d{4}\s)?\d\d:\d\d:\d\d)\s*:?((?P<device>\S+)\s*:?)?\s+[\w\:\s]*%?(?P<asa_short_msg>(asa|ASA)-[\w\-]*-?(\d+)-(?P<sid>\d+)):\s+(?P<userdata>(?:Teardown\slocal-host\s)(?P<iface>[^:]*):(?P<src_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s\S+\s(?P<duration>\S+).*)"
   event_type=event
   date={normalize_date($date)}
   device={resolv($device)}
   plugin_sid={$sid}
   src_ip={$src_ip}
   userdata1={$asa_short_msg}
   userdata2={$iface}
   userdata3={$duration}
   userdata4={$userdata}

6. Save the file and then restart vigilante-agent:

   /etc/init.d/vigilante-agent restart

7. Check regular expressions and also field assignments within the file /var/log/cisco-asa.log.

   We recommend that you use any of the utilities available on the Internet to test that the Python regular expressions you added match the logs.

After creating a new rule, you will also need to define a new event type ID to assign to events matching the new rule.

However, if you have many new event types to add, you should use the plugin .sql file instead, because it is much faster. The web UI is better when you have only one or two event types to add.

Note: If you do not perform this task, these events will not appear in the Security Events view in the Sheriff CSM web UI.

To add new event types

This procedure adds a custom event type to an existing or newly developed plugin, using the Sheriff CSM web UI.

1. Go to Configuration > Threat Intelligence > Data Source.

2. Double-click a data source to open it.

3. Click Insert New Event Type.

   The Add New Event Type dialog box appears.

4. Fill in the fields, make value selections from the lists, and then click Update.

Add New Event Type fields

Field/List	Description
Name*	Event description. The text you introduce here refers to the event name, which appears in the Event Name column on the Security Events (SIEM) page.
Event type ID*	Refers to the plugin_sid. Note that you may not use a plugin_sid already in use. If you use the same ID twice, Sheriff CSM returns an error.
Category	Event type categories depend on the event type ID.
Subcategory	Select a subcategory for the new event type. For example, an event type ID might have subcategories such as Attack, Brute Force, or Policy.
Reliability*	Values range from 0 to 10, with 10 being the greatest reliability.
Priority*	Values range from 0 to 5, with 5 being the highest priority.

Values with an asterisk (*) are required.