Here is a quick example of how OSSEC[1] can be helpful to perform threat hunting. OSSEC is a free security monitoring tool/log management platform which has many features related to detecting malicious activity on a live system like the rootkit detection or syscheck modules. Here is an example of rules that can be deployed to track malicious processes running on a host (it can be seen as an extension of the existing rootkit detection features). What do I mean by malicious processes? Think about crypto miners. They are plenty of suspicious processes that can be extracted from malicious scripts (see my previous diary[2] about this topic).

OSSEC has a nice feature which allows monitoring the output of a system command. A basic rule coming in any freshly deployed OSSEC agent is the disk space monitoring. OSSEC performed a ‘df’ command at regular interval and searched for ’100%’ in the output:

<rule id="531" level="7" ignore="7200"> <if_sid>530</if_sid> <match>ossec: output: 'df -h': /dev/</match> <regex>100%</regex> <description>Partition usage reached 100% (disk space monitor).</description> <group>low_diskspace,</group> </rule>

The idea is to search for malicious running processes on a system using the same technique. In the case of trojaned systems, commands like /bin/ps could be replaced to hide some processes. A better approach is to use the /proc virtual filesystem to list the running processes. Here is the command that I use:

# find /proc -name comm -exec cat "{}" \; 2>/dev/null |sort -u

It searches for /proc/<pid>/comm files that expose the process's command name associated with the process. Example of generated output:

accounts-daemon acpi_thermal_pm apache2 arpwatch ata_sff atd bash charger_manager cpuhp/0 cpuhp/1 cron crypto dbus-daemon devfreq_wq ecryptfs-kthrea edac-poller ext4-rsv-conver find gdbus gmain ib-comp-wq …

Let’s define this command in OSSEC by adding an entry in $OSSEC_HOME/etc/ossec.conf:

<localfile> <log_format>full_command</log_format> <command>find /proc -name comm -exec cat "{}" \; 2>/dev/null |sort -u</command> <frequency>180</frequency> </localfile>

The ‘full_command’ type helps to return the output as a single line to be easily parsed later. Now, the create a rule in $OSSEC_HOME/rules/local_rules.xml:

<rule id="100405" level="7" ignore="7200"> <if_sid>530</if_sid> <match>ossec: output: 'find /proc</match> <regex>Duck.sh|accounts-daemon|bonn.sh|kworker34|minerd|minergate|minexmr|mixnerdx|myatd|polkitd|rootv2.sh|jaav|jva|kw.sh|kxjd|mule|mutex</regex> <description>Searching for suspicious processes</description> <group>hunting,</group> </rule>

The regex has been created from a list of processes found in a crypto miner installation script. Deploy the updated config files, restart the OSSEC processes. Now, let's create a fake suspicious process on a monitored host and wait for a few minutes. You should get the following alert:

OSSEC HIDS Notification. 2018 Sep 20 08:18:20 Received From: (shiva) 192.168.254.8->find /proc -name comm -exec cat "{}" \; 2>/dev/null |sort -u Rule: 100405 fired (level 7) -> "Searching for suspicious processes" Portion of the log(s): ossec: output: 'find /proc -name comm -exec cat "{}" \; 2>/dev/null |sort -u': (sd-pam) accounts-daemon acpi_thermal_pm apache2 arpwatch ata_sff atd bash charger_manager cpuhp/0 cpuhp/1 cron crypto dbus-daemon devfreq_wq ecryptfs-kthrea edac-poller ext4-rsv-conver find --END OF NOTIFICATION

It's time to investigate!

Note that this simple alert may generate a lot of false positives! Another approach could be to check the process name combined with its working directory because many crypto miners use common process names (ex: 'apache'). But 'apache' running from /tmp is definitively suspicious! Happy hunting!

If you want to learn more about how to use OSSEC for threat hunting, I'll do a training at DeepSec (Vienna, Austria) in November about this topic[3].

[1] https://www.ossec.net

[2] https://isc.sans.edu/forums/diary/Crypto+Mining+Is+More+Popular+Than+Ever/24050/

[3] https://deepsec.net/speaker.html#WSLOT378

Xavier Mertens (@xme)

Senior ISC Handler - Freelance Cyber Security Consultant

PGP Key