The XML schema is explained in the Microsoft MSDN documentation. The central configuration points are:

MaxItems. The maximum number of items that should be batched before being sent. It is recommended to use a lower MaxItems value for security-critical events such as log deletion.

The maximum number of items that should be batched before being sent. It is recommended to use a lower MaxItems value for security-critical events such as log deletion. MaxLatencyTime. The maximum time (in milliseconds) that can elapse before event logs are forwarded, provided the MaxItems value is not exceeded. This should be treated as the maximum latency you are willing to tolerate waiting for any given event.

The maximum time (in milliseconds) that can elapse before event logs are forwarded, provided the MaxItems value is not exceeded. This should be treated as the maximum latency you are willing to tolerate waiting for any given event. Query. The query is the heart of the subscription and defines which event IDs should be sent and which should be suppressed. XPath queries can be used to filtering even further, see below.

The query is the heart of the subscription and defines which event IDs should be sent and which should be suppressed. XPath queries can be used to filtering even further, see below. ContentFormat. Determines whether the entire text of the event entry is sent (larger, but more verbose) or just the binary data. For human readability and parsing, we use RenderedText for all subscriptions.

Determines whether the entire text of the event entry is sent (larger, but more verbose) or just the binary data. For human readability and parsing, we use for all subscriptions. LogFile. The log destination on the remote Event Log Collector. While Windows Event Forwarding will use the native ‘Forwarded Logs’ event channel, we will rely on custom log channels, see below.

As the subscriptions are XML files, they can be source-controlled in a Git repository. When combined with a continuous integration (CI) pipeline, WEF subscription modification, revision, collaboration, and deployment become reliable and repeatable. An example workflow could allow the InfoSec team to directly modify the WEF subscriptions to collect security-critical event logs, while allowing the Support and Operations team to file a pull request with a subscription to collect crash logs for their own usage.

Filtering Events with XPath

In order to maintain a high-fidelity event database or SIEM, it is important to filter down events to the subset relevant to an organization’s infrastructure. WEF supports XPath as a query language to implement such filters. The following is only a short primer on WEF XPath filtering, but hopefully will get you started with producing and testing your own custom filtering to make the most out of WEF.



XPath is essentially a method for selecting specific XML nodes from an XML document, and WEF supports a subset of XPath 1.0. The primary restriction is that only XML elements that represent events can be selected by an event selector. All valid selectors start with Event or * .



For simple queries that select events from a single source, using an XPath expression is fine. If the XPath expression is a compound expression that contains more than 20 expressions or you are querying for events from multiple sources, then you must use a structured XML query, see Consuming Events. For details on the elements of a structured XML query, see Query Schema. There are 4 main selectors:

QueryList. The root node of the query

The root node of the query Query (QueryListType). Defines the set of selectors and suppressors that are used to include and exclude events from the result set

Defines the set of selectors and suppressors that are used to include and exclude events from the result set Select. An XPath query that identifies what to include in the result set

An XPath query that identifies what to include in the result set Suppress. An XPath query that identifies what to exclude in the result set

The following shows a structured XML query that specifies a set of selectors and suppressors. This filter will grab all of the specified event IDs except those matching the SECURITY_LOCAL_SYSTEM_RID.

<Query><![CDATA[

<QueryList>

<Query Id="0" Path="Security">

<!-- 4624: An account was successfully logged on. -->

<!-- 4625: An account failed to log on. -->

<!-- 4634: An account was successfully logged off. -->

<!-- 4672: Special privileges assigned to a new logon, administrative logins -sa, -ada, etc. -->

<!-- 4775: An account could not be mapped for logon. -->

<!-- 4777: The domain controller failed to validate the credentials for an account. -->

<!-- Suppress SECURITY_LOCAL_SYSTEM_RID A special account used by the OS, noisy -->

<Select Path="Security">*[System[(EventID=4624 or EventID=4625 or EventID=4634 or EventID=4672 or EventID=4775 or EventID=4777)]]</Select>

<Suppress Path="Security">*[EventData[Data[1]="S-1-5-18"]]</Suppress>

</Query>

</QueryList>]]></Query>

You can use the Event Schema Elements as a starting point for your selector and suppressor queries. A few quick tips:

You can always use the Event Viewer UI to build a filter, then switch to the XML view and see how it’s represented in XPath.

If a selector and suppressor select the same event, the suppressor takes precedence.

WEF supports standard XML concepts such as <!— comments --> or CDATA literal text data.

Custom Windows Event Channels

As described in the blog post Creating Custom Windows Event Forwarding Logs, WEF can be extended with additional custom event channels. Extending the number of event channels available provides a few primary benefits:

Each event channel can have an independent maximum size and rotation strategy.

Each event channel can be used as a unique identifier for tagging data for ingestion into a SIEM.

Event channels may be placed on different disks or storage devices for improving disk I/O.

See below for instructions for defining and deploying custom event channels.

Deploying WEF

The following is an enumeration of installation and configuration steps for a typical WEF deployment.

Configure Auditing

In addition to the default Security, System, and Application logs generated by Windows, there are additional auditing settings available that are not enabled by default. These settings enable Windows to generate events that can be invaluable during the course of an investigation. For example, you can enable detailed process, registry, and file auditing among many others.



The most straightforward way to configure these settings is by creating Group Policy Objects (GPOs) for them. You might consider creating separate policies for your Domain Controllers, servers, and workstations depending on your log capacity and risk profile. Auditing recommendations can be found at Microsoft — Security Auditing.



It is highly recommended that you also account for PowerShell logging, and enable Script Block, Module, and Transcript logging.

Deploy GPOs

The easiest way to manage WEF at scale is to create a series of GPOs that will configure subcomponents of the pipeline. For instance, with a multi-site network, you may wish for the following:

Workstation Auditing GPO : Configures all of the Workstation-appropriate security and PowerShell logging settings, adjusts event log size and retention strategy, adjusts firewall rules, enables WinRM, etc.

: Configures all of the Workstation-appropriate security and PowerShell logging settings, adjusts event log size and retention strategy, adjusts firewall rules, enables WinRM, etc. Member Server Auditing GPO : Configures all of the Server-appropriate security and PowerShell logging settings, adjusts event log size and retention strategy, adjusts firewall rules, enables WinRM, etc.

: Configures all of the Server-appropriate security and PowerShell logging settings, adjusts event log size and retention strategy, adjusts firewall rules, enables WinRM, etc. Domain Controller Auditing GPO : Configures all of the DC-appropriate security and PowerShell logging settings, adjusts event log size and retention strategy, adjusts firewall rules, enables WinRM, etc.

: Configures all of the DC-appropriate security and PowerShell logging settings, adjusts event log size and retention strategy, adjusts firewall rules, enables WinRM, etc. Windows Event Collector GPO : Configures the WEC servers to have the WEC service automatically start, enables WinRM, adjust firewall rules, etc.

: Configures the WEC servers to have the WEC service automatically start, enables WinRM, adjust firewall rules, etc. US-WEST WEF Management GPO: Configures the target subscription manager setting to point to your West Coast WEC server.

Configures the target subscription manager setting to point to your West Coast WEC server. US-EAST WEF Management GPO: Configures the target subscription manager setting to point to your East Coast WEC server.

Examples of how to configure the GPOs can be found on the “Use Windows Event Forwarding to help with intrusion detection” post by Microsoft, or can be viewed in our GitHub Repository.

Define and Deploy Subscriptions

After configuring auditing settings across your fleet, the next step is to determine which of those logs you would like to collect and centralize via the WEF infrastructure. Palantir’s WEF library contains a curated series of subscriptions for you to adopt or modify to suit your needs, see below for a more detailed description.



Subscriptions can be defined and deployed from XML definitions or can be manually created in the Event Viewer GUI. To deploy descriptions from XML files, place them all in a single directory and navigate to that directory using PowerShell. The following script can be used to create and enable the Subscriptions that we provide in this repo.



NOTE: This script is specifically designed to work with the Subscription filenames we provide in our WEF library. You may have to modify it if your XML filenames don’t match your actual Subscription names.

Write-Output "Starting the Windows Event Collector Service"

Start-Service wecsvc

Set-Service wecsvc -StartupType Automatic Write-Output "Creating custom event subscriptions"

cd c:\folder-containing-xml-subscriptions\

cmd /c "for /r %i in (*.xml) do wecutil cs %i" Write-Output "Enabling custom event subscriptions"

cmd /c "for /r %i in (*.xml) do wecutil ss %~ni /e:true"

Configure Collectors

After you’ve defined the events you would like to collect via Subscriptions, it’s time to configure one or more servers to act as event collectors (also commonly referred to as Subscription Managers). This is also generally accomplished via a GPO, as described in the “Deploy GPOs” section above.



If configuring WEC servers by hand, it is important to start the “Windows Event Collector” service and to configure it to start at boot. You will additionally need to enable WinRM and allow inbound connectivity on TCP/5985 (Kerberos) or TCP/5986 (HTTPS).

Deploy Channels

Custom event channels are not a requirement in a WEF setup, but they do provide additional benefits outlined above in the “Windows Event Channels” section. Generally, this process involves creating a manifest file and building a resulting DLL from it, see Creating Custom Windows Event Forwarding Logs for details. Palantir’s WEF library provides a manifest and a pre-compiled DLL. The following steps to deploy the DLL have to be executed on each Subscription Manager:

Unload the existing manifest via wevtutil um C:\windows\system32\CustomEventChannels.man Copy your newly created CustomEventChannels.man and CustomEventChannels.dll files into c:\windows\system32 . Import the new manifest via wevtutil im C:\windows\system32\CustomEventChannels.man . This creates the defined channels and log files on the WEC servers. We also recommend increasing the size of each channel to 4GB:

$xml = wevtutil el | select-string -pattern “WEC”

foreach ($subscription in $xml) {

wevtutil sl $subscription /ms:4194304

}

Deploy PowerShell Transcript Logging

Once you have functional WEC servers, you should consider turning them into dual-purpose PowerShell transcript servers. As PowerShell transcription logging creates files over an SMB share, it will not use the native WEF pipeline.



We can take advantage of the existing infrastructure and simplify aggregation and collection through an SMB share, a security script, and additional GPOs. An overview of how to do so can be found in the blog post Microsoft ❤s the Blue Team.

Extend WEF

Once a WEF pipeline has been created, it can be leveraged for reporting custom events. With strong assurances that any given event will eventually be indexed in a SIEM, both security and productivity tools can be deployed and write to a unified log facility. Our WEF library contains two such extensions, Autoruns-To-WineventLog and EMETDiag, see below.

Shoot the Trouble

It can be frustrating to stand up logging infrastructure, only to discover that it’s not sending any of the logs you expected it to. Although sometimes unintuitive, there are a few key tools you can use to gain deeper insight into where a breakdown exists.



Start by reviewing the necessary components described in the “WEF Overview” section above. Ensure all of the required components exist in your environment and are configured correctly. If you’re in the testing phase, consider setting the Subscription Manager refresh interval to a small value such as 60 seconds. This will ensure that logs are offloaded from your clients in a timely basis and reduce the amount of time you need to wait for logs to arrive. If you ever need to force push logs to the Subscription manager, running gpupdate /force from the client will also force a check-in.



Additionally, information about errors or misconfigurations can be found in the Microsoft-Windows-Eventlog-ForwardingPlugin Event Log Channel on each of your clients. This event log is helpful for determining when ACLs are misconfigured on event logs, Subscriptions are somehow invalid, or when logging channels are missing from a host.



On a subscription manager, the Event Viewer tool can help you gain insight into the status of each subscription by clicking on the “Subscriptions” option in the left hand column, selecting a Subscription, and clicking “Runtime status”.

Palantir’s WEF Library

The following sections describe the configuration and tools provided in Palantir WEF library on GitHub: https://github.com/palantir/windows-event-forwarding/. It is important to note that the subscriptions and configurations provided will not solve all security use cases and may not run in all environments. It is highly recommended that the configurations are tested and tweaked for each organization. While we have done our best to find a good trade-off between signal and noise, we greatly welcome and encourage community participation in this project by filing a pull request or opening a GitHub issue.

Subscriptions

In addition to our own definitions, our subscription library contains subscriptions from IADGOV’s excellent Event Forwarding Guidance repository as well as Microsoft’s documentation. The exhaustive list of WEF subscriptions can be found in the GitHub project; they cover the following scenarios:

Account management, lockout, kerberos, and authentication events

Process execution and termination events

Powershell module, script block, and operational events

Advanced security events (e.g. autoruns, code integrity, EMET, etc.)

Operating system events (e.g. package installation, driver loading, DNS events, etc.)

Anti-forensic events (e.g. log deletion)

Note that our WEF subscriptions assume you will be using the provided event channels, see below. If you do not choose to do so, you will need to change the target log file in each subscription.

Event Channels

The Palantir WEF library contains a collection of custom Windows Event Channels, please refer to the README for a complete list. The channels are provided as a standard manifest file as well as a precompiled DLL. If you’re like us and don’t trust random DLLs, feel free to use our manifest file and build your own DLL.



The Event Channel manifest provided in this project consists of 16 individual providers, each with 7 channels. Channels follow a standard naming scheme of WEC[#], where the number is related to the provider.

Autoruns-To-WineventLog

Sysinternal’s Autoruns tool is used to search across different components of the Windows operating system to enumerate areas that are commonly used for persistence by malware and attack tools. While it’s commonly used on an ad hoc basis by forensic investigators, we wanted a steady stream of this information from our entire fleet. Having the ability to search through freshly reported persistence indicators is an invaluable dataset when hunting for badness.



To accomplish this, we wrote a PowerShell script that we call Autoruns-to-Wineventlog. This script downloads the latest version of Autoruns onto the host, sets up a scheduled task to kick off Autoruns on a daily basis, and then parses the resulting CSV data into the “Autoruns” Windows Event Log channel. The resulting data is then ingested into our SIEM via the WEF pipeline.



The code for Autoruns-to-Wineventlog serves as a representative sample of how easy it is to plug custom security tooling into a WEF pipeline.

EMETDiag

While officially sunset and slated to be replaced with Defender Exploit Guard, the Enhanced Mitigation Experience Toolset (EMET) provides an invaluable defense-in-depth layer against memory corruption and other exploit techniques. While configuring and using EMET is out of the scope of this article, there are instances where an application may not play well with the protection mechanisms offered, causing instability and crashes.



When troubleshooting EMET crashes internally, we rely on a custom PowerShell script called EMETDiag that can be remotely pushed via our systems management suite, or deployed by hand. Once deployed, it automatically queries the EMET configuration, pulls back the most recent EMET events, application crashes, and other related data, and then summarizes and writes the data to a custom event that is indexed by WEF.



Leveraging WEF allows for near-instant generation, forwarding, and indexing of data that can be used for troubleshooting purposes. Once indexed in the SIEM, it is available for immediate use by the Desktop Engineering team for analysis.



The code for EMETDiag is slated for release in the GitHub project in the near future.

Further Reading and Acknowledgements

Many open source publications were referenced for the development of our WEF library, and we wish to acknowledge those who have contributed to this effort: