Imagine a scenario, where you are downloading 500 MB text file that contains over 60 million logged events in your network, then putting it a compiler so that it is in a readable format when you upload it to your SQL table. You then begin searching for TCP connection that appears to be automated, lasting exactly two minutes, with roughly 5kb of data being sent from local hosts to outside IP addresses in the 50,000 port range. Did I mention, that a 500 MB file only represents a 15-second period in a 24-hour day that you trying to analyze for signs of a breach?

What I just described isn’t just a Network Analyst’s bad Monday morning – it’s a problem plaguing network security in general. 60 Minutes recently reported that, despite having network security appliances in place, Home Depot’s analysts missed vital security alarms. The fact that those events that could have exposed the hackers were buried under hundreds of millions of events, played major role in why it was missed. There is a very likely chance that malicious activity could be as little as two events – a connection request and connection established. Even worse, if it is going through a well-known protocol, stop and think for moment about a situation, where you have to examine the DNS and http traffic. Try to not to shrill when you think about how many DNS requests your network is going to get in the next 15 minutes, let alone after you have been notified that someone from the outside added themselves to your domain at minimum four months ago, but even that is tentative, because who would think to monitor the Active Directory server, or to record profile logons.

If the hardware problems of recording this amount of data weren’t complex enough, the other question is: how to distinguish between what is routine and what is security related? Understanding how your network operates is a key point – as an analyst, you have to know what you are looking for. When the security information and event management (SIEM) is getting 500 thousand events per hour, that you are looking at, you need to know what you are looking for. If you do not know why your network traffic is way it is, you can’t optimize, yet alone begin to develop a security model.

As an analyst, you tend to look at things in the context of data, not realizing that it is people generating the data. You have to take a stance where you are looking for the abnormal, the deviations from the pattern of data that doesn’t belong to a person. Questions such as the following are essential:

Why is the user generating SSH requests?

What has changed?

When did it start?

Has it stopped?

What tools are we missing to help identify this behavior sooner?

You cannot be static in your analysis. Network traffic can be described as the behavior of a network, and if you are going to be the person that prevents or finds the breach, you need to understand that your vision is not enough.