Note: This is the first in a two-part blog series. This post covers the higher level strategy of the Adversary Resilience Methodology. In part two, we’ll put the methodology into practice and show you the nitty gritty technical details on how to do everything covered in this post.

Background and Intro

At DEFCON 24, Will Schroeder, Rohan Vazarkar and I released BloodHound. Since then, BloodHound has seen adoption from numerous pentest/red team firms and an awesome community of BloodHound users has congregated in the BloodHound Slack. The community adoption of it has been humbling. If you’re not familiar with BloodHound, check out the “Introducing BloodHound” blog post or watch our DEFCON presentation.

Early on, we quickly realized that while BloodHound is a great tool for attackers, the potential applications for defense are even more compelling. Many people in the community echoed that sentiment. The UI already provides several interesting features for defenders, including nested group membership visualization, outbound local admin privilege and object control statistics, and attack path discovery and analysis; however, these features barely scratch the surface of what is possible.

In this post, we’ll illustrate some of the current challenges enterprises face when trying to secure their Active Directory environment. Then, we’ll detail a new methodology that uses the untapped power of BloodHound’s attack graph to efficiently achieve an Active Directory that is highly resilient against the most attacks most likely to be discovered and executed by an adversary.

The (Necessarily) Reactive Nature of Enterprise Security

Over time, every Active Directory environment becomes an unwieldy, complex web of user rights, permissions, and behaviors. Adversaries (be they penetration testers, red teamers, or real attackers) commonly exploit that complexity, using accounts with permissions that defenders never meant to give them, or did not realize they had.

John Lambert has a fantastic blog post titled, “Defenders think in lists. Attackers think in graphs. As long as this is true, attackers win.” In his blog post, he advises the reader to learn how to recognize “list-based” thinking. The fatal flaw of list-based thinking, of course, is in not recognizing the connections between seemingly discrete systems. “My list of Domain Admin users,” says the list-based thinker, “only log onto this list of computers; therefore, if those computers are secure, so are the Domain Admins.” Use BloodHound, and you’ll quickly see the fallibility of the list-based mindset.

Many enterprises find themselves trapped in a seemingly inescapable loop of purely reactionary security remediation. Any organization that gets hacked, whether by a pentest firm that had permission or by a real attacker, will have the same question: how? Then either by reading the pentester’s report, or after performing a lengthy and thorough (read: expensive) investigation, the organization will obtain a breach narrative which describes the step-by-step actions the attackers took to complete their objective. A breach narrative may read like this:

A user was phished. That user had local admin rights on their own system. The attackers stole the built-in local admin password, which was the same on every system. The attackers found a system where a Domain Admin was logged on, then used Mimikatz to steal that Domain Admin’s credential from that system.

This breach narrative is then turned into a list of findings with associated recommendations for each finding. That list may look like this:

Now, there’s nothing wrong with any of these recommendations, and every organization should definitely be doing these things, but anyone who’s been doing enterprise security long enough knows none of these controls are ever perfect in practice:

End-user security awareness training: no matter how many times you repeat yourself, there will always be a user who wants to click on every link they see. Effectively auditing local admin group memberships is difficult for numerous reasons: insufficient local admin group analysis tooling, lack of alerting when new principals are added to a local admin group, and nested security groups obscure the true number of users with local admin rights to any given system. LAPS is an effective solution to shared local administrator account passwords, but in our experience as Red Teamers, there’s commonly a group of exempt hosts on most corporate networks. Credential Guard is effective at mitigating credential theft, but requires at least Windows 10, while most organizations are still running Windows 7, not to mention the hardware requirements for Credential Guard to be truly effective.

What’s more, by the time you’ve had a chance to deploy those controls and implement an audit methodology and plan, it’s probably time for your next regularly scheduled pentest (or a real attacker compromises you again). Assuming you deployed the above controls perfectly, the breach narrative this time may read like this:

A user’s password was guessed. The attackers connected to the corporate VPN, which doesn’t enforce multi-factor authentication. A high privilege service account was logged on to that system. The attackers injected into a process running as the service account. That service account had the “Add Members” privilege on the Domain Admins group.

Again, this narrative then gets turned into a list of findings:

You can probably see where this is going. Even in a perfect world, where all remediations were applied, only one attack path of many was stopped. The techniques and procedures used change, as vendors and defenders add security measures, but attackers continue to succeed. For many reasons, enterprises get stuck in this endless loop of responding to attacker behaviors and capabilities, hardly ever having time to try and get a leg up on adversaries. Visually, this cycle is straightforward:

Above: Enterprises commonly wind up in the endless loop of reacting to security incidents.

The Power (and Limits) of BloodHound

A typical Red Team report will include an attack path narrative, which outlines step-by-step how the team went from no access to their objective. Those attack paths almost always include lateral movement and privilege escalation through Active Directory. BloodHound’s attack graph tracks not just one, but all attack paths that rely on stealing credentials, explicit and delegated local admin privileges, and explicit and delegated securable object control.

Above: In this simple example, KSperling is an admin on Desktop21. On Desktop21 the AntiVirus user was logged on, and that user is a Domain Admin. As an attacker, we read this as, “Use the Ksperling user to pivot to Desktop21, then use Mimikatz to steal the AntiVirus user’s password.”

For example, we can see all of the shortest attack paths targeting the Domain Admins group:

Above: All shortest paths from any user to the Domain Admins group.

Zooming in on the tail end of the attack paths, which end at the Domain Admins group, gives us an idea of how an attacker could execute these paths:

Above: Detail showing the edge names in the attack paths.

Notice the green user icon one hop away from the Domain Admins group (with the giant red arrow pointing at it). There are 9 inbound edges targeting that user and indeed, many attack paths in the rendered graph traverse that node. By mousing over that user, all of the paths that traverse that node are highlighted in blue:

Above: Nearly all of the available shortest paths traverse the “USER99” node.

You couldn’t be blamed for thinking that getting rid of that user would eliminate a huge number of attack paths; however, we are only seeing the shortest paths to Domain Admin here, not all paths. Either way, the graph allows us to test how effective disabling or deleting that user would actually be. You can delete a node from the attack graph at the Neo4j console with this query:

MATCH (u:User {name:’USER99@CONTOSO.LOCAL’}))

DETACH DELETE u

After deleting the user, simply right click on the “Domain Admins” and hit “Shortest paths to here” to recalculate the shortest paths.

Above: The new shortest paths to Domain Admin, after deleting USER99.

As you can see, the attack paths have changed in some areas, but in fact, all of the users that previously had a path to DA still do — they just take one more step to get there. The above paths were always there, but not shown because they were not the shortest paths.

The fact is, any well-populated attack graph will have more paths than you could ever hope to eliminate with this whack-a-mole methodology. You can certainly attempt to query for and render all possible attack paths, but the query will likely never finish, and even if it did, you’d have an unreadable mess on your screen. Imagine, for instance, a helpdesk user get phished. If that helpdesk user has local admin rights on 5,000 systems, that’s potentially 5,000 starting points the attacker could pivot to on the way to a Domain Admin account. To illustrate this, we found all of the attack paths from one user in this graph to the Domain Admins group, with any length up to 5. This is the resulting image:

Above: All paths of maximum length 5 from one user to the Domain Admins group.

As you can see, the true complexity of the graph quickly outgrows the interface, and precludes any possibility of knocking out edges one by one to cut down on attack paths.

The Resilience Methodology

Recall earlier the description of the reactive enterprise security methodology. How long do you suppose it takes to go through one loop of that methodology? Months? Years?

Any good methodology needs to have very clearly defined goals. The goals of the Resilience Methodology are simple:

Find all attack paths an attacker is most likely to find and execute before an attacker can

Severely reduce the number of attack paths

Hinder adversaries by mitigating attack paths we just can’t get rid of

Maintain an Active Directory that is highly resilient against stolen credential and object control attacks

We can achieve these goals using a four-phase, looping methodology:

Above: The Active Directory Resilience Methodology

Phase 1: Enumerate Attack Paths

The first step is to collect the following pieces of information:

The local administrator group memberships across all domain-joined systems

User session locations

Security group memberships

Access Control Entries (ACEs) from securable AD objects including users, groups, and the domain objects.

Active Directory domain trusts

This step is made very easy thanks to SharpHound, the current BloodHound data collector written by Rohan Vazarkar, based on the original PowerShell collector written by Will Schroeder. You can download the latest version of SharpHound here. SharpHound is written in C# and is fully free and open source, so you can audit the code before running it in your network. Also check out Rohan’s great blog post on the technical details of SharpHound here.

For details on running SharpHound, see the “Phase 1: Enumerate Attack Paths” section of the companion blog post, to be released next Monday.

Phase 2: Analyze Attack Paths

The Resilience Methodology is all about reducing attack surface in Active Directory, and BloodHound allows you to analyze incoming attack paths targeting any node. If you need to protect a certain set of systems, you can do that. If you need to reduce inter-domain attack paths, you can do that too. Your analysis phase should be driven by what your resilience goal is. For the sake of demonstrating the methodology, we’ll choose a very common, universal goal that any organization would have: reduce attack paths that result in the compromise of a Domain Admin user account. First, we’ll collect some stats regarding the number of computers and users that have a path to Domain Admin. For details on how to generate these stats, see the “Phase 2: Analyze Attack Paths” section of the companion blog post, to be released next Monday.

Currently, the analysis phase is highly manual, and can require quite a bit of tedious inspection of group memberships, local admin rights, user sessions, and object control edges. In the future, we hope to automate a lot of this process, but for now, roll up your sleeves and get as comfortable as you can with the BloodHound UI and Cypher.

We can also measure the attack path possibilities across the enterprise in a single chart. For this, we can learn from the vulnerability management discipline, which has fairly mature visualizations to measure, for example, vulnerabilities on all systems across the enterprise. A highly simplified chart might look like this:

Above: A simple network of five computers, sorted by the number of high and critical vulnerabilities present on each system.

We can generate a very similar chart with BloodHound data, but instead of showing how many vulnerabilities each system has, we want to know how many systems can be compromised if an attacker lands on each system. For example, consider a network where all users have local admin rights, and the built-in local admin password is the same on all systems:

Above: In the same simple network of five computers, attack paths reaching all other four computers are possible from each system. The five computers below the horizontal axis are the computers in our network, the four computers above the horizontal axis are the number of systems that could be reached through an attack path starting at any of those five computers.

In our example database, the real chart currently looks like this:

Above: The attack path possibilities from all 12,000+ computers in the enterprise. Note the vertical axis is logarithmic in scale, so the chart is highly compressed at higher values.

Keep this chart in mind, as we will be coming back to it in the next section.

Phase 3: Generate Resilience Hypotheses

The insights you gained during the attack path analysis phase will inform your resilience hypotheses. In the example database, we observed that Domain Admins were logged on to many computers, relative to the total number of computers in the domain. This is a fairly common issue, and the typical suggestion from a pentest or red team may be to limit Domain Admins to PAWS (Privileged Access Workstations) and/or Domain Controller systems only.

We may formally express our hypothesis as:

If we take the following actions, then only 5% of users and 5% of computers should be able to reach a Domain Admin user: - Restrict Domain Admin accounts to only log on to Domain Controllers.

The resilience methodology lets us efficiently and accurately measure just how effective that recommendation will be before we move forward with implementing it in our real network. In Cypher, we can discover all of the systems where Domain Admins were logged on, where those systems were not Domain Controllers. For technical details, see “Phase 3: Generate Resilience Hypotheses” in the companion blog post, to be released next Monday.

Above: The graph of DAs logging onto non-domain controller systems.

The next step is to take all of those sessions and remove them from the graph. This will let us re-measure the attack path possibilities in the graph and generate new metrics. In Cypher, we can efficiently remove those edges by modifying the original query, this time to delete the offending edges. At this point you may want to create a copy of the graph database so that you can go back to the original copy if necessary.

For technical details, see “Phase 3: Generate Resilience Hypotheses” in the companion blog post, to be released next Monday.

Now we’ve simulated this change in our graph. Recall that earlier we generated the following chart, which showed the attack path possibilities from any system in the enterprise:

And now the same chart after we’ve removed the offending user session edges:

Above: The enterprise-wide attack path possibilities chart after removing the offending user session edges. The green area indicates attack paths that have been removed.

While we’ve made some progress, it may not be nearly enough as we’d hoped. In a real enterprise, there can be thousands of reasons why this change wasn’t nearly as effective as we’d expect. In the BloodHound example database, we can re-analyze this new graph and see what the new shortest paths to DA look like:

Above: The new shortest paths to DA, now that DAs no longer log on to non-Domain Controller systems.

Inspecting this graph closely, we can see that there are indeed attack paths to require pivoting to a Domain Controller system, and the edges enabling those paths traverse 15 security groups that all have local admin rights on those domain controllers:

Above: Several security groups have local admin rights on Domain Controllers.

Investigating these security groups’ local admin privileges on domain controllers closely, we can decide which of those security groups actually need local admin rights on DCs, and which do not. We can again simulate pruning those security groups’ local admin rights on DCs, so that only the required privileges remain. At this point, we can adjust our formal hypothesis statement to reflect that change:

If we take the following actions, then only 5% of users and 5% of computers should be able to reach a Domain Admin user: - Restrict Domain Admin accounts to only log on to Domain Controllers. - Remove unnecessary local admin privileges on Domain Controllers.

After simulating that change in the graph, we can again regenerate our domain admin exposure metrics, and regenerate our enterprise-wide attack path analysis chart:

This progress looks good in a table. How about a visualization of the attack paths analysis?

Above: The enterprise-wide attack path possibilities chart after removing the offending user session edges. The green area indicates attack paths that have been removed.

The huge bright green section indicates we’ve eliminated many thousands of attack paths possible from any system in the enterprise. The remaining blue area indicates attack paths are still possible, but we are inching closer and closer to the red dotted line target.

Phase 4: Deploy Prioritized Fixes

The final phase of the resilience methodology is to take the prioritized fixes you discovered to be effective during the hypotheses generation phase, and put them into action in your real environment. This phase will, of course, take the same amount of time as it normally would, but the key is in knowing how effective those changes are going to be before making them.

We believe that using the resilience methodology can help enterprises efficiently achieve and maintain an Active Directory that is highly resilient against modern abuse of privilege attacks; in short, identify all attack paths and eliminate as many as possible. Over time, and with the right analysis, we’ve seen a real environment go from this…

Above: The initial enterprise-wide attack path visualization. If an attacker lands on any system, there is an attack path available to them to every other computer in the network.

…to this, after only four cycles of the remediation methodology:

Above: A very small set of systems provide an attack path to DA, and the remaining attack paths in the network are known and mitigated with other controls.

Desired End State

The ultimate goal of the Active Directory Adversary Resilience Methodology is right there in its name: achieve and maintain an Active Directory that is highly resilient against intelligent adversaries. Critically, the resilience of Active Directory should be exhaustively understood, empirically measured, and efficiently improved. With the hard knowledge and insights that the methodology offers, organizations can reduce overall attack surface to a manageable problem set, and understand and mitigate the remaining attack surface. Further, knowing the remaining key terrain (i.e.: the systems that continue to provide the most risk) will enable organizations to confidently assign higher-cost monitoring resources, hunt operations, and non-scalable compensating controls to that subset of systems.

Future Work

There are several plans we have for the resilience methodology going forward. Most importantly is automating as much of Phase 3 (Attack Path Analysis) as possible. Currently this phase is highly manual and requires intimate familiarity with Cypher and Active Directory security best practices. Automating this phase will further enable enterprises to stand up their own internal resilience practices.

Additionally, we want to visually track changes to attack paths over time. Temporal awareness will enable enterprises to automatically be alerted if new attack paths appear due to some change in the environment, such as a user being added to a security group, or a high privilege user logging onto a machine.

Finally, we’ll be adding more node and edge types to BloodHound, including GPO and OU structures. We also hope to blog about how you can add your own custom nodes, edges, node properties, and edge properties, and run analytics that take those new data points into consideration when measuring attack paths.

If you have any questions about the resilience methodology, please don’t hesitate to contact me at andy@specterops.io, or join us in the BloodHound Slack, where the community of over 1,200 users is always happy to help: http://bloodhoundgang.herokuapp.com