Introduction

A few months ago, while I was testing a custom APT that I developed for attack simulations in an enterprise Windows environment that allows access to Internet via proxy, I found that the HTTPs staged Meterpreter payload was behaving unexpectedly. To say the truth, at the beginning I was not sure if it was a problem related to the Meterpreter injection in memory that my APT was doing or something else. As the APT had to be prepared to deal with proxy environments, then I had to find out the exact problem and fix it.

After doing a thorough analysis of the situation, I found out that the Meterpreter payload I was using (windows/meterpreter/reverse_https, Framework version: 4.12.40-dev) may not be working properly.

Before starting with the technical details, I will provide data about the testing environment:

Victim OS / IP: Windows 8.1 x64 Enterprise /10.x.x.189

Internet access: via an authenticated proxy (“Automatically detect settings” IE configuration / DHCP option).

Proxy socket: 10.x.x.20:8080

External IP via proxy: 190.x.x.x

Attacker machine: 190.y.y.y

Meterpreter payload: windows/meterpreter/reverse_https

Note: as a reminder, the reverse_https payload is a staged one. That is, the first code that is executed in the victim machine will download and inject in memory (via reflective injection) the actual Meterpreter DLL code (metsrv.x86.dll or metsrv.x64.dll).

The following screenshot shows the external IP of the victim machine:

The following screenshot shows the proxy configuration (Automatically detect settings) of the victim machine:

The following screenshot shows the use of “autoprox.exe” on the victim machine. Observe that a proxy configuration was obtained via DHCP (option 252):

In the above image it can be observed that, for “www.google.com", the proxy 10.x.x.20:8080 has to be used. This can also be learnt by manually downloading and inspecting the rules contained in the wpad.dat file (its location was provided via the option 252 of DHCP).

Note: according to my analysis, autoprox.exe (by pierrelc@microsoft.com) will use the Windows API to search first for the proxy settings received via DHCP and then if it fails, it will search for proxy settings that can be obtained via DNS.

Analysis

During the analysis of the problem, I will be changing a few lines of code of the Meterpreter payload and testing it in the victim machine, therefore it is required to create a backdoored binary with a HTTPS reverse meterpreter staged payload (windows/meterpreter/reverse_https) or use a web delivery module. Whatever you want to use, is ok.

Note: a simple backdoored binary can be created with Shellter and any trusted binary such as putty.exe, otherwise, use the Metasploit web delivery payload with Powershell. Remember that we will be modifying the stage payload and not the stager, therefore you just need to create one backdoored binary for all the experiment.

Let’s execute the backdoored binary on the victim machine and observe what happens in the Metasploit listener that is running on the attacker machine.

The following screenshot shows that the MSF handler is running on the victim machines (PORT 443), and then a connection is established with the victim machine (SRC PORT 18904):

In the above image, it can be observed that the victim machine is reaching the handler and we are supposedly getting a Meterpreter shell. However, it was impossible to get a valid answer for any command I introduced and then the session is closed.

From a high level perspective, when the stager payload (a small piece of code) is executed on the victim machine, it connects back to the listener to download a bigger piece of code (the stage Meterpreter payload), injects it in memory and give the control to it. The loaded Meterpreter payload will connect again with the listener allowing the interaction with the affected system.

From what we can see so far, the stager was successfully executed and was able to reach the listener through the proxy. However, when the stage payload was injected (if it worked), something is going wrong and it dies.

Note: in case you are wondering, the AV was verified and no detection was observed. Also, in case the network administrator decided to spy the HTTPs content, I manually created a PEM certificate, configured the listener to make use of it and then compared the fingerprint of the just created certificate against the fingerprint observed with the browser when the Metasploit listener was visited manually to make sure the certificate was not being replaced in transit. This motivated me to continue looking for the problem in other place.

The next, perhaps obvious, step would be to sniff the traffic from the victim machine to understand more about what is happening (from a blackbox perspective).

The following screenshot shows the traffic captured with Wireshark on the victim machine:

In the above image it can be observed a TCP connection between the victim machine (10.x.x.189) and the the proxy server (10.x.x.20:8080), where a CONNECT method is sent (first packet) from the victim asking for a secure communication (SSL/TLS) with the attacker machine (190.x.x.x:443). In addition, observe the NTLM authentication used in the request (NTLMSSP_AUTH) and the response from the proxy server is a “Connection established” (HTTP/1.1 200). After that, an SSL/TLS handshake took place.

It worth mentioning that the above image shows the traffic sent and received during the first part, that is, when the stager payload was executed. After the connection is established, a classic SSL/TLS handshake is performed between the two ends (the client and the server), and then, within the encrypted channel, the stage payload will be transferred from the attacker machine to the victim.

Now that we confirmed that the first part (staging) of the Meterpreter “deployment” was working, what follows is to understand what is happening with the second part, that is, the communication between the stage payload and the listener. In order to do that, we just need to continue analyzing the traffic captured with Wireshark.

The following screenshot shows what would be the last part of the communication between the stager payload and the listener, and then an attempt of reaching the attacker machine directly from the victim (without using the proxy):

In the first 5 packets of the above image, we can see the TCP connection termination phase (FIN,ACK; ACK; FIN,ACK; ACK) between the victim machine (10.x.x.189) and the proxy server (10.x.x.20). Then, it can be observed that the 6th packet contains a TCP SYN flag (to initiate a TCP handshake) sent from the victim machine to the attacker machine directly, that is, without using the proxy server as intermediary. Finally, observe the 7th packet received by the victim machine from the gateway indicating the destination (attacker machine) is not directly reachable from this network (remember I told you that it was required to use a proxy server to reach Internet).

So, after observing this traffic capture and seeing that the Meterpreter session died, we can think that the Meterpreter stage payload was unable to reach the listener because, for some reason, it tried to reach it directly, that is, without using the system proxy server in the same way the stager did.

What we are going to do now is to download Meterpreter source code, and try to understand what could be the root cause of this behavior. To do this, we should follow the “Building — Windows” guide published in the Rapid7 github (go to references for a link).

Now, as suggested by the guide, we can use Visual Studio 2013 to open the project solution file (\metasploit-payloads\c\meterpreter\workspace\meterpreter.sln) and start exploring the source code.

After exploring the code, we can observe that within the “server_transport_winhttp.c” source code file there is a proxy settings logic implemented (please, go to the references to locate the source file quickly).

The following screenshot shows part of the code where the proxy setting is evaluated by Meterpreter:

As I learnt from the Meterpreter reverse_https related threads in github, it will try to use (firstly) the WinHTTP Windows API for getting access to the Internet and in this portion of code, we are seeing exactly that.

As we can see in the code it has plenty of dprintf call sentences that are used for debugging purposes, and that would provide valuable information during our runtime analysis.

In order to make the debugging information available for us, it is required to modify the DEBUGTRACE pre-processor constant in the common.h source code header file that will make the server (Meterpreter DLL loaded in the victim) to produce debug output that can be read using Visual Studio’s _Output_ window, DebugView from SysInternals, or _Windbg_.

The following screenshot shows the original DEBUGTRACE constant commented out in the common.h source code file:

The following screenshot shows the required modification to the source code to obtain debugging information:

Now, it is time to build the solution and copy the resulting “metsrv.x86.dll” binary file saved at “\metasploit-payloads\c\meterpreter\output\x86\” to the attacker machine (where the metasploit listener is), to the corresponding path (in my case, it is /usr/share/metasploit-framework/vendor/bundle/ruby/2.3.0/gems/metasploit-payloads-1.1.26/data/meterpreter/).

On the debugging machine, let’s run the “DebugView” program and then execute the backdoored binary again to have the Meterpreter stager running on it.

The following screenshot shows the debugging output produced on the victim machine:

From the debugging information (logs) generated by Meterpreter, it can be observed that the lines 70 through 74 corresponds to the lines 48 through 52 of the server_transport_winhttp.c source code file, where the dprintf sentences are. In particular, the line 71 (“[PROXY] AutoDetect: yes”) indicates that the proxy “AutoDetect” setting was found in the victim machine. However, the proxy URL obtained was NULL. Finally, after that, it can be observed that the stage tried to send a GET request (on line 75).

Thanks to the debugging output generated by Meterpreter, we are now closer to the problem root. It looks like the piece of code that handles the Windows proxy setting is not properly implemented. In order to solve the problem, we have to analyze the code, modify it and test it.

As building the Meterpreter C solution multiple times, copying the resulting metsrv DLL to the attacker machine and testing it with the victim is too much time consuming I thought it would be easier and painless to replicate the proxy handling piece of code in Python (ctypes to the rescue) and modify it multiple times in the victim machine.

The following is, more or less, the Meterpreter proxy piece of code that can be found in the analyzed version of the server_transport_winhttp.c source code file, but written in Python:

The following screenshot shows the execution of the script on the victim machine:

The output of the script shows the same information that was obtained in the debugging logs. The proxy auto configuration setting was detected, but no proxy address was obtained.

If you check the code again you will realize that the DHCP and DNS possibilities are within the “if” block that evaluates the autoconfiguration URL (ieConfig.lpszAutoConfigUrl). However, this block would not be executed if only the AutoDetect option is enable, and that is exactly what is happening on this particular victim machine.

In this particular scenario (with this victim machine), the proxy configuration is being obtained via DHCP through the option 252.

The following screenshot shows DHCP transaction packets sniffed on the victim machine:

Observe from the DHCP transaction sniffed on the victim machine that the DHCP answer, from the server, contains the option 252 (Private/Proxy autodiscovery) with the proxy URL that should be used to obtain information about the proxy. Remember that this is what we obtained before using the autoprox.exe tool.

Before continuing, it is important to understand the three alternatives that Windows provides for proxy configuration:

Automatically detect settings: use the URL obtained via DHCP (options 252) or request the WPAD hostname via DNS, LLMNR or NBNS (if enabled). Use automatic configuration script: download the configuration script from the specified URL and use it for determining when to use proxy servers. Proxy server: manually configured proxy server for different protocols.

So, now that we have more precise information about the problem root cause, I will slightly modify the code to specifically consider the Auto Detect possibility. Let’s first do it in Python, and if it works then update the Meterpreter C code and build the Meterpreter payload.

The following is the modified Python code:

In the modified code it can be observed that it now considers the possibility of a proxy configured via DHCP/DNS. Let’s now run it and see how it behaves.

The following screenshot shows the output of the modified Python code run on the victim machine:

Observe that it successfully detected the proxy configuration that was obtained via DHCP and it shows the exact same proxy we observed at the beginning of this article (10.x.x.20).

Now that we know that this code works, let’s update the Meterpreter C source code (server_transport_winhttp.c) to test it with our backdoored binary.

The following extract of code shows the updated piece on the Meterpreter source code:

In the dark grey zone, it can be observed the updated portion. After modifying it, build the solution again, copy the resulting metsrv Meterpreter DLL to the listener machine and run the listener again to wait for the client.

The following screenshot shows the listener running on the attacker machine:

Observe how it was possible to successfully obtain a Meterpreter session when the victim machine uses the proxy “Auto Detect” configuration (DHCP option 252 in this case).

Problem root cause

Now, it is time to discuss something you may have wondered when reading this article:

Why did the stager was able to reach the attacker machine in first place?

What is the difference between the stager payload and the stage one in terms of communications?

In order to find the answer for those questions, we first need to understand how Meterpreter works at the moment of this writing.

Let’s start by the beginning: the Windows API provides two mechanisms or interfaces to communicate via HTTP(s): WinInet and WinHTTP. In the context of Meterpreter, there are two features that are interesting for us when dealing with HTTPs communication layer:

The ability to validate the certificate signature presented by the HTTPs server (Metasploit listener running on the attacker machine) to prevent the content inspection by agents such as L7 network firewalls. In other words, we desire to perform certificate pinning. The ability to transparently use the current’s user proxy configuration to be able to reach the listener through Internet.

It turns out to be that both features cannot be found in the same Windows API, that is:

WinInet:

Is transparently proxy aware, which means that if the current user system proxy configuration is working for Internet Explorer, then it works for WinInet powered applications.

Does not provide mechanisms to perform a custom validation of a SSL/TLS certificate.

WinHTTP:

Allows to trivially implement a custom verification of the SSL certificate presented by a server.

Does not use the current user system proxy configuration transparently.

Now, in terms of Meterpreter, we have two different stagers payloads that can be used:

The reverse_https Meterpreter payload uses WinInet Windows API, which means it cannot perform a certificate validation, but will use the proxy system transparently. That is, if the user can access Internet via IE, then the stager can also do it.

Meterpreter payload uses WinInet Windows API, which means it cannot perform a certificate validation, but will use the proxy system transparently. That is, if the user can access Internet via IE, then the stager can also do it. The reverse_winhttps Meterpreter payload uses WinHTTP Windows API, which means it can perform a certificate validation, but the system proxy will have to be “used” manually.

In the case of the Meterpreter payload itself (the stage payload), it uses WinHTTP Windows API by default and will fallback to WinInet in case of error (see the documentation to understand a particular error condition with old proxy implementations), except if the user decided to use “paranoid mode”, because WinInet would not be able to validate the certificate, and this is considered a priority.

Note: in the Meterpreter context, “paranoid mode” means that the SSL/TLS certificate signature HAS to be verified and if it was replaced on wire (e.g. a Palo Alto Network firewall being inspecting the content), then the stage should not be downloaded and therefore the session should not be initiated. If the user requires the use of “paranoid mode” for a particular escenario, then the stager will have to use WinHTTP.

Now we have enough background to understand why we faced this problem. I was using the “reverse_https” Meterpreter payload (without caring about “paranoid mode” for testing purposes), which means that the stager used the WinInet API to reach the listener, that is, it was transparently using the current user proxy configuration that was properly working. However, as the Meterpreter stage payload uses, by default, the WinHTTP API and it has, according to my criteria, a bug, then it was not able to reach back the listener on the attacker machine. I think this provides an answer to both questions.

Proxy identification approach

Another question that we didn’t answer is: what would be the best approach to obtain the current user proxy configuration when using the WinHTTP Windows API?

In order to provide an answer for that we need to find out what is the precedence when you have more than one proxy configured on the system and what does Windows do when one option is not working (does it try with another option?).

According to what I found, the proxy settings in the Internet Option configuration dialog box are presented in the order of their precedence. First, the “Automatically detect settings” option is checked, next the “Use automatic configuration script” option is checked and finally the “Use a proxy for your LAN…” is checked.

In addition, a sample code for using the WinHTTP API can be found in the “Developer code sample” of Microsoft MSDN that states:

// Begin processing the proxy settings in the following order:

// 1) Auto-Detect if configured.

// 2) Auto-Config URL if configured.

// 3) Static Proxy Settings if configured

This suggests the same order of precedence we already mentioned.

Fault tolerant implementation

A last question that I have is what happens if a host is configured with multiple proxy options and one of them, with precedence, is not working? Does Windows will continue with the next option until it finds one that works?

In order to provide an answer, we could perform a little experiment or expend hours and hours reversing the Windows components that involve this (mainly wininet.dll), so let’s start by doing the experiment that will, for sure, be less time consuming.

Lab settings

In order to further analyze the Windows proxy settings and capabilities, I created a lab environment with the following features:

A Windows domain with one Domain Controller

Domain: lab.bransh.com

DC IP: 192.168.0.1

DHCP service (192.168.0.100–150)

Three Microsoft Forefront TMG (Thread Management Gateway)

tmg1.lab.bransh.com: 192.168.0.10

tmg2.lab.bransh.com: 192.168.0.11

tmg3.lab.bransh.com: 192.168.0.12

Every TMG server has two network interfaces: the “internal” interface (range 192.168.0.x) is connected to the domain and allows clients to reach Internet through it. The “external” interface is connected to a different network and is used by the Proxy to get direct Internet access.

A Windows client (Windows 8.1 x64)

IP via DHCP

Proxy configuration:

Via DHCP (option 252): tmg1.lab.bransh.com

Via script: http://tmg2.lab.bransh.com/wpad.dat

Manual: tmg3.lab.bransh.com:8080

The client cannot directly reach Internet

Firefox browser is configured to use the system proxy

The following screenshot shows the proxy settings configured in the Windows client host

The following screenshot shows the proxy set via DHCP using the option 252:

Note: the “Automatically detect settings” option can find the proxy settings either via DHCP or via DNS. When using the Windows API, it is possible to specify which one is desired or both.

By means of a simple code that uses the API provided by Windows, it is possible to test a few proxy scenarios. Again, I write the code in Python, as it is very easy to modify and run the code without the need of compiling a C/C++ code in the testing machine every time a modification is needed. However you can do it in the language you prefer:

The code above has two important functions:

GetProxyInfoList(pProxyConfig, target_url): This function evaluates the proxy configuration for the current user, and returns a list of proxy network sockets (IP:PORT) that could be used for the specified URL. It is important to remark that the proxy list contains proxy addresses that could potentially be used to access the Url. However, it does not mean that the proxy servers are actually working. For example, the list could contain the proxy read from a WPAD.DAT file that was specified via the “Use automatic configuration script” option, but the proxy may not be available when trying to access the target URL.

This function evaluates the proxy configuration for the current user, and returns a list of proxy network sockets (IP:PORT) that could be used for the specified URL. It is important to remark that the proxy list contains proxy addresses that could potentially be used to access the Url. However, it does not mean that the proxy servers are actually working. For example, the list could contain the proxy read from a WPAD.DAT file that was specified via the “Use automatic configuration script” option, but the proxy may not be available when trying to access the target URL. CheckProxyStatus(proxy, target_server, target_port): This function will test a proxy against a target server and port (using the root resource URI: /) to verify if the proxy is actually providing access to the resource. This function will help to decide if a proxy, when more than one is available, can be used or not.

Testing scenario #1

In this scenario, the internal network interfaces (192.168.0.x) of the proxy servers tmg1 and tmg2 are disabled after the client machine started. This means that the only option for the client machines to access Internet would be through the proxy server TMG3.

The following screenshot shows the output of the script. In addition, it shows how IE and Firefox deals with the situation:

The testing script shows the following:

The option “Automatically detect settings” is enabled and the obtained proxy is “192.168.0.10:8080” (Windows downloaded the WPAD.PAC file in background and cached the obtained configuration before the proxy internal interface was disabled). However, the proxy is not working. As the internal interface of TMG1 was disabled, it was not possible to actually reach it through the network (a timeout was obtained). The option “Use automatic configuration script” is enabled and the obtained proxy is “192.168.0.11:8080” (Windows downloaded the WPAD.PAC file in background and cached the obtained configuration before the proxy internal interface was disabled). However, the proxy is not working. As the internal interface of TMG2 was disabled, it was not possible to actually reach it through the network (a timeout was obtained). The manually configured proxy server is “tmg3.lab.bransh.com:8080”. This proxy was successfully used and it was possible to send a request through it.

Observe that neither IE nor Firefox were able to reach Internet with the presented configuration. However, a custom application that uses tmg3 as a proxy server would be able to successfully do it.

Testing scenario #2

In this scenario, very similar to the #1, the internal network interfaces (192.168.0.x) of the proxy servers tmg1 and tmg2 are disabled before the client machine started. This means that the only option for the client machines to access Internet would be through the proxy server TMG3.

The following screenshot shows the output of the script. In addition, it shows how IE and Firefox deals with the situation:

When running our testing code, we can observe the following:

The option “Automatically detect settings” is enabled (tmg1.lab.bransh.com/wpad.dat), but no proxy was obtained. This occurred because the proxy server (tmg1) was not reachable when the host received the DHCP configuration (and the option 252 in particular), therefore it was not able to download the wpad.dat proxy configuration file. The option “Use automatic configuration script” is enabled and the provided URL for the configuration file is “tmg2.lab.bransh.com/wpad.dat”. However, it was not possible to download the configuration script because the server is not reachable. The manually configured proxy server is “tmg3.lab.bransh.com:8080”. This proxy was successfully used and it was possible to send a request through it.

Observe that IE was able to understand the configuration and reach Internet. However Firefox was not.

Testing scenario #3

In this scenario, the internal network interface (192.168.0.11) of the proxy server TMG2 was disabled before the client machine started. This means that client machines can access Internet through proxy servers TMG1 and TMG3.

The following screenshot shows the output of the script. In addition, it shows how IE and Firefox deals with the situation:

When running our testing code, we can observe the following:

The option “Automatically detect settings” is enabled and it is possible to access Internet through the proxy obtained (192.168.0.10:8080). The option “Use automatic configuration script” is enabled and the provided URL for the configuration file is “tmg2.lab.bransh.com/wpad.dat”. However, as the network interface of this proxy was disabled, it was not possible to download the configuration script. The manually configured proxy server is “tmg3.lab.bransh.com:8080”. This proxy was successfully used and it was possible to send a request through it.

In addition, observe that IE was able to understand the configuration and reach Internet. However Firefox was not.

Testing scenario #4

In this scenario, only the internal network interface (192.168.0.11) of the proxy server TMG2 is enabled:

When running our testing code, we can observe the following:

The option “Automatically detect settings” is enabled and it is not possible to access Internet through the proxy (192.168.0.10:8080). The option “Use automatic configuration script” is enabled and the provided URL for the configuration file is “tmg2.lab.bransh.com/wpad.dat”. In addition, the obtained proxy is 192.168.0.11:8080 and it is possible to reach Internet using it. The manually configured proxy server is “tmg3.lab.bransh.com:8080”. This proxy is not reachable and a TIMEOUT is obtained.

In addition, observe that IE was not able to understand the configuration and reach Internet. However Firefox successfully used the configuration and got access to Internet.

Testing scenario #5

In this scenario, the internal network interfaces of all three proxy servers are enabled. However, the external interface of the servers TMG1 and TMG2 were disabled:

When running our testing code, we can observe the following:

The option “Automatically detect settings” is enabled, the specificed proxy (192.168.0.10:8080) is reachable. However it answers with an error (status code 502) indicating that it is not possible to reach Internet through it. The option “Use automatic configuration script” is also enabled, the specificed proxy (192.168.0.11:8080) is reachable. However it answers with an error (status code 502) indicating that it is not possible to reach Internet through it. The manually configured proxy server is “tmg3.lab.bransh.com:8080”. This proxy is reachable and it does provide access to Internet.

Observe that neither IE nor Firefox were able to access Internet. However, a custom application that uses TMG3 as a proxy server would be able to successfully do it.

Conclusion

In certain scenarios, like the one exposed in the first part of this post, we will find that our favorite tool does not behaves as expected. In those situations we have mainly two options: try to find another solution or get our hands dirty and make it work. For the particular enterprise escenario I described, the fix applied to Meterpreter worked properly and after compiling its Dll, it was possible to make it work using the proxy configuration described. I’m not sure if this fix will be applied to the Meterpreter code, but if you find yourself with something like this, now you know what to do.

On the other hand, we saw that Windows tries to use the proxy configuration in order (according to the precedence we already talked). However, it seems that once a proxy was obtained (e.g. scenario #1), if it does not work, Windows does not try to use another available option. Also, we saw that Internet Explorer and Firefox, when configured as “Use system proxy settings”, do not behave in the same way when looking for a Proxy. Finally, we also saw that in both cases, when a proxy is reachable but it does not provide Internet access for any reason (e.g. the Internet link died), they will not try to use a different one that may work.

Considering the results, we can see that we do have the necessary API functions to evaluate all the proxy configurations and even test them to see if they actually allow to access an Internet resource. Therefore, with a few more lines of code we could make our APT solution more robust so that it works even under this kind of scenarios. However, I have to admit that this are very uncommon scenarios, where a client workstation has more than one proxy configured, and I don’t really think why an administrator could end up with this kind of mess. On the other hand, I’m not completely sure if it would be a good idea to make our APT work even if IE is not working. What if a host is believed to be disconnected from the Internet, but suddenly it starts showing Internet activity by cleverly using the available proxies. This may be strange for a blue team, perhaps.

As a final conclusion, I would say that making our APT solution as robust as IE is would be enough to make it work in most cases. If IE is able to reach Internet, then the APT will be as well.

References