The other day my coworker informs me, ‘hey, there’s a weird process making network connections on your box.’ A dreaded string of words if there ever were any for the security-conscious developer.

This is what he pastes to me:

/usr/bin/curl -G —location —max-time 60 —silent —write-out %{http_code} —data since=2009-12-31%2023%3A0%3A0 —data program=prof —data count=1 —output /dev/null http://131.159.28.73/popcon.php;

Now, I fully expected this to be some normal activity observed and misinterpreted, but as I saw this I began to grow distressed. Curl POSTing to some random IP address and popcon.php?

The computer in question was not a production box, but my development VM environment. This didn’t do much to ease my worries, though. Developers are often the weakest links in the security chain. They have access to all of the source code, the ability to push to repos and often direct access to production boxes. If my dev environment was hacked, I would not be having a fun time of it.

Now, you’re probably asking yourself: how did he know this was running? Great question!

The company I work for, we kind of make it our business to know what exactly is running on your boxes, and alerting you of strange activity. New network activity from a process, well, that falls in that category. Like any good company, we dogfood our product on our dev and production boxes. My coworker saw the email alert about the strange curl process, gave me the heads up, and I proceeded right away to investigating it by diving into our user interface:

curl command in question

Right away, I noticed a few weird things: it’s new activity (which I observed by viewing the lonely bar in the executable history diagram), and the user that ran it was ‘root’, which was even more alarming. Looking at the process tree, I observed that its parent process was a shell script.

In Linux, all new processes are created by fork() and execve() from a parent process. We designed our UI so you can walk up the process tree to the parent process all the way up to init (pid=1), just by clicking on the parent process in the tree. So, I did exactly that. What launched this shell script? Was it some insidious piece of malware running active in memory? A new init script?