I traveled from VMworld to the lab last Wednesday, and during that time, something infected two websites I control.

I suspect the servers were used as part of a Syn Flood attack. The servers, both using WordPress, would come up and serve their web pages, but then they would quickly run out of cache by processes that were difficult to track.

+ Also on Network World: Analyzing real WordPress hacking attempts +

They initially made contact with some IPs located conveniently in Russia, then lots of syn traffic, and interesting session waits and listens. It took about two minutes before the sites cratered from resource drainage, and the errantly injected processes dominated then effectively cratered the servers from their intended use.

In turn, the sites have Debian underneath. I could get to root and Debian for about a minute until the shell performance deteriorated to a total stop.

The first clues were found by using netstat –a to see where traffic was going. At first, numerous IPs were connected. I tried to use iptables to drop their connections. Then after reboot, other IPs would show up, until I filled up a decent table of dropped IP addresses.

My guess: Something in the init files phoned home, got instructions, then went to work, chewing through the instances resources until they dominated the instance. After a dozen or so reboots, I had them all, or at least no new IPs started popping up.

The two WordPress servers were using 4.5 and needed to be upgraded to 4.6. Both had the free version of Wordfence, the WordPress security plugin, which in this case was as strong as Kleenex. Is the $8.75/month version of Wordfence worth it to block foreign IPs? I’m reconsidering my decision not to get the professional version. Seems like usury to me, but yes, for the time spent doing forensics—which at this writing are far from complete—it may have been worth the money, save that I wouldn’t have learned much.

I re-did the servers: one from scratch, the other one from updated snapshots provided by diligent backups. One of the sites needed a fresh face, anyway.

Lessons learned:

These were targeted by because they were either Debian or WordPress, as other assets weren’t touched.

It’ll take me longer than I expected to find the root cause of the crack, and while I’m doing that, I’m not doing real work in the lab.

Backups saved my bacon (and I’m a vegetarian).

Know and document your configurations if you want any hope of forensic success. You can’t guess logical configurations—you have to KNOW them through documenting them AND having the doc accessible.

If you host things yourself rather than a cloud provider, you’re on your own but may have a faster time of stanching infections as a direct result of not needing an intermediary—if you’re totally savvy with both the platform and the app.

I don’t have valuable assets on my sites, but for those who do, documenting those assets can be come instantaneously tremendously important.

WordPress backup doesn’t backup host configuration files. You do. I did. Whew!

I wonder when someone like Sophos will look at WordPress and provide an interesting alternative to the freeware version of Wordfence (which I otherwise kind of like).

The sites are up today. I’m not looking for click count. They’re there as my exercise at “being on the web.” I expect more attempts. Amusingly, they didn’t hit my honeypot. Syslog files saved (actually the entire /var/log); I’ll meander through it as time permits. Until then, cron is pinging the site with more regularity.