The publication of a video showing the murder of American journalist Steven Joel Sotloff surprised even the terrorists who supported the killing. Why? Because a little-known research group known as Site Intelligence spotted the video (and alerted the media) before even the jihadis did.

Site (it stands for ‘Search for International Terrorist Entities’) is a for-profit US-based consultancy group that monitors and tracks the activities of “international terrorists and the global jihadist network”.

The group has ties to both the US government and commercial clients and specialises in monitoring the online activities of jihadis, watching discussions over the best suicide vests and how to launder money play out over forums – before issuing warnings to their clients (they also keep a regularly updated blog).

A profile of Site’s co-founder and director Rita Katz by the New Yorker magazine in 2006 described Site’s customers as “people in government […] frustrated by how long it takes to get information through official channels” as well as “people in corporate security and in the media”.

“Each day, Katz finds about a half-dozen items on the Arabic message boards that are worth distributing,” writes Benjamin Wallace-Wells in the New Yorker’s profile.

“Her researchers, who monitor English-language jihadist Web sites, often find a few more. Some are propaganda: videos taking responsibility for attacks, statements of intent to attack, announcements of allegiances or splits. Others involve tactics and weapons.”

In 2004 Katz was commended by the FBI for her contributions to counterterrorism investigations, and in 2007 her group was credited with finding the first videos of Osama bin Laden after three years of silence.