When the four-star general Stanley A McChrystal received an email asking if he might like to spend a month of his extremely busy life being trailed by a Kabul-based freelance journalist called Michael Hastings, he could have been forgiven for issuing a polite but firm reply to the effect that such a project would regrettably not dovetail with his already-packed schedule.

The General was, after all, in the middle of prosecuting the longest-running war in American history. "I was expecting actually no access or perhaps, you know, one or two days or a 45-minute interview," Hastings recalled this week. "Instead, the response was: 'Hey, why don't you come over to Paris when we're going on a Nato trip? You can join us in Paris next week, see the general, meet him, and then come to Kabul a few weeks after that to see him in the war zone.'"

It's impossible to know what exactly persuaded McCrystal's press staff to invite Hastings into their inner sanctum, where he would be privy to a frat-boy atmosphere and culture of contempt for the White House which would ultimately this week force the General to resign from his job as commander of US forces in Afghanistan.

But two words on the journalist's initial pitch appear to have tickled their fancy: Rolling Stone. "They wanted to reach a different demographic than they had during the other profiles," Hastings said this week. "A number of young officers read Rolling Stone. They figured they would reach an audience that they normally wouldn't reach."

In making that fateful calculation, the general's PR staff appear to have figured that they were setting up a journalistic hagiography which would see their boss lobbed softball questions by a star-struck pop journalist, before being showcased in the pages of fashionable music magazine alongside such glitzy celebrities as Lady GaGa and Russell Brand.

They were quite wrong, of course. And their apparent surprise at the forensic tone of the article reveals a profound ignorance about the nature of Rolling Stone: it may look like a fluffy music magazine, but for more than 40 years it has also been a forum for serious, agenda-setting journalism in the lofty fields of politics and popular culture.

The article Hastings produced is part of a tradition of "long form" reportage which stretches back to the magazine's earliest days, in which writers are instructed to spend long periods with their subjects, building a close relationship in order to gain access and insights that normal journalists would be unable to obtain.

"Some people have expressed surprise that the McChrystal piece appeared in Rolling Stone," says Simon Dumenco, a media columnist for Advertising Age. "But I would say that actually, it could only have appeared in that magazine, which is one of the few publications that still has a commitment to long-form journalism. Reading the piece, you get the impression that its iconic stature in the pop-culture firmament is what gave Hastings access, and that McChrystal's staff were seduced into being indiscreet in front of him, because they grew to feel comfortable with him and thought he worked for a magazine which might make them look cool."

They certainly weren't the first people to make that misjudgement. Rolling Stone's most famous scribe was Hunter S Thompson, who joined the magazine shortly after it was founded, and who, in his devotion to "gonzo" reportage, would often spend weeks consuming drink and drugs with the very people he was supposed to be interviewing.

Thompson made waves with famous articles like "Fear and Loathing in Las Vegas", together with his coverage of the presidency of Richard Nixon. Another early staffer, Tom Wolfe, built his reputation on accounts of the early US space programme which would eventually become his book The Right Stuff.

Rolling Stone was founded in San Francisco in November 1967, at the height of the counter-culture movement, by a university drop-out called Jann Wenner, who financed its earliest editions with the help of a $7,500 loan from his family.

"The spirit was that rock'n'roll was not merely music, but an expression of a new generation," says Victor Navasky, the chairman of the Columbia Journalism Review. "They stood for a revolutionary principle, mixed up with new-age spiritualism, and were interested in politics as well as music. People presumed it was a music magazine, but it was far more than that."

In the early years, Wenner cultivated an extraordinarily talented team to achieve his vision in the fortnightly publication, launching the careers of now-famous writers such as Joe Klein, PJ O'Rourke and Cameron Crowe, and a little-known photographer called Annie Leibovitz.

His magazine quickly grew, from being the in-house publication of the hippy movement to one of the most fashionable and prestigious titles in the US. The band Dr Hook and the Medicine Show released a hit satirical song poking fun at the desire of supposedly reclusive musicians such as Bob Dylan to bare their souls in order to appear on its cover.

Wenner duly became extremely wealthy, able to add some of the greatest reporters of his era to his payroll, including a post-Watergate Carl Bernstein, who published a famous investigation into the CIAs' monitoring of journalists. He was also able to build a small stable of magazines including Men's Journal and US Weekly.

Crucially, Wenner, who is now 64 and remains the title's editor, has managed in recent years to keep the title true to its founding principles. In an era when the attention spans of readers are said to be in terminal decline, Rolling Stone still boasts 1.4 million readers, almost all subscribers who pay just $30 (£20) for two-year subscriptions. Yet the title's relatively highbrow nature and young demographic makes the title attractive to advertisers.

Though the magazine carries its share of fluff these days, its inner pages remain a sanctuary for in-depth reportage. In recent years, it has invested heavily in Matt Taibbi's award-winning coverage of the financial crisis. While it has also begun to carry shorter pieces, its treatment of even the most over-exposed celebrities remains conspicuously intelligent.

From a financial point of view, Rolling Stone is also not saddled with the vast debts of other publications whose corporate owners have entered into ill-though-out merger and acquisition deals in recent decades. And Wenner appears to be unconcerned by the fact that its traditional style of journalism is unsuited to the internet.

"At a moment in our culture when Newsweek is for sale, and the market for weekly, bi-weekly and monthly titles is supposed to be disappearing, it's not accidental that it was a magazine that brought McChrystal down," adds Navasky. "This kind of journalism is what magazines should do. It's what they were made for."

In an era where the business of serious newsgathering is widely held to be on life support, the big question is of course whether owners of other, threatened highbrow titles will now learn from Rolling Stone's success.

Rolling Stone's greatest hits

Archie Bland

Fear and Loathing in Las Vegas (1971)

No writer is more closely associated with the magazine than Hunter S Thompson, whose most famous work appeared as a serialisation after it was turned down by Sports Illustrated. With Ralph Steadman's unforgettable illustrations, it defined the "decade of dope" – and it is hard to imagine it in any other publication of similar stature.

The CIA and the Media (1977)

After his Watergate exposé with Bob Woodward, Carl Bernstein was half of the most fêted journalistic duo in America when he joined Rolling Stone. His seminal piece detailed how the CIA used a network of 400 journalists to exert influence in the media, a classically anti-establishment piece of investigative work that chimed perfectly with Rolling Stone's style.

The Bonfire of the Vanities (1984)

In desperate need of discipline for his sprawling planned novel, the longtime contributor Tom Wolfe agreed to publish it in fortnightly instalments. Although he rewrote it significantly by the time it appeared in book form, the publicity that the episodes generated was a huge boon for Wolfe – to say nothing of the $200,000 that editor Jann Wenner paid him for the rights.

The Make-Believe Maverick (2008)

Tim Dickinson's profile of the Republican candidate John McCain before the last presidential election painted McCain as petulant and abusive, and detailed a particularly vile insult he was said to have aimed at his wife.

Great American Bubble Machine (2009)