A month or so ago New Yorker pop critic Sasha Frere-Jones wrote a column about the state of rap, starting with the proposition ("proclamation" would be too bombastic a word) that 2009 was, in fact, the year of hip-hop's death. I read it and couldn't find a thing to disagree with. My only quibble was that he might have called it earlier. Perhaps 2006, when Nas released Hip Hop Is Dead. Or even 2004, when Timbaland "repeatedly voiced … a frustration with pop music, particularly the hip-hop end of it" (according to his New York Times interviewer, one Sasha Frere-Jones) and further declared: ''It's time for me to retire, because it ain't the same … I'm tired of stuff now, even stuff that I do." (He also, said, mindblowingly, that "Coldplay and Radiohead are the illest groups to me. That's music".) That same year, 2004, Jay-Z also confessed – on the eve of his (ha ha) retirement and moving on to bigger, more challenging fields of endeavour – that he too was "bored" with hip-hop. Rap had become "corny", he said, and accordingly he no longer felt peer pressure to raise his game (something underlined by the steady decline of his output after 2001's magisterial The Blueprint).

As I read Frere-Jones's piece, I also knew there'd be complaints and counter-arguments galore. And sure enough they came – droves of pissed-off fanboys brandishing obscure mixtapes and overlooked albums as proof of the genre's continued vitality. Some whined that the sample on which his genre survey was based was too small (Jay-Z's new slab of going-through-the-motions, efforts by Kid Cudi and Wu-Tang clansman Raekwon, unsigned rapper Freddie Gibbs) while others questioned the entitlement of a white fortysomething to pronounce on the vital signs of a black pop genre in the first place. I don't know, but I'd have thought 25 years of attentive fandom would at least justify having an opinion. Plus it's not as though this kind of gloom-and-doomy assessment of hip-hop hasn't been voiced repeatedly by black critics and black fans, not to mention the performers themselves.

Pundits who deem something to be in decline are invariably accused of nostalgia, so another angle of retort was that Frere-Jones was pining for the Lost Golden Age: the late 80s/early 90s, rap in its first flush of artistic maturity, but still a genre primarily oriented around samples and breakbeats. The era of DJ/producers like the Bomb Squad and Eric B, Marley Marl and Prince Paul, Premier and Pete Rock. But you don't need to go back that far to locate a peak now passed. You just have to think of the first four years of this decade, which was the continuation in full force of a late 90s resurgence of mainstream rap that effortlessly managed to be commercial and street at the same time, combining pop hooks and jagged rhythmic innovation, glitzy entertainment and edge. This seven-year-long surge was largely but not exclusively driven by the Dirty South: cities like Atlanta, New Orleans, Memphis and Houston; producers like Timbaland, Neptunes, Mannie Fresh, Lil Jon, and Mr Collipark; MCs like Ludacris, Missy Elliott, Three 6 Mafia, Clipse, Ying Yang Twins, and those Cash Money hot boys Juvenile, BG and Lil Wayne. But the rest of the US played its part, from the Ruff Ryders family (DMX, the Lox, Eve, plus producer Swizz Beatz) through Ja Rule and Nelly, to the Dre/Eminem/50 Cent axis.

Underground rap fans sniffed at this brash, bolshy sound, based not on the breaks-and-samples template of classic hip-hop (partly because licensing samples had become too costly) but favouring instead synthesiser riffs and refrains modelled on techno-rave and 80s pulp movie soundtracks. The drum machine rhythms had an 80s vibe too, the double-time hi-hats and 808 bass-booms reactivating that whole other side of early hip-hop based around electro not looped breaks, Bambaataa not JB. Backpackers also complained about all these crossover rap hits with R&B choruses, which they saw as selling out the ideal of hip-hop as a showcase for MC virtuosity. But even as the ascendant street rap sound borrowed R&B's hook power and gloss, the nu-skool rap influenced R&B. By the turn of the millennium the genre were less separate than Siamese twins (something symbolised by the union of Beyoncé and Jay-Z). Together street rap and nu-R&B flooded global pop music with rhythmic pizzazz and in-yer-face attitude. The fall-out, just in the UK alone, includes the "chav-pop" swarm of girl groups and boy bands, MIA, and grime (not so much in the MC-ing, which owes more to jungle and dancehall, but in terms of beats and production, plus what would prove to be false expectations for mega-fame and Puffy/Jay-Z style transmedia empire building).

It's the vigour and invention of the first third of the Noughties that makes the last five years of rap look stalled and sapped, not old-skool days so remote only grey-hairs remember them. By any sensible metric, rap has slipped hugely from where it was when this decade began. It's not dominating the pop charts anymore, and neither is it irrigating the mainstream with new beats, styles, and slanguage. It's not producing major album-length statements, give or take an 808s & Heartbreak (revealingly, not rapped but sung). It's not even coming up with compelling new personalities. The last, by my reckoning, were Lil Wayne (whose debut was released in 1999) and Kanye West (who debuted in early 2004). West has turned out to be a mixed blessing, while Wayne spread his brilliance thin across innumerable mixtapes, plus 2008's uneven Tha Carter III. Some swear by TI and Young Jeezy as charismatic artists, but neither came up with a MC persona we've not seen before. And, for these last three or four years, rap has been a desperately unmemorable procession of cookie-cutter ballers – Jim Jones, Gucci Mane, Yung Doc, Soulja Boy, Lil Boosie, Gummi Bares – whose lyrics trudge a hedonic treadmill of bling and booty, punctuated by the occasional inane dance-craze. Even the sound of rap – always its saving grace in the absence of political engagement or MC-as-poet depth – deteriorated in the second half of this decade. The odd angles and eerie spaces in productions by Mannie Fresh or Mr Collipark were flattened out, replaced by portentous digi-synth fanfares and lumbering beats, a brittle bass-less blare that seemed pre-degraded to 128kbps to cut through better via YouTube and mobile phone ("ringtone rap", some called it), rendered all the more cheapo-sounding and plastic non-fantastic by the endless Auto-Tune fad.

One of the most interesting observations in Frere-Jones's piece is that rap producers are abandoning swing and syncopation for more pulse-based club rhythms (house/trance/electro-pop), resulting in a shift to a European rather than African-American feel. Flo Rida's Right Round, based on Dead or Alive's Eighties Hi-NRG hit, is a good example, and new nadir. Actually, I still hear quite a lot of bump and skitter in street rap but there's a pedestrian familiarity to the beats: they do the job solidly enough but they're the rhythmic equivalent of comfort food, reflexively tugging at your hips and shoulders but never approaching the stark strangeness of early Noughties productions like Ludacris's What's Your Fantasy or J-Kwon's Tipsy.

I quizzed Josiah Schirmacher, a young DJ friend who disagreed vehemently with the New Yorker piece and he replied that there was plenty of life in hip-hop but it was all "on the local level", pointing to styles like jerk, as favored by teenagers in Los Angeles. This was another story of the hip-hop Noughties: the succession of city-based sounds, starting with New Orleans bounce and continuing with crunk, hyphy, snap, juke, etc, which hatch as regional styles but thanks to the marvels of the internet (especially YouTube) are chased avidly by an international cadre of largely white, middle-class beat-nerds. I was one for a while, but then started to feel that underneath the cool local quirks (for instance, in the Bay Area, hyphy MCs shout out to freeway exits, which is how the different neighbourhoods know themselves, as opposed to, say, wards in New Orleans) all these sounds were, at base, the same. Electro variant + goofy dance + bawdy lyrics + (optional) drug-of-choice (E, with hyphy; purple drank aka cough syrup in other places, and so on). In a funny way, the pasty-faced, steroid-popping northwest England scene donk is a distant cousin of all these black American sounds: same anonymous rapping, same humorously boastful/sexist lyrics, same bling videos, same utterly local orientation offset by the occasional nationwide hit. The Blackout Crew, basically, are Cold Flamez.

Haven't talked about underground rap yet, but it doesn't exactly impose itself on your consciousness, does it? Like the lo-fi indie it resembles, this sector puttered on much like it did through the 90s, odd flashes of genius (Cannibal Ox, Dilla, Quasimoto/Madlib etc) amid the crate-digging antiquarianism. Barely creating a ripple in the larger pop culture, undie rap is probably pretty content with its niche, a haven of "quality" in a mercenary world. This stuff bears the same relationship to Dirty South type-rap that someone like Elvis Costello did with rock after 1984 (and, what d'ya know, Costello recently teamed up with the Roots to perform some of his classics on a US chat show). But as with the late-80s "golden age", the late 90s/early 00s surge showed that during rap's heyday phases the most innovative music rises to the top; it's not something you have to seek out, because it dominates radio and music-video channels, booms from passing cars.

The "Death of …" piece is a genre of criticism that's fallen into disrepute (there was a period when you'd be constantly tripping over essays announcing the End of something: art, theory, rock, rave ). People now seem to feel that "no genre ever really dies" (to adapt the Neptunes/NERD motto). Was this in fact one of the problems with the Noughties? No genre went gently into that good night: they all clung on, cluttering up the musical landscape. This not only made it harder for new things to emerge, it's meant that we've all come to forget that, in fact, totally new things have emerged in the past. There was, for instance, a time when hip-hop didn't exist. The refusal to admit that a genre can die (which doesn't mean literally disappear – it may even generate good stuff now and then –but refers to stagnation, irrelevance, becoming uncoupled from the zeitgeist) is a denial of the possibility of change, renewal, the unexpected. The very vitality of a form of music implies the possibility of its eventual death.

I sympathise with the Frere-Jones dissenters; it must be galling, having built up all that expertise and knowledge, to have your subcultural capital voided by some old git in a bow tie (compulsory at the New Yorker, don't you know) airily declaring the area obsolete. One of the cunning rhetorical ruses used in these critical turf wars between enthusiasts versus curmudgeons is to suggest that the latter are projecting their physical decrepitude on to the state of music. But you could just as easily reverse that and argue that the young are projecting their physical vitality on to the senescent body of pop (every fibre of their hormonally flushed being shouts "it still LIVES!"). I won't say that hip-hop is dead. But it does seem to be doing a good impersonation of being at death's door. More to the point, judging by its output in recent years, it's become a deadening force: as a listening experience, but also as something that maintains a deadlock on the musical imagination (and personal ambitions) of Black American youth. I doubt very much that this demographic has no more surprises up its sleeves in terms of sound and style, judging by past form(s) (jazz, rhythm and blues, funk, house, et al ). But that New Thing won't come until they tire of hip-hop themselves and turn against it.