Over the last year or so, I've found myself at a lot of nightclubs for Vice and Noisey's Big Night Out series. I've trodden the same sticky floors as creatine-riddled uni lads, hemp-clad crusties, suburban emo kids, south-of-Sloane-Square debutantes and Arg from Towie. I've seen fisticuffs, break-ups, make-outs, running mascara and spilt vodka cocktails congealing on the basement floor of club culture. I once saw a man masturbate into his own flip-flops, and another get stabbed with an EpiPen at a student union bar in Newcastle.

What I've been looking for is the heart of the UK club scene, the uniting factor that brings such disparate groups of people together to essentially do the same thing in different places with different music and different clothes. I'm interested in both the similarities and separations in our drinking and dancing culture, and why we all feel compelled to do things we regret on the weekend in order make the rest of the week bearable. Clearly, drugs, alcohol and the chance of getting laid play a massive part in the reasons why people put themselves through it all. But I think there's something much more ephemeral and harder to explain – something purer, even – holding it all together.

Britain is a country that genuinely loves to party, and always has done. You've only got to look at our long and proud history of banquets, music hall knees-ups and bashment raves to understand this. For a supposedly Protestant, sexless country, we sure do like a drink and a shag. It's the eternal dichotomy of British life, and it suits everyone just fine. This is a country that gave us the Wigan Casino, the Boy's Own rave-in-a-field, The Blue Note and Fabric. We have a long, proud, brave and innovative nightlife heritage. But has the creeping city-centre homogenisation which has rendered every town in the country from Truro to Cumbernauld a pedestrianised deadzone, with only differing-sized Body Shops to distinguish between them, affected our nightlife? Has it become just another excuse to squeeze money out of a beleaguered population?

In a word: yes. The sad fact is that clubbing in the UK is no longer the preserve of the hip, the alternative, or even the hedonistic amongst us. You've only got to survey the queues of just about any establishment bigger than a Wetherspoons to understand that. The country's high-street clubs have been taken over by people who don't dance, don't take drugs, don't dress up, but do down pitchers of Barbicide-blue booze and take pictures of themselves in the toilets.

It's a tragedy, but one that says more about the establishments than the people. A lot of clubs in the UK barely fit their trade descriptions, instead acting as nothing more than pubs that stay open past 1am and play loud music. Legally they're nightclubs, but they seem to fulfil little of that halycon rave dream of community, euphoria and cool promised to a generation raised on Human Traffic and tales of the Haçienda. Instead, they're basically modern saloons, where the punters wear Superdry rather than Stetsons, and they're seemingly just as dangerous (even if, because all the tumblers are now plastic, you stand slightly less chance of being scarred for life because of your haircut than you used to). Other clubs seem more neutered than their rough-and-ready predecessors. Bouncers have credentials, DJs have degrees and club owners have responsibilities. They're basically Pizza Expresses with retinal scanners on the door. This doesn't make anything any safer, however; it just makes it less interesting.

It begs the question of whether there's even an underground to speak of any more. It's as if we face a grim future where our teenagers are forced to tag along with Championship footballers and ITV2 starlets in order to get into the half-decent clubs. But luckily there is still a teeming underground at odds with such sanitised versions of nightlife, and it's as healthy as any great clubbing era in British history.

Even the most casual chart observer realises that British electronic music has made something of a comeback in recent years. Dubstep and deep house have laid the groundwork for the return of electronic breaks, beats and bass which you can hear everywhere from Fabric to Virgin Active – think garage-house duo Disclosure, who recently had a No 1 album, and house producer Breach, who produced one of the hits of the summer in Jack. That fact that even your mum has probably heard of these acts means that the question of whether British dance music is back has been emphatically answered. This time, however, you can't help feeling that it's never going to go away again.

Dance musicians have been called the new rock stars before, but we've come a long way since the "God is a DJ" days, when Ibiza heroes flew their Learjets into mountains of cocaine and disappeared into infinite Pacha residencies. Yes, there are still many DJs out there who aim to rock football stadiums with laser-blazing live shows, in which they levitate above the thousands in Swarovski-encrusted, skull-shaped DJ booths. But nowadays, the new clubbing wave's discerning dance music DJs are more akin to the underground kings of hardcore punk than they are to bloated rockers doing Christ poses on the front cover of music mags. They've realised that it's their non-conformist ethic that made them so popular in the first place, and they're much keener to encourage a dancefloor-focused atmosphere. A DJ such as Glasgow's Jackmaster probably plays to as many people in the course of a year as someone like Oakey once did, but he prefers to spin to dedicated audiences in mid-sized, dark and dingy venues like Elephant & Castle's Corsica Studios or Glasgow's Make Do, rather than at soulless superclubs.

The new wave of DJs have an understanding of how a club should feel that's helped to maintain a healthy scene. Their fanbases appreciate this in the same way that Black Flag's fanbase did. It keeps things intimate, it retains the soul and it makes the audience feel more like they're a part of the scene, rather than just a hapless punter being ripped off at every turn. Going to clubs that encourage this, like roadblock London electronic nights Trouble Vision and Oscillate Wildly, or watching just about any Boiler Room event, you can't help but feel like you're part of something breathlessly exciting. What's more, there are hundreds of small clubs like these across the country.

Those dark days in which the coolest thing to do was find a a bar in Dalston, surround yourself with minions and try to spot Alexa Chung are thankfully no more. The new clubbing vibe is urban, futuristic, but undeniably influenced by the mid-90s, the last great era of clubbing culture, which put an emphasis on the music. Like all great scenes it pays its respects, yet strives to do something new. It's historic and iconoclastic in equal measures, using new technology to create and promote itself, but totally getting what made things so great the first time around.

The underground also understands how to supersize with style. You only need to look at Manchester's enormous Warehouse Project event series to see how big and cutting-edge lineups can come together harmoniously. The temporary mega-club mixes superstars, legends and forward-thinking new talent in a way that few other events do. This year, they've put clubbing godfathers such as the Chemical Brothers and stadium fillers like Avicii on bills with up-and-comers like Eats Everything and Bicep.

Perhaps the best element of the new clubbing underground, however, is its diversity. The internet allows us to indulge our every cultural fetishism and clubbing brings those desires to life. I've been to Yorkshire bassline, indie, metal, drum'n'bass, gabba and psytrance nights for Big Night Out; there's a thriving night to satisfy even the most niche of tastes, and each one of them can be a success. I mean, I saw Arg and Mark Wright play at a shopping centre in Milton Keynes and that was still packed out.

Socio-economic factors are, of course, feeding into this boom (just as they did in Thatcher's Britain when acid house raves erupted around the M25). It seemed that at the dawn of the credit crunch, we all thought we'd be forced to sit in our living rooms listening to Swans and drinking Tennent's until some new Blair led us out of the darkness. But then people seemed to realise that that wasn't going to happen, that the jobs we were looking for weren't going to come, and that… who cares anyway? You could still enjoy yourself with enough JSA money for the night bus and a gram of MDMA.

That, when it comes down to it, is what is really at the heart of the UK's clubbing culture. Whether you're a goth from Doncaster or a Glaswegian gabba head, we all need a release. And as long as life's endless night out is always inconveniently interrupted by weekdays and responsibility, we'll always crave an outlet for escapism.

The UK club scene is in rude health right now, financially, culturally and spiritually. There's some great music about, and some terrible music too. But I'm not sure how much that really matters, because it seems to me that the soundtracks and the looks are purely incidental to the fact that Britain just needs to forget its problems on the dancefloor right now. I mean, what else are people going to do? Work?

Watch Big Night Out on noisey.vice.com