Huddled together in the chill January wind, the players listened as a PPE fresher in a black cape read the rules of the game: a Quaffle through a hoop would score 10 points, capturing the Snitch would yield a bountiful 30, and under no circumstances was there to be any "grabbing of broomsticks". With that, they were off: two teams, with seven players each, racing round a playing field and trying to shoot a basketball through hula-hoops.

To onlookers it may have seemed outlandish and bizarre, but to these mostly teenage Oxford students it was the realisation of a dream. For Quidditch, the game they grew up reading about in the pages of Harry Potter books, is no longer a fictional activity played by witches and wizards in the air. It is a fast-paced and disconcertingly rough team sport that is played firmly on the ground and results in very real cuts and bruises.

"It's fantastic," said Amy Wipfler, a 20-year-old Californian and passionate Harry Potter fan studying at Oxford on her junior year abroad. "All of a sudden you meet these people equally as enamoured with [Harry Potter]," she said. "That, in a way, is what's magical because all of a sudden you all have that same drive to want to be part of Hogwarts ... Everyone secretly wishes it was real."

Not everyone playing in Saturday's mini-tournament – which, with four college teams and plenty of extras, was the biggest that the university has seen since the sport's arrival in November – shares her love of JK Rowling, however. Angus Barry, founder of the first college team at Oxford, said that although a fan he had made an effort to make people "see Quidditch as a sport in its own right".

He added: "Some people really do love Harry Potter and that's why they're here. Other people just like the game."

Known as Muggle Quidditch to those for whom JK Rowling's lexicon is as familiar as any entry in the dictionary, the game was adapted for non-wizards around seven years ago in the US, where it has since caught on and become a familiar pastime for students at some of the country's best-known institutions, including Yale, Harvard and Tufts (Wipfler's college). Instead of flying, players run with broomsticks between their legs, and instead of a golden ball with wings attached, the Snitch is a person dressed in yellow. Although tackling is frequent and being hit by a volleyball, or "bludger", is likely, the "spirit of Quidditch" is encouraged. As one player for the University college team put it: "If you're massive and there's a little person, don't run into them."

Despite its success in the States and Canada, Muggle Quidditch has been slow to catch on in the home of the Harry Potter stories, although it has been attempted by students at universities including Nottingham and Warwick and there has been talk of setting up a British league.

Barry and his fellow players hope that its time has finally come. "The word is spreading," he said. "I think each time we've played we've pretty much doubled our numbers. We started off with 20, then 40, then 80." Many of the players would like to see Quidditch recognised as an official university sport, a step that would open the way to a Varsity match against Cambridge and a Half Blue, such as Ultimate Frisbee and Eton Fives.

But that, said Barry, remains a "long-term plan". For now, the sport is happily unrecognised and unregulated. Elbows are grazed; spray-painted broomsticks are broken. But the mood remains defiantly buoyant: as John Waite, a 19-year-old Material Science fresher, pointed out, it's hard to take yourself too seriously with a broom between your legs. "The comedy element ... gives it a novelty factor which makes it a lot of fun...You don't take it too seriously. It's just a lot of fun to play."

Wipfler, who decorated her broomstick with sequins and named it the Pink Panther, is unsure about the future of the sport. "I think it has potential, but the fact is if you haven't read the books you're not going to have the same love of the sport," she said. But, for the moment, it is only becoming more popular.

Rowena Francis, a Philosophy and German student from Croydon, was one of many first-time players and, to cheers from St Hilda's, she caught the snitch. After 90 minutes, half-a-dozen matches, and one cut lip, the tournament was brought to an end. With balls under arms and brooms in hand, many retreated, happily, to the pub.