What happened at Wembley and on the streets of Newcastle over the weekend was shocking - but it doesn't represent a return to the dark old days of football hooliganism.

High-profile outbreaks of violence involving fans are much rarer today than they were 20 or 30 years ago. The scale of trouble now compared to then doesn't bear comparison - either in terms of the number of people involved or the level of organisation.

Football has moved on thanks to banning orders and better, more sophisticated policing.

And while it is too simplistic to say that the higher cost of watching football has pushed unsavoury elements out, there has been a shift in the way people are expected to behave inside grounds. Offensive chants are still way too commonplace but actual fighting doesn't happen very often.

Viewpoint: Millwall violence Phil McNulty Chief football writer "It is a rare occurrence to see fans fighting each other in British football stadiums these days - and these were Millwall supporters fighting their own in the worst episode of violence witnessed at the new Wembley. "While it was not quite a return to the darkest days of the past, there were images that could have been plucked straight from that era." Read more of Phil's comments

It would be incredibly naive to think it doesn't still go on, most frequently away from grounds. Ask some supporters, in fact, and they will tell you the menacing presence of thugs intent on using football as a cover for causing trouble has never gone away.

So was this weekend just another reminder that the game shouldn't be complacent or are there deeper, more significant lessons to be learned?

The first point to make is that, while both incidents looked terrible for football, they were very different. One inside the home of English football, the other in the centre of Newcastle.

I was at Wembley on Saturday, having bought a ticket for the Millwall end from the FA. Although I rarely go to the Den these days I have followed the club either as a fan or as a journalist since the late 1980s.

The first I knew of the worst of the trouble - which had been taking place on the other side of the stadium to us - was when a relative texted me as we were walking back to Wembley Park Tube station to ask whether we were OK. I had been too busy watching the game to notice the fighting - broadcast live by ESPN - which had been happening away from us.

Having said that - and with the value of hindsight - there was unquestionably a menacing edge to the crowd, which I hadn't experienced for a while and certainly not at previous Wembley games involving Millwall.

Just behind us a scuffle broke out towards the end of the first half which seemed to take forever to sort out. I tried to keep my eyes on the action but it was hard to ignore as, all around me, people were pointing and watching what was happening.

Others were looking to see if they could get involved and for a moment it felt like it was going to spark into something much more serious. Eventually stewards stepped in and removed one or two fans but it was still lingering at half-time when I noticed a huge escalation in the number of police on the concourse and around the bars and food outlets.

I wondered whether a couple of Wigan supporters had got into the wrong section of the ground, or whether fans from another club had managed to get some tickets in the hope of starting trouble.

Football violence in England and Wales 2011-12 Arrests at international and domestic games - 2,363 (down 24%)

Average arrests per match - 0.72

Arrests inside stadiums - 1,215

Arrests outside stadiums - 1,148

Banning orders - 2,750 (down from 3,173) Source: Home Office figures gathered by the Office for National Statistics (ONS)

But, sadly, fights between Millwall fans are not uncommon. I have seen it a few times down the years and often it's over the smallest of things. But in a charged-up atmosphere those small incidents can quickly escalate as other fans looking for trouble get involved.

The next step is usually an intervention by the police which can take it up another gear as innocent people get mistakenly drawn into the fray. Having re-watched footage of Saturday's trouble that looks to be the case here, although it is still too early to tell.

Nothing should be ruled out but my sense is that if rival fans from another club had been involved this would have been much bigger and much harder to bring under control.

I have heard and read various theories. Frank Maloney, the boxing promoter and lifelong Millwall fan, wrote in the Telegraph external-link that the trouble started after a young girl was pushed over by a drunken fan. Maloney says that when he refused to apologise to the girl's father a fight broke out and it went from there.

Another source told me that the confrontation possibly started because of a disagreement between members of rival criminal families. While this might sound a bit too Sopranos, it is actually highly plausible for a club with deep roots in some of the toughest parts of south London.

Whatever the flashpoint, here are some inescapable facts which Millwall and the FA will have to now grapple with.

The first is that the 17:15 BST kick-off time meant there were a lot of fans who were absolutely paralytic by the time the game started. There have also been reports of drugs use - something I suspect is right.

TV dictating kick-off times is all part of modern football but there have to be times when high-risk games like this one make security a priority. An earlier start-time might have reduced the potential for trouble. Closing the bars in the ground might have also helped, although I suspect that could have made matters worse.

Police say footage is being studied to identify the trouble-makers

Putting the kick-off time to one side, security seemed unusually relaxed for such a big match. A lot of drunken fans should have been turned away. And supporters were not searched properly going into the ground. I walked in with a large rucksack and, despite clearing three lines of stewards, not a single one asked me to open it.

The second point was the lack of policing and stewarding inside the ground. While it is right that young stewards on low wages and with no experience of controlling dangerous situations should not be asked to intervene in outbreaks of serious disorder, where were the police or the more senior stewards?

While not excusing the perpetrators, the trouble could have been brought under control quite quickly had the police moved faster and dealt with it in the right way.

The last point is one which may take longer to get to the bottom of. Are police and magistrates becoming less committed on the enforcement of banning orders, allowing well-known thugs to slowly return to football?

A large number of Millwall fans were issued with 10-year banning orders in 2002 following a major riot after a play-off game against Birmingham at the Den. Those will now have lapsed and the question for the authorities is whether banning orders should be issued for life, not just for a decade.

Some in football are also worried that when serious incidents of violence do occur, magistrates and judges are not cracking down on hooligans seriously enough by issuing the banning orders in the first place.

This might explain the apparent anomaly with the number of arrests and banning orders in English football going down over the last few years, even though there are still reports of trouble at and around games.

Which brings me to the clashes involving Newcastle fans following the Tyne-Wear derby on Sunday. Twenty-nine people were arrested and Newcastle said they were "embarrassed" by the violence which followed the club's 3-0 defeat at St James' Park.

As with the Millwall case, the exact circumstances are still being looked into. But again questions have to be asked about the policing and security arrangements.

Police rightly predicted the game would be a flashpoint and warned fans beforehand. But they still weren't able to prevent the disorder which even saw one fan squaring up to a police horse.

"There's always going to be a minority that spoil it for people and that's what happened on Sunday. "They get caught up in this testosterone-built atmosphere for violence." Steve Wraith, Newcastle supporter "There were a lot of people who were drunk and there was drug element. There were bits of tension. It wasn't dealt with - it certainly wouldn't happen at our ground and I don't think it would happen at any other ground. "It beggared belief that this was occurring and there was no action being taken by the authorities." Bob Asprey, Millwall Supporters' Club chairman

Were there enough police on duty and were there fans involved who were known troublemakers or subject to banning orders? If there were, why were they allowed to congregate in the city centre, so close to St James' Park?

Perhaps one answer lies in a recent test case which saw West Yorkshire police lose to Leeds United on the question of who should pick up the tab for securing the areas and streets surrounding football grounds.

The police argued that Leeds should pay for those costs but the Court of Appeal dismissed their case. Police forces around the country now know beyond any doubt that they are ultimately responsible for clearing up and paying for football's mess when it spills over into areas around stadiums.

Is it possible that, at a time of cuts, fewer resources are being devoted to keeping football safe?

Whatever the reasons for the two nasty incidents we saw at the weekend, they serve as an important reminder that for all the progress English football has made in the last 20 years, there is still a sizeable element who wish to use the game as an excuse for causing mayhem.

The game must seize this opportunity to ram home the message that they aren't welcome.