After 12 years riding a bike in London, I recently came to a shocking conclusion: what scares me the most as a cyclist is not the cars, not the lorries and not even the taxi passengers who unexpectedly wrench their door open in front of you – but other cyclists.

From busy roads to quiet estate pathways, canal towpaths to back streets, they move with astonishingly little regard for anyone else, leaving old and young, pedestrians and fellow two-wheelers spinning in their self-righteous wake. They buy a bike and they buy a bell, and they make me despair because everywhere they go they set the cause of cycling back.

If you consider yourself a cyclist and are about to head below the line or send a letter to the editor with a rage you haven't felt since the last pedestrian typing on an iPhone stepped right out in front of you without looking – hold off a moment. This isn't an attack on cyclists or cycling. Far from it. This is an attack on a kind of new boy racer who is giving cyclists a bad name.

Cycling is – rightly – booming: bike shops are the new coffee shops, springing up in huge numbers on every high street; in London the Barclays bikes have been taken up with enthusiasm; and the Tour de France, starting this weekend, will undoubtedly enthuse even more people to get on two wheels when it passes through Yorkshire. This is great ... but with all this comes increased congestion, and too many cyclists – who see this overcrowding as an affront to their freedom – react with a self-righteous fury and a "screw you" attitude.

You might correctly point out that cyclists are too often the victims of the road and that motor vehicles are still the main danger to life and limb for cyclists and pedestrians alike. According to CTC, the national cycling charity, in 2012, 98% of serious or fatal pedestrian injuries in urban areas were due to collisions with motor vehicles. And between 2008 and 2012, nine pedestrians were killed by cycles, while 1,361 were killed by cars.

But none of this excuses the breathtakingly bad behaviour of a significant minority of cyclists. No less an authority than Chris Hoy remarked on this in an interview for the Telegraph, saying that nothing winds him up more than bad manners on a bike. "When I'm out on a bike and I see someone doing something stupid I will absolutely have a word with them at the next set of lights," he said, adding: "There was a guy who was riding like an idiot, jumping lights, cutting up the pavement, and I just said: 'You're not helping matters here. If you want respect, you have to earn it.'"

Chris Hoy … 'When I’m out on a bike and I see ­someone doing something stupid I will absolutely have a word with them at the next set of lights.' Photograph: Danny Lawson/PA

If you travel in a city during rush hour, the chances are you will have seen someone on a bicycle pull a stunt that, had it been perpetrated by a car driver, would have seen them dragged from the wheel and strung from the nearest lamp-post. And you will have seen it time and time again.

This is why they are a danger both to the cause of cycling and to cyclists themselves. While none of us is perfect, these people slip into a realm where the rules of the road are irrelevant, and common decency goes out the window. They are the ones who are increasingly cutting us all up (pedestrian, car or cyclist), pushing us aside or screaming at us because we've slowed down to allow a child or older person live.

In fact, I'd say car drivers in general deserve more credit than they get from the cycling fraternity, because I don't recall ever seeing a car driver spot a space on a fully loaded zebra crossing that's a couple of centimetres wider than his car, put his head down, cross to the wrong side of the road, and bolt through at full speed. Nor do you see many cars flash on to a footpath behind a tight knot of pedestrians and proceed to ring their bell furiously until everyone jumps out of their way. And I probably don't even need to start talking about red lights, do I?

But the physical hazards posed by the new boy racers are not the real danger to cyclists – it's the damage to the cycling brand they constitute that's truly problematic.

To the harassed pedestrian for whom he tinged bell has become a tense precursor to either an earful of abuse or near-death experience, the moron has become the norm. The new boy racers represent all cyclists. And that's why they are really dangerous, because cycling needs to progress and be embraced, but the more badly behaved and unaccountable cyclists are perceived to be, the greater the likelihood that we'll be forced into restrictive and bureaucratic measures that set the whole cause back.

Compulsory insurance is one of those cure-alls for accountability that gets regularly trotted out, which is fine in principle, but although the person who just killed your dog might be able to offer compensation, you'd still have to catch them first. Registration is another oft-proposed solution to the problem. Japan is the only country that requires all bikes to be registered. Switzerland and others phased it out because of the cost and the bureaucracy. Now, the UK loves a bit of red tape, but registering every single bike? I don't think so. And apart from that, the freedom of cycling is a big draw to many of us. Take that away and it becomes another bureaucratically laden drag. So nothing would hold the cause of cycling back more than registration.

Interestingly, while we're on the subject of Japan, it has a large cycling population and many cycling laws – all of which are completely ignored. Cyclists regularly ride on paths and, indeed, police will even direct them on to walkways if they see them on roads. And yet cyclists, drivers and pedestrians get along fine. How does it work? In a word, politeness – one of those Japanese concepts with no direct translation into English.

The unfortunate truth, as CTC points out, is that congestion is only going to get worse and that we all – drivers and cyclists – have to meet in the middle and be considerate.

"If you go to Tokyo, one of the busiest cities in the world, you don't see the type of behaviour you consistently see in London. Everyone understands that the city is hugely busy and that they have to behave accordingly. We would like to see cycle awareness training as part of the driving test because everyone needs to understand the other point of view," says CTC spokesperson Jacqui Shannon.

More popular than ever … the commuter bike hire scheme in London has been taken up enthusiastically. Photograph: Nick Ansell/PA

The organisation would also like to see Bikeability – a 21st-century version of the cycling proficiency test – as part of the school curriculum. That would help with the next generation, but it's the current crop of speeding rudeboys high on a cocktail of fury and selfishness who are the problem, and they are unlikely to think they need any training.

But enough of regulation and training, what about the laws of the highway? To me, as a cyclist, I see a car coming and I am 99% confident that it will behave in a predictable manner, because drivers are trained to stick to the rules. Surely riders could do the same? Unfortunately though, when I see another rider, I have no idea if they are going to slow down, speed up, stop, run over a pedestrian and say it was my fault or do a wheelie down the wrong side of the road.

Last year, the Metropolitan police launched Operation Safeway in response to the deaths of six cyclists who were killed on the roads in London between 5 and 18 November. Officers manned major junctions and trouble spots and handed out a total of 13,818 fines – with 4,085 going to cyclists. Of course, a driver can do far more damage in a car – or worse, a lorry – than any cyclist, but that's still a lot of cyclists jumping red lights and whizzing down footpaths. Besides, the process of holding drivers to account is easy because you have a means of identifying the miscreant and catching up with them at a later date. However, an officer witnessing a dangerous cyclist can do little more than shake his fist in their direction.

I wouldn't call myself a cyclist with a capital c, but I know many of them, and they're a sensitive bunch. So I'm writing this anonymously because I've had enough of being intimidated on the roads as it is. Maybe it's time for me to put the bike away and start walking instead.

But don't let me stop you. If you're caught up in the cycling boom and are tempted to join in the revolution, go ahead. Enjoy Le Grand Depart this weekend; get your bike and your bell and your Lycra (if you must), but don't forget the most important piece of equipment: a brain.