In London's Burlington Gardens, centre of moneyed Mayfair and arguably the epicentre of old-money Britain, we are being gazed down upon sternly by Gottfried Leibniz, the philosopher-mathematician who developed infinitesimal calculus, by Georges Baron Cuvier, the groundbreaking anthropologist, and by Carl Linnaeus, the botanist who founded binomial nomenclature. Bright guys, even if their busts at the back of the Royal Academy of Arts don't necessarily portray them as a barrel of laughs.

Their stone eyes seem narrowly focused on a semi-naked man, his torso honed and buffed as he pouts and sways just inside the darkened doorway of the London flagship store of Abercrombie & Fitch, while he has his photo taken with and by gaggles of Euro-students.

Inside, music softly thuds, subtle enough at the moment not to trickle too far past the doors behind naked-boy; the lighting is low, louche even, the ambience somehow reminiscent of one of the more tasteful periods during the last days of ancient Rome. Gaze up, and a huge stars and stripes flutters from a proud flagpole. Nothing wrong with that, surely; just around the corner in Bond Street is the bronze, rather lovely statue of Churchill and Roosevelt laughing on a bench: it's entitled Allies. The flag's not the problem: this is. Gaze to your right, and you glimpse the very beginning of Savile Row. This is the problem.

Until now, with A&F technically just out of sight of most of the Row, the proud old tailor shops could just about pretend it didn't exist along with its loudly branded sweatshirts, its Eurotrash and its queues. They were able to get on with what they do so well – making bespoke suits and being slightly scary.

But the problem is that they can't ignore the brash Americans any more. A&F recently lodged plans to open a children's store at 3 Savile Row; in plain, horrifying, screaming-spoilt-brat view of the quietly-spoken men with the pins and the worsted. Worse, number 3, as the one-time HQ of Apple (there was once another company called that, youngsters), was where the Beatles played their famous rooftop last gig. Modern heritage, but heritage nevertheless.

So the good tailors are objecting. The chap from Gieves & Hawkes told Westminster council's planners that the retail development was "totally out of character". David Coleridge of H. Huntsman & Sons wrote that "the arrival of Abercrombie & Fitch at the end of Savile Row has dramatically changed not just the tone, but the safety, of the street". One tailor was able to be less pussyfooted by being quoted anonymously: "I don't think anyone objects to moving forward, but a chain store selling crappy clothes to ghastly people isn't really the direction in which we should be travelling."

So, is A&F really that bad? I went in. It's almost tasteful, much back-lighting and soft smells, particularly around the aftershave bit, but pretty loud. A couple of pretty staff are gyrating on the top-floor balcony to a number of songs about being beautiful and one called I Rock, I Sweat, I Dance. Hordes of, basically, rich European students are fingering and mussing up walls and trays of pastel clothing. Apart from two of the good-looking staff, there is not a black face in the store, and I'd guess only two people over 40. Most shoppers are French, Italian, German, and already good-looking and well-dressed enough not to need any other clothes, ever.

The clothes aren't, actually, crappy. They're very preppy – sweatshirts, polo shirts, fleeces, boxers, cardigans to be flung over well-bred shoulders à la Jay Gatsby, though these are soft and cuddly as opposed to the itchy garments of the 30s. They look well made, a good few in China. They are also, and this is a surprise to me for a chain, eye-wateringly expensive. The A&F "sell" might be happily unfettered by such restrictive critical considerations as, for instance, taste, but they're not above adopting old-school Savile Row prices: £80 for a shirt; £200 for a child's sailing windcheater. The rich kids are grabbing armsful, and I wonder anew about the Euro-crisis, though it does explain why there are no Greek voices.

Outside, Mathilde from Bavaria, one of the brighter and less sulky Euros taking photos of friends outside the front door, is mystified by any row. This A&F will be her first port of call again when she returns to the city during the Olympics. "It is expensive, yes, but there is style, and quality. We love it, love going in," she says. Did she see no problem in wearing a big shouty shirt advertising the company where she bought it? She looks at me as if I've handed her a cormorant. "That's the appeal. At home, all our friends know where we've bought it," she exclaims.

And there's the rub. A&F has utterly sewn up branding heaven; it can get youngsters to pay through the nose to further advertise the brand for free, to show their fathers have money. And, thus, this phenomenally successful company can just possibly get by without worrying about my taste, or whether I actually like it; or, quite probably, if they win the planning decision, what Savile Row thinks of them. The money will be made, the further expansions come – already, in neighbouring Regent Street, there's a 30-metre billboard featuring "bare men" to advertise the taste and style of A&F.

Actually, for what it counts, I don't think I do like A&F very much. It bought the name of a fine old American proper hunting company which went bust in the 70s, jammed both feet in the door during the dawn of branding and made fortunes by never overestimating the intelligence of the pack-instinct American teenager. Staff at its Milan store were until recently forced to do 10 press-ups if they hadn't been "cheerful" enough. It once sold a woman's T-shirt emblazoned "Who needs a brain when you've got these?"

It should care about one thing. The problem, actually, is physical. It's outside the store. The young Italians, as ever, are the worst. They do that thing they do when their tour buses pull up in cities of all clustering heedlessly together, blocking the pavement with their hair and gestures and sulks and phone-cameras and making you step in front of a hurtling taxi: I can quite see the point about safety. And surely the Savile pavement of the kids' store (if allowed) would be similar if not far worse. Until Westminster Council can pass a bye-law stopping sulky ox-people gangling together and sprawling on to the road with no perception of the existence of other humans, this application should surely be put back on the rack.