When a taxi driver asks: "Did you see the England game last night?" you just know he is not happy. I was asked that question on Wednesday night, about to catch the train back to the south-west, when the cabbie launched into a scathing attack over England's performance against Wales on Tuesday.

Of course I can understand fans being unhappy that England did not crucify Wales but, as I told the Welsh lads at Bristol City the next day – Robert Earnshaw's miss aside – I can't recall Joe Hart being particularly stretched by the opposition. A win is a win, isn't it? During the match itself the Wales fans were ridiculing England's fourth place in the world rankings but, if you look at Fabio Capello's record in charge, it's pretty impressive. Under Capello England have lost only one qualifying match – away to Ukraine – and suffered only one home defeat – in a friendly against France.

Yes, fans want goals and action but, as a player, if you go on international duty for 10 days – and that's 10 days of hard graft by the way – and come away with six points, then that's job done. It might make England seem boring, an air of the old Arsenal about them, but for most of us it's results that matter.

It was interesting, then, to hear Capello talk about a "Wembley factor" having played its part in England's performance against Wales. Under Capello England have suffered only one home defeat, – in all competitions – so the results themselves would appear to suggest otherwise. But I always recall, on his initial appointment to the job, Capello talking about the mental weakness of English players, an area he felt needed improving. I'm not sure I totally agree – although I think teams can always benefit from the influence of sports psychology – but there is a pattern of media scrutiny and national expectation that England teams routinely have to deal with. One of the advertising slogans before the last World Cup was "England Expects". It says it all really, doesn't it?

To that it may be worth adding "Capello expects". From what I know of the man he wants a team to go and win 3-0, just as England did against Bulgaria on Friday night. For Capello a 1-0 victory is no way near good enough.

Still I wonder if the "Wembley factor" retains some sort of relevance. In the old days playing at Wembley used to be awe-inspiring. Driving in on the team bus you would catch a glimpse of the pitch before walking through corridors of memorabilia from the greats. There was a spot on the wall, just above the team bath, where Peter Shilton used to practise throwing the ball to warm up – a habit I have borrowed myself – and it was amazing to imagine him there. All those old photographs of players in the team bath, drinking champagne and celebrating. I sound nostalgic, and perhaps I'm showing my age, but that sense of history was deeply inspiring. Of course the new Wembley has immaculate facilities but it's more akin to walking through an airport terminal – decidedly clinical.

I can't help wondering if the foreign teams that come to play at Wembley feel the same way. In the old days the Wembley towers were iconic, intrinsically associated with football. Nowadays it is the England players – rather than the stadium itself – who fit that bill, global superstars that every opposition player worth his salt wants to beat. Such is the decline of the importance of the stadium itself that in a recent opinion poll it was a concert – not a football game – that was voted the most iconic event ever to be held at Wembley (in case you're wondering it was the band Muse).

But why should we rely on Wembley to inspire a nation anyway? In Italy and Spain the national team play in various stadiums around the country. Personally I have fond memories of the England Tour, as I call it, when we played at a variety of different grounds, from Manchester to Sunderland, while the new Wembley was being built. I think that period did much to galvanise national support for the team as fans who would never usually have had the opportunity to secure a ticket to Wembley suddenly had the national team playing on their doorstep. It was fantastic.

Imagine if that could continue in some way – it would provide a huge incentive for clubs like Bristol City wanting to build a new stadium if they knew they would have a chance of hosting an international friendly every few years. And then we could reserve Wembley for competitive games only – the really special occasions. Perhaps that way those dastardly polls will one day rank an England performance as the most memorable thing ever to happen under the famous Wembley arch.

David James has donated his fee for this column to charity