Chelsea Football Club are in talks to quit their 105-year old home at Stamford Bridge and build a ground on the site of the soon- to-be-demolished Earls Court exhibition centre to hold at least 60,000 spectators, the Guardian has learned.

The Premier League champions, owned by the Russian billionaire Roman Abramovich, are considering a move to the prime west London site just half a mile from their existing home amid growing concern they are losing ground to rivals with bigger and bigger stadiums.

Discussions have been kept secret because the move could torpedo a plan by the leading architect Sir Terry Farrell to transform Earls Court into a new residential enclave with more than 8,000 new homes. The scheme enters the latest phase of public consultation this week and is being undertaken with fellow landowners, Transport for London and the London Borough of Hammersmith and Fulham.

"The discussion is now on again," said a source close to the deal. "It is largely because the owners are progressing alternative uses for the site and there's lots more urgency for Chelsea to make a decision. From Chelsea's point of view this is their last opportunity to get a new ground and stay in the same area they have been in for over a century."

Chelsea flirted with acquiring the same site four years ago but talks came to nothing. Now the site is larger and Chelsea's chairman, Bruce Buck, has been warned the club faces a "deficit" as a result of Stamford Bridge's lack of capacity.

"There have been discussions about it and the club is clearly considering its next step," confirmed a source close to Chelsea, who added that negotiations are at an early stage and no deal has been signed.

The club has met the site's owner, Capital and Counties, in recent months and Chelsea and its advisers are holding "a series of key meetings to decide whether to pursue a bid or not", according to a source close to the talks.

A new stadium would not be ready until 2015 because Earls Court is scheduled to host the 2012 Olympic volleyball competition before the exhibition centre is demolished. After 73 years in which it has hosted gigs by the likes of Led Zeppelin, Oasis and Madonna, its economic viability has been compromised by the establishment of major new concert and conference venues elsewhere in London, including the 02 arena at the Millennium Dome.

Tonight Buck said it was "very difficult for us to make the philosophical decision that we are going to move on", but conceded that the lack of capacity at Stamford Bridge left it out of pocket compared with other clubs.

"Certainly we wouldn't leave west London or thereabouts and there are very few sites available," he said. "We have to do things with our other commercial activities to make up the deficit that is created by the fact we don't have a 60,000 seat stadium. We can't say that we will never move or have a new stadium but at the moment, it's not at the front of our agenda."

However, Chelsea insiders said Buck is keen to boost matchday takings because Uefa is introducing rules limiting the ability of super-rich owners to bankroll clubs without squaring spending with revenues. Despite winning the league last season, the club was only fifth in terms of average attendance in football's top flight behind Manchester United, Arsenal, Manchester City and Liverpool. Stamford Bridge accommodates around 41,000 fans compared with 76,000 at Manchester United's Old Trafford ground and 60,000 at Arsenal's Emirates stadium.

Tottenham Hotspur and West Ham United recently made bids to occupy the 80,000 seat Olympic stadium.

The emergence of Chelsea's renewed interest in Earls Court is awkward for Capital and Counties, which has launched a public charm offensive for its housing project employing Edelman, the international public relations company. It is promoting the "four villages and a high street" vision for the area and declined to comment on negotiations with Chelsea.

"Our vision for Earls Court is for a world class residential-led development delivering thousands of new homes and jobs, and creating a remarkable new place in London," a spokesman said. "As part of that we maintain discussions with a wide range of stakeholders and neighbouring landowners including both local authorities, TfL, the GLA and the local community."