Sometimes, when flying over a landscape, you see a seam of unexpected fecundity – lush trees, richer green – that indicates the presence of water or a change to a more fertile soil. Something similar is happening across London. If property values could be made visible (and often they are, by increases in new construction), you would already see a long strip of intensification, in a city that already is hardly a desert, running from east to west. Over the next few years, it will become more and more apparent.

This is the effect of the underground Nile called Crossrail and it will show what happens when £14.8bn of public money is streamed underground in order to irrigate a city and its development opportunities above. Its current signs include diversions, closed roads, trucks of dirt scaring cyclists, references in estate agents' particulars, fluorescent-suited workers, hoardings that give no clue to the chasms behind them, informative graphics and the saturated light that shines in computer-generated images of future buildings. Also such things as a once-ramshackle city farm in Stepney, east London, now spruce and confident, various redecorated community centres and support for a literary festival in Islington. This is due to a programme which obliges Crossrail's contractors to make donations to the communities in which they are working.

Whether Crossrail is value for money is disputed or, according to one expert, unknowable. What it is is a line or, rather, a stretched, horizontal letter Y that runs from the gin-and-golf territories of Maidenhead in the west, through Heathrow airport, and then the dense mass of underground tunnels that already serve the West End and the City of London. It continues east, nourishing property development as it goes, and bifurcates beneath that city farm. The northerly arm goes on past 2012-land, stopping at Stratford, the place where in the name of regeneration almost every new London railway in the last two decades has been obliged to call, before heading off to the Essex suburb of Shenfield. The southern arm burrows under the Lehman-haunted blocks and towers of Canary Wharf, crosses the Thames and finds a place, Woolwich, that benevolent transport initiatives have until now mostly forgotten.

This branch ends at Abbey Wood, in a part of south-east London whose hills and woods, if it were not also the most forgotten corner of the capital, could almost be as famous as Hampstead's. Abbey Wood is also close to Thamesmead, the 1960s utopian-dystopian new town (they filmed Clockwork Orange there) that, along with Canary Wharf, the Olympic Park and Heathrow airport, makes the Crossrail route into a tour of government-encouraged megaprojects.

It is a heavy railway running through the middle of a city whose 200m-long trains will carry up to 1,500 passengers, which is nearly twice as many as existing tube trains, 24-30 times per hour. This means that, if you stand on a platform in, say, Bond Street at a busy time, the contents of a Premier League football stadium could go past you in 60 minutes. Crossrail is 118km long, partly with the help of reusing existing track, but also with 42km of new tunnels. It promises to increase London's public transport capacity by 10%, and dramatically reduce journey times: 26 minutes from Bond Street to Heathrow, seven minutes from the City of London to Canary Wharf, 40-50 minutes from its extremities in Essex and Berkshire to the West End.

It comes with obligatory superlatives. It is the largest infrastructure project in Europe, costing more, for example, than the London Olympics. Less expectedly, it claims to be the largest archaeological site in Britain, an inadvertent probe through a plague pit, a Roman road, a madhouse cemetery, a Mesolithic "tool-making factory". It is, apparently, the largest art commission in Britain: Crossrail is an urban and a cultural event, as well as engineering.

"Because we believe," goes the official explanation, "in the value of art and architecture, place-making and cultural identity, we have established the Crossrail Art Programme, bringing together world-class architects and designers, business-world sponsors, artistic creativity and local identity." Partnerships have been formed with some of London's most famous galleries: the Lisson, White Cube, Victoria Miro, Sadie Coles and Gagosian. One of the resulting works has so far been revealed: Cloud Index by the American artist Spencer Finch. Here, images of clouds, "in the tradition of Turner and Constable will be embedded into the 120m-long glass canopy of Paddington's new Crossrail station, with 25 different types identified".

Above ground, Crossrail is a cross-section through steep gradients of wealth and property, suburb and city, world-famous cityscape and obscure brownfield. Below, such distinctions disappear. It is a uniform republic of artificial light, grey-brown mud dotted with orange workers and puddled here and there with water. There is precision and roughness, both engineered machinery and expedient temporary structures of nailed-together planks. After grinding through virgin muck for several miles, the tunnels arrive within 5mm of their intended position. At Tottenham Court Road, Crossrail's construction had to pass within 80cm of the Northern Line's tunnel, a feat that in engineering terms involved both the accuracy and potential for disaster of William Tell's thing with the arrow and the apple.

The vast cavern beneath Stepney Green where the line divides into two spurs, to Abbey Wood and to Shenfield in Essex. Photograph: Courtesy of Crossrail

Everyone has been drilled in safety procedures and wears the helmets, glasses, jackets, gloves and boots now obligatory on building sites, while also carrying metal tags, as miners do, which means they can be identified if incinerated. For additional insurance, the entrances to excavations are watched over by statues of St Barbara, patron saint of tunnelling. Workers won't go in unless tunnels have been blessed.

The stars, underground, are four pairs of 1,000-tonne tunnel boring machines, or TBMs, which work almost nonstop. They are like long subterranean ships, steel structures with gangways, machinery and control rooms, together with canteens and lavatories, compactly arranged into just-enough space. They also resemble worms: they chew earth, excreting it along their 140 metre length into conveyors that carry it to the surface. Three-quarters of it is then transported by truck, rail and river to form a new nature reserve at Wallasea island in Essex.

Just behind the chewing part, three-tonne segments of concrete are vacuum-lifted into place to form the circular tunnel lining. A man deftly finishes the job, flicking and turning a huge bolt with more ease than one might turn an Allen key on an Ikea table. Unlike an Ikea table, the results of his work will last for centuries – the minimum design life is officially 120 years. The German-made TBMs cost £10m each and are part of a global economy of digging machinery. They get taken apart, shipped and reassembled in other countries, used again and again over years in countries all over the world. The Crossrail ones are new. They also, like ships, have been given women's names, including Elizabeth and Victoria after the queens, and Jessica and Ellie after the beloved Olympians. They move at speeds of up to 260 metres per week, and 100 metres per week on average, subject to the state of the ground.

Their route has been investigated in advance for obstacles such as sewers and electrical cables, between which the huge machines have to thread their way, but they are also preceded by sensors in case of surprises. At Canary Wharf, they unexpectedly met some concrete test piles inserted in the early days of the development's construction and then forgotten.

Their crews work briskly, in eight-hour shifts, incentivised for speed. Many are in their 50s, men from coalmining communities with Welsh and Yorkshire accents who, since the closing of the pits, have moved on to the Channel tunnel and the Jubilee line extension. They are well paid, I am told, for their skill and for the deprivation of spending so much of their lives cut off from the sky. Miners, I am also told, make the best gardeners, as they want to spend as much of their leisure time as possible in the fresh air.

The grandest part of this underworld is at Stepney Green. Above ground, through half-closed eyes, you could imagine yourself in Somerset, with the medieval church of St Dunstan, ringed by mature trees, and foregrounded with the farm's sheep and chickens. If you turn around, Crossrail's gantry crane, which among other things lowers and raises bits of the TBMs, punctures the illusion, but even this does not prepare you for the huge shaft dropping into a huger pair of brown caverns below ground. It is one of the the largest such spaces in Europe, apparently. This is where the eastern arms of the Y join, which means that for a while the space has to be the width of two tunnels plus a bit extra, and, since its cross-section is circular, its height increases with its width.

In its underground-ness and abundant volume, there might be some resemblance to a Bond villain's lair or an Indiana Jones treasure chamber, but it's superficial. The atmosphere is calm and methodical, of routinely performing tasks that have been done thousands of times before, and with the fundamentally simple aim of getting from A to B. If engineering is sometimes called an adventure, down here every effort seems to be made to neutralise it. For very good reason, I imagine: over-excitement is not conducive to safety or getting the job done. Even above ground, when talking to the chief executive of Crossrail, Andrew Wolstenholme, in his Canary Wharf office, I find he speaks in risk-managed sentences, their content differing not much from the organisation's press releases and website.

He tells me that construction work is half complete and that the project is "absolutely within its funding envelope" and on target for its phased openings in 2018-19. "Every train will arrive all the time on the time." The project will generate "£42bn of economic value". Ninety-five per cent of a "Crossrail pound" is spent with British-based companies, 60% of them small or medium-sized, and 57% of them in the regions. (In Sheffield, an entire station has been prefabricated for erection in London.) It will be a "world-class railway" with a "legacy of skills and training", which will also "leave a healthier and safer industry". Its procurement of contracts has been a "fantastic achievement", with "not a single legal challenge".

It is, he says, an example of a relatively new-found ability for large British construction projects to run smoothly. Not that long ago, it was taken for granted that the budgets for projects such as Crossrail would be vaporised by overruns and that completion dates were works of fiction. Heathrow's Terminal 5, bar some soon-forgotten glitches with baggage handling, helped to change that perception, as did the Olympics, even if the latter's claim to be on budget was helped by the early more-than tripling of the fancifully low figure set when London was bidding. In the case of Crossrail, the project seems to be proceeding with a remarkable lack of disturbance. "Do not underestimate UK plc's ability to deliver such projects," pronounces Wolstenholme; Britain "has developed a skill set and a reputation second to none".

Tunnelling machine 'Elizabeth' breaks into the specially built chamber at Stepney Green earlier this month. Photograph: Courtesy of Crossrail

Projects this big make their own universe, including nine new stations, and new developments that exploit the increased land values that go with Crossrail's arrival. Also the Tunnelling and Underground Construction Academy, or Tuca, a large hangar-like building in Ilford, Essex, the majority of whose cost has been paid for by Crossrail. Here, in a facility that "doesn't exist anywhere else in the world", full-size mock-ups of tunnel building sites are created, where noise and hazards are simulated. They have a large robotic machine for spraying concrete, which Boris Johnson, never shy of either toys or photo-opportunities, has test-driven. Tuca's purpose is to train new engineers, to expand the workforce and replace, when they retire, the fiftysomethings who currently make up much of it.

It is built in the belief that there will be more tunnels to be dug in the indefinite future. Those involved believe the skills will be needed on HS2, the high-speed rail route from London to the north, and on Crossrail 2, which would run north-south across the capital. The need to sustain this successful industry becomes part of the case for further huge rail projects, which begins to suggest a backwards logic: big things must be built to keep busy the people who build them.

The stations, according to the ubiquitous lingo of projects such as this, are "world class" and Wolstenholme says that "we hold design very high up in our value set". Crossrail is acutely conscious of the precedent of the Jubilee line extensions that, under the leadership of Roland Paoletti, who died recently, were much admired for their architecture and won several awards. They were also over budget, for reasons that were disputed.

The Crossrail stations have been designed more than once during the long gestation of the project. The versions that will be built will be a touch more cautious than those built under Paoletti's reign – there won't be quite as much of the plunging drama of the Jubilee line stations at Westminster or Canary Wharf, or the heroic cast-iron and concrete details of those works, although the long glass canopy at Paddington will not be exactly modest. The most expressive architecture will be found at Whitechapel, seemingly to honour the area's exciting vibrant diversity, which is probably a mistake: the station's multiple levels would have enough drama without the swooping, curving roofs and straining metal struts, which the architects have felt obliged to add to the design.

With the stations goes 3m square feet (or about six Gherkins) of commercial development, spread across the capital and managed in partnership with property developers, which are mostly in the gridded style of rectangular windows and straight pillars and lintels currently favoured by the more respectable end of the business. Specifications of the cladding materials vary depending whether you are in Woolwich or Bond Street, while in Soho the developer Derwent London will make its building swing a bit, with shiny black, gold and ripples of multicoloured strips. The architecture lets its hair down a bit more with a crystalline, gold-clad mountain of flats in Holborn and the Goslett Yard theatre, a cantilevered glass ziggurat near Tottenham Court Road. At Canary Wharf, a shiny cylinder designed by Foster and Partners, which is supposed to look like a ship in the dock, will surmount several layers of subaqueous retail.

Beyond the commercial development on Crossrail's own territories, there will be yet more in areas surrounding the stations, initiated by others. An effect of Crossrail, combined with the earlier Thameslink project, will be to make Farringdon the only station with direct connections to four of the city's airports. Farringdon was formerly a lull in central London's rush, the dreamy volumes of whose Victorian station served one of the least useful stops. Now it is intensifying, with Amazon and Goldman Sachs moving into the area.

At Whitechapel, the borough of Tower Hamlets has sponsored a masterplan whereby the station's former backlands will be made into a mini-Dubai, with a mini-skyscraper, which seems positively to welcome the sort of trashy architecture that generally needs no encouragement. In unsung places such as Hayes and Abbey Wood, the effect will be profound.

Then, finally, there is the 1m square feet of public realm that comes with Crossrail: squares, plazas and pedestrianised streets designed mostly by the large engineering consultancies to whom our cities' open spaces are now entrusted. At Paddington, Tottenham Court Road, Liverpool Street and elsewhere, they convert places previously full of traffic into zones distinguished by what can now be recognised as Crossrail's default style: responsible, uncontroversial, a bit arid. An expanse of paving next to Centre Point looks particularly desolate, but it will be an improvement on the big roads that were there before.

Crossrail, then, is a mighty work of engineering more dramatic in its making than its completion. It doesn't have the romance of a Channel tunnel or a high-speed train, its model being the not-especially-sexy Parisian RER regional transport system. It is most significant as an urban engine, a device to serve the continuing expansion of London's population and the city's continuing success as a vast re-circulator of money. It has got to this point with a relative lack of controversy. There have been objections to the fact that not all its stations will be wheelchair-accessible and the Financial Times has argued the government missed an opportunity to harvest the increases in property values around stations, with the result that private landowners get windfalls and the public purse gets less return than it might for its investment. There were disputes about the locations of some of its holes and spoil heaps, which sometimes resulted in victories for the objectors.

More serious is the fact that someone has to pay, sometimes those who do not obviously benefit. Business rates are going up for large and small companies, whether or not they gain from the minutes saved travelling to Heathrow or Canary Wharf. Almost any improvement to British cities means gentrification, which means unequal effects for those communities in whose name such projects tend to be promoted. National government has so far lacked the will or wit to make places better without also making them less affordable.

It has also been asked whether Crossrail is the best way of spending nearly £15bn on the country's infrastructure. It doesn't connect with other recent transport investments such as the Eurostar station at St Pancras and, somewhat bizarrely, Terminal 5 at Heathrow. It doesn't necessarily serve the areas of greatest expansion around London.

These oddities reflect the fact that it has been extraordinarily long in the planning. Versions of it were proposed in the 19th century and in the 1940s and the 1970s, before being put forward again in 1989. Parliament deemed it too expensive in 1994 and it was nearly frozen again in 2008, when the looming financial crisis made such huge expenditure look like a bad idea. It is rumoured that it only got through because the then transport minister, Andrew Adonis, also an enthusiast for HS2, slipped it under Gordon Brown's nose when the prime minister was distracted by thoughts of holding a snap election. According to Tony Travers, an expert in London planning at the London School of Economics, "putting the funding together took years and required valiant efforts by London First and the City of London Corporation. The government had to be lobbied for nearly two decades."

Travers adds: "Crossrail is an example of the odd way we plan projects. In brief, either you get this one big project or the money disappears. There is never an alternative use for the public resources. It is very hard to be sure about value for money. Having said that, major railways in London or the south-east are almost always going to give better cost-benefit figures than the same project anywhere else in the UK." In other words, Crossrail is being built not because it is definitely the best way of spending the money, but because it was so large and so persistently put forward that the government grew tired of saying no.

Jessica and Ellie don't know all this, as they bore inexorably on, and their crews probably don't care much. There is a marked contrast between the certainty and clarity of purpose underground and the confusion that swirls on the surface. But, when it is completed, almost no one will care greatly whether it is value for money, unless something goes badly wrong. As with the Olympics, the sheer impressive fact of its existence will cause sceptics to forget their doubts.