During the summer of 1913, in a field just south of Cairo on the eastern bank of the Nile, an American engineer called Frank Shuman stood before a gathering of Egypt's colonial elite, including the British consul-general Lord Kitchener, and switched on his new invention. Gallons of water soon spilled from a pump, saturating the soil by his feet. Behind him stood row upon row of curved mirrors held aloft on metal cradles, each directed towards the fierce sun overhead. As the sun's rays hit the mirrors, they were reflected towards a thin glass pipe containing water. The now super-heated water turned to steam, resulting in enough pressure to drive the pumps used to irrigate the surrounding fields where Egypt's lucrative cotton crop was grown. It was an invention, claimed Shuman, which could help Egypt become far less reliant on the coal being imported at great expense from Britain's mines.

"The human race must finally utilise direct sun power or revert to barbarism," wrote Shuman in a letter to Scientific American magazine the following year. But the outbreak of the first world war just a few months later abruptly ended his dream and his solar troughs were soon broken up for scrap, with the metal being used for the war effort. Barbarism, it seemed, had prevailed.

Almost a century later, a convoy of air-conditioned coaches sweeps through the affluent suburb of Maadi – where Shuman had demonstrated his fledgling solar panels – continuing south for 90km towards Kuraymat, an area of flat, uninhabited desert near the city of Beni Suef. The high-level international delegation of CEOs, politicians, financiers and scientists has come to visit a brand new "hybrid" power station that uses both natural gas and solar panels to generate electricity. Before the coaches reach the facility's security gates, its 6,000 parabolic troughs – each six metres tall with a combined surface area of 130,000sq metres – are already visible from the perimeter road. Even though the panels account for just one seventh of the power plant's 150MW generating capacity, the Egyptian government, which has been pushing to develop the site since 1997, hopes to prove to the delegation that it is the desert sun – not fossil fuels, such as gas, coal and oil – that should be used not only to generate far more of the electricity across the Middle East and North Africa (Mena), but, crucially, for neighbouring Europe, too.

Gerhard Knies, a German particle physicist, was the first person to estimate how much solar energy was required to meet humanity's demand for electricity. In 1986, in direct response to the Chernobyl nuclear accident, he scribbled down some figures and arrived at the following remarkable conclusion: in just six hours, the world's deserts receive more energy from the sun than humans consume in a year. If even a tiny fraction of this energy could be harnessed – an area of Saharan desert the size of Wales could, in theory, power the whole of Europe – Knies believed we could move beyond dirty and dangerous fuels for ever. Echoing Schuman's own frustrations, Knies later asked whether "we are really, as a species, so stupid" not to make better use of this resource. Over the next two decades, he worked – often alone – to drive this idea into public consciousness.

The culmination of his efforts is "Desertec", a largely German-led initiative that aims to provide 15% of Europe's electricity by 2050 through a vast network of solar and wind farms stretching right across the Mena region and connecting to continental Europe via special high voltage, direct current transmission cables, which lose only around 3% of the electricity they carry per 1,000km. The tentative total cost of building the project has been estimated at €400bn (£342bn).

Until now, Desertec has been seen by many observers as little more than a mirage in the sand; the fanciful plan of well-meaning dreamers. After all, the technical, political, security and financial hurdles can, each on their own, appear to be utterly insurmountable. But over the past two years, the initiative has received significant support from some of the biggest corporate names in Germany, a country that already leads Europe when it comes to adopting and developing renewable energy, particularly solar. In the autumn of 2009, an "international" consortium of companies formed the Desertec Industrial Initiative (Dii) with weighty companies, such as E.ON, Munich Re, Siemens and Deutsche Bank, all signing up as "shareholders". Germany's announcement earlier this year that, in the wake of the Fukushima disaster, it was to speed up its total phase-out of nuclear power suddenly pulled the Desertec idea into much sharper focus. Coupled with faltering international negotiations and increasingly dire warnings on climate change – just last month the International Energy Agency warned that the world is headed for irreversible climate change if it doesn't start reducing carbon emissions within five years – it would seem the time is now right for an idea of such scale and ambition.

Last month, at its annual conference in Cairo, Dii confirmed to the world that the first phase of the Desertec plan is set to begin in Morocco next year with the construction of a 500MW solar farm near to the desert city of Ouarzazate. The 12sq km project would act as a "reference project" that, much like Egypt's own project at Kuraymat, would help convince both investors and politicians that similar farms could be repeated across the Mena region in the coming years and decades.

"It's all systems go in Morocco," announced Paul van Son, Dii's CEO, to the visiting delegates. Talks, he added, were – given their shared close proximity, along with Morocco, to western Europe's grid – already under way with Tunisia and Algeria about joining the "first phase" of Desertec. Countries such as Egypt, Syria, Libya and Saudi Arabia would be expected to join in the "scale-up" phase from 2020 onwards, once extra transmission cables were laid across the Mediterranean and via Turkey, with the whole venture becoming financially self-sustaining by 2035.

Van Son swats away any talk that the Desertec project is built on a precarious foundation of presumption, naivety and hope. "Yes, the current global financial crisis has clearly not been very helpful, but everyone also realises that being dependent on fossil fuels creates vulnerability," he says.

He also rejects any notion that Desertec carries with it even a whiff of neo-colonialism. Earlier this year such sentiments were raised by Daniel Ayuk Mbi Egbe of the African Network for Solar Energy. "Many Africans are sceptical [about Desertec]," he said. "[Europeans] make promises, but at the end of the day, they bring their engineers, they bring their equipment, and they go. It's a new form of resource exploitation, just like in the past." Other Mena-based speakers made similar points, not least that any electricity generated will first be desperately needed by local populations as they fight poverty.

"When the idea for Desertec was first announced there was anger and irritation from the Arab League," admits Van Son. "They didn't understand it at first, but we explained that it would benefit their members, too. We explained it would be a cooperative process and they became more relaxed. It's a win-win for all, we stressed. The relationship is all positive now."

Desertec should also be supported, argue its champions, because it will improve energy security by helping to diversify supply. At present, says Van Son, Europeans are vulnerable to the so-called "energy weapon", namely, when an energy-rich country holds its neighbours to ransom by restricting or denying supply. Think Russia and its gas, he says. Or a terrorist attack on an oil pipeline. Desertec will help to dilute these threats.

He is bemused, though, that the current domination of Dii by German companies should rouse suspicion. (There was not a single political or corporate representative from the UK at the conference, yet at least half hailed from Germany.) "Yes, the initiative came from Germany. But there are 15 different nationalities involved, including companies such as HSBC and Morgan Stanley. This is just the start."

A common question at the conference is: "Who is going to pay for Desertec?" There is talk of loans from development institutions such as the World Bank (the route being taken by Morocco). The presence of German banks suggests they are considering being key lenders, too. But there is also the implication that much of the burden will fall on the European taxpayer, either through EU subsidies, or tariffs added to their energy bills.

Angelika Niebler, a Christian Democrat MEP from Germany, travelled to Cairo as a member of the European parliament's energy committee. She says it is "too early" to talk about EU financing but adds: "Energy is going to be a bigger priority for the EU in coming years than agriculture has been in the past and Desertec will surely feature."

Hans Josef-Fell, a representative of Germany's Green party, is also in Cairo for the conference. "There is a fear in Germany that paying for green electricity direct from North Africa will be too heavy a burden on our consumers," he says. Germany already has among the highest electricity prices in Europe, in part because of a huge wave of renewable energy installations across the country.

Europe, particularly Germany, seems to increasingly know what it wants from Desertec. But what of its Mena partners? Obaïd Amrane, a board member of the Moroccan Agency for Solar Energy, the government body responsible for overseeing Desertec's first plant, says his country has its own plans for the electricity generated at the facility – and for the other four that will follow by 2020 – and it doesn't necessarily include selling it to Europe.

"By 2020, we are expecting a doubling of electricity consumption in Morocco, as the population and standard of living grow," he says. "At the moment, we are 97% dependent on foreign energy which is becoming increasingly unsustainable. But we are now aiming to have 42% capacity of renewable electricity by 2020. We will build extra capacity beyond what Morocco needs if someone wants us to, but we will need a big share of the electricity produced by these projects."

Such sentiments propose another challenge for Desertec: how will it guarantee that the electricity Europe needs is sent down the transmission cables and not just all consumed locally? And how will Mena countries justify selling the electricity to Europe – where the retail price of electricity can be up to 20 times more expensive – if the local population is, say, experiencing regular blackouts?

At the visitor centre at Kuraymat, bottles of chilled water are being distributed ahead of a tour of the parabolic troughs. The mid-morning November sun is already heating the engine oil-like fluid inside the troughs' receiver tubes – a technology not that far removed from Shuman's century-old design – up towards 400C.

The technical questions are coming thick and fast for Bodo Becker, the operations manager at Flagsol, the German company that specialises in building concentrated solar power (CSP) plants in the deserts of the US, Spain and now Egypt. The leading query is how the troughs perform in such harsh conditions.

"We only have one sandstorm, on average, pass through here each year," he says, "but we tilt the troughs down and away from the wind whenever it gets up beyond 12 metres per second, as they act like giant sails."

Keeping them clean is the main challenge, he adds. "Due to the dusty conditions, we are witnessing about 2% degradation every day in performance, so we need to clean them daily. We use about 39 cubic metres of demineralised water each day for cleaning across the whole site."

This surprises many delegates, as they have previously been told at the conference that CSP troughs need cleaning weekly compared to photovoltaic panels which need cleaning monthly. Either way, it highlights yet another challenge for Desertec: can enough local water ever be secured for cleaning duties? The Nile is just a few miles from Kuraymat, but some countries aim to push much deeper into their deserts to build such facilities. "Dry cleaning" technologies are being developed, but they reduce the generating efficiency at the plant. Either way, the super-heated transfer fluid requires cooling before it can loop back to the troughs for re-use, and, as with cleaning, water is the cheapest and easiest way to do this. Until "dry cooling" technologies are further advanced, it could limit solar farms to the desert fringes close to large bodies of water.

Somewhat counter-intuitively, some countries, such as Jordan, now favour wind over solar as a source of desert energy, because it is currently more affordable and isn't so water-intensive. But it is suspected that it will be many years before a single desert energy technology comes to dominate the market. Some within the industry advocate photovoltaic panels, but, currently, CSP is more popular. However, even within CSP, there are loyalists for parabolic troughs and others for "solar towers", which rely on hundreds of pivoting mirrors laid out on the ground to track the sun and direct its rays towards one fixed point at the top of a giant tower.

Whichever technology succeeds, it is already clear which nation in particular will win out as Desertec develops in the coming decades. One member of the visiting delegation asks Becker where the troughs are made.

"The metal cradles were made here in Egypt, but the glass troughs were all made in Germany," he says. "And only two companies in the world make the glass tube receivers, which is where the main intellectual property of this technology lays – Schott Solar and Siemens." Both companies are German.