Tucked into Higgs's luggage was the reason he had been invited. The notes for his highly contentious lecture overturned some of the most deeply-held beliefs of the resident experts. They proposed something remarkable, that an invisible field, which stretches throughout the entire universe, holds the key to one of the greatest mysteries of modern science - the nature of matter and mass.

Higgs was pondering the talk and how it would go down among the biggest brains in physics, when it all suddenly became too much for him. Out of the window, he glimpsed a roadsign to Princeton. It was enough to trigger a fit of panic. Shaking, Higgs pulled into a lay-by and sat there panting, waiting to regain his composure.

As expected, Higgs faced a barrage of questions, but to his relief none of the eminent scientists in the audience found a flaw in his thinking. The lecture became a landmark in the folklore of theoretical physics and quickly set the stage for what was to become the most spectacular quest in modern science.

More than 40 years later, Higgs is still largely unknown beyond his field, but that is about to change. The multibillion pound race to discover if his theory is right is finally nearing its climax. Either way, the answer will propel 21st-century physics into a new and uncertain era. Higgs, who turned 78 in May, is clearly a Nobel prizewinner in waiting. "I have to ask my GP to keep me alive," he says, when we meet in his Edinburgh apartment.

Higgs rarely gives interviews; it's not so much that he refuses as lets the requests gather dust until it's too late. The phone goes unanswered, pleas through friends come to nothing, emails evaporate in the ether. Good old letters are his preferred means of communication. Luckily, Higgs has found a few hours to spare before rushing off to join his wife for another round of Monteverdi madrigals at the festival that first attracted him to the city in 1949. He tells the story of his unwitting discovery of something in the emptiness that surrounds us. "It has consequences," says Higgs, pausing to fold his arms so that each hand can rub the opposite's elbow. "If it wasn't there, we wouldn't be here."

Higgs was born in Newcastle in 1929, but the family moved around with his father's job, as a sound engineer with the BBC. He missed a lot of early schooling. Bouts of serious asthma drifted into pneumonia ("not funny when there aren't any drugs") and he was kept home and taught by his parents. As a young boy, Higgs was raised by his mother in Bristol while his father relocated to Bedford. "She was very motivated to push me," he says. "My father, I think he was just rather scared of children."

At Cotham Grammar School in Bristol, Higgs would stand at the back of morning assembly, reading the names of the school's most honoured alumni. Appearing more times than any other was the Nobel prize-winning physicist and founding father of quantum mechanics, Paul Dirac, who, like Stephen Hawking, took the seat at Cambridge that was once occupied by Sir Isaac Newton. It was Dirac's work that enthralled Higgs and put him on the path to study theoretical physics. "It's about understanding! Understanding the world!" Higgs says, his voice full of excitement.

When illness wasn't disrupting Higgs's education, the war was. Bristol had already been thumped by German bombers, the old centre almost completely flattened, but the outskirts where he lived and went to school took hits, too, from bombs shed by planes almost as an afterthought as they turned home from raids on the oil storage depots and ports at Avonmouth. "One of the first things I did on arriving at school was to break my left arm falling into a bomb crater," Higgs says. Later, the family was forced to leave home when a cluster of unexploded bombs was discovered across the road.

The family was not reunited until the end of the war, when Peter, aged 17, joined City of London School, specialising in mathematics. Among the gifted, he was the odd one out. He alone had no desire to go to Oxford or Cambridge, the thought enough to make him shudder. "They all wondered why I wasn't going to do the same," he says. "I think some of the family attitude to Oxford and Cambridge had rubbed off on me, which was that those places were all very well for the children of the idle rich to go and waste their time and that of their tutors, but if you were serious about university, you went somewhere else."

In Higgs's case, somewhere else was King's College, London, and it was there that it became clear he was hopeless at experiments. "There were accidents," he says, refusing to elaborate.

In his early 30s, Higgs moved to Edinburgh University, where he became interested in what must be one of the most curious puzzles in physics: why the objects around us weigh anything.

Until recently, few even questioned where mass comes from. Newton coined the term in 1687 in his famous tome, Principia Mathematica, and for 200 years scientists were happy to think of mass as something that simply existed. Some objects had more mass than others - a brick versus a book, say - and that was that. But scientists now know the world is not so simple. While a brick weighs as much as the atoms inside it, according to the best theory physicists have - one that has passed decades of tests with flying colours - the basic building blocks inside atoms weigh nothing at all. As matter is broken down to ever smaller constituents, from molecules to atoms to quarks, mass appears to evaporate before our eyes. Physicists have never fully understood why.

While working on the conundrum, Higgs came up with an elegant mechanism to solve the problem. It showed that at the very beginning of the universe, the smallest building blocks of nature were truly weightless, but became heavy a fraction of a second later, when the fireball of the big bang cooled. His theory was a breakthrough in itself, but something more profound dropped out of his calculations.

Higgs's theory showed that mass was produced by a new type of field that clings to particles wherever they are, dragging on them and making the heavy. Some particles find the field more sticky than others. Particles of light are oblivious to it. Others have to wade through it like an elephant in tar. So, in theory, particles can weigh nothing, but as soon as they are in the field, they get heavy.

Scientists now know that Higgs's extraordinary field, or something very similar to it, played a key role in the formation of the universe. Without it, the cosmos would not have exploded into the rich, infinite galaxies we see today. The spinning disc of cosmic dust that collapsed 4.5 billion years ago to form our solar system would never have been. No planets would have formed, nor a sun to warm them. Life would not have stood a chance.

In late summer 1964, two years before he would give his Princeton lecture, Higgs rushed out a succinct letter, packed with mathematical formulae that backed his discovery and sent it to a leading physics journal run from Cern, the European nuclear research organisation in Geneva. The paper was published almost immediately, but went largely unnoticed. Higgs planned a second paper, to emphasise his discovery, but for now that would have to wait.

Through CND meetings in Edinburgh - Higgs had been an activist while studying in London - he had met Jo, an American linguist and his future wife. The two had planned a weekend's camping in the west Highlands, on the recommendation of a friend who'd read the place had the lowest rainfall in Scotland. As it happened, the trip was a disaster. "It turned out she'd misread it. It was the highest rainfall in all of Scotland," Higgs says.

The scientist took the chance to retreat to Edinburgh and write his second paper, this time elaborating on the true implications of his work. In autumn 1964, he sent it to the same journal for publishing, but astonishingly the Cern editors rejected it. Evidently, it was considered "of no obvious relevance to physics". He quickly sent it to America's leading physics journal, where it appeared later that year.

Despite Cern's misgivings, Higgs's ideas now exploded into the world of theoretical physics and thousands wanted to be first to prove Higgs right. Detecting the field itself is thought to be impossible with modern technology, but Higgs also predicted a particle that is created in the field, and finding this would be the proof they sought. Officially, the particle is called the Higgs boson, but its elusive nature and fundamental role in the creation of the universe led a prominent scientist to rename it the God particle.

The name has stuck, but makes Higgs wince and raises the hackles of other theorists. "I wish he hadn't done it," he says. "I have to explain to people it was a joke. I'm an atheist, but I have an uneasy feeling that playing around with names like that could be unnecessarily offensive to people who are religious."

Strictly, the particle should bear the names of three scientists. Unknown to Higgs at the time, two Belgian physicists at the Free University in Brussels were working on the same problem. Using completely different maths, they reached the same staggering conclusion - that a never-seen field must pervade the universe and confer mass on almost everything in it. Robert Brout and François Englert didn't doubt their discovery, but checked and checked for mistakes before publishing. Their paper was published in August 1964, a few weeks before Higg's first paper, which was in press at the time.

It makes for an awkward situation, not least for Higgs, who agrees all three should share credit for the discovery. He recounts a tale when a colleague referred to the "Higgs mechanism" in a lecture in Germany more than two decades ago. In the front row, a look of displeasure flushed over one of the men in the audience. Realising his mistake, the speaker said, "Of course, I know this was also discovered by others, but I refer to it by the person with the shortest name." "My name has five letters, too," piped Brout.

A few months ago, Brout and Englert, who are close as brothers and finish each other's sentences, talked to me about the events long ago. After publishing their work, the two were having a beer on the balcony of a 17th-century cafe overlooking a Brussels park. "In the spring of 1964 we were both extremely excited," said Brout. "For the first time in my life, I felt what it might be to be a great physicist." Neither, he says, blames Higgs for their work being sidelined.

Whatever name it takes, many scientists believe that finding the particle will not only reveal the origin of mass, but will nudge open the door to a new realm of unknowns. We can see only 4% of the matter that makes up the universe. The Higgs particle may shed light on the rest - the dark matter in which galaxies form, and the dark energy that drives the expansion of the universe, for example. The particle may also shed light on string theory, an ambitious but powerful way of viewing the universe that sees every particle not as a point, but as a vibrating string of energy, where different frequencies create different particles.

Drive a few miles west from Geneva airport, with Mont Blanc behind you and the Jura mountains ahead, and you'll soon find yourself 80 metres above a giant underground particle collider, powerful enough to recreate, for a split-second, the earliest moments of the big bang. It is here at Cern, the organisation that famously rejected Higgs's idea, that the race to discover the God particle began. "Isn't that ironic?" he says.

It is a race that Higgs has followed only from the sidelines. In the decades since his discovery, he claims he struggled to keep up with the work of the new generation of bright young scientists. "The point came when people were doing things I didn't feel competent to do myself. I'm not being modest, I honestly get lost. I was lucky in spotting what I did when I did, but there comes a point where you realise what you're doing is not going to be much good."

Cern was created in the aftermath of the second world war jointly by 11 European governments, its role to probe deeper into the atom than anyone had before. Politically, it was an easy sell. The energy locked up inside the atom had just brought the war to an appalling close and few governments were willing to stand by while others learned more of the atom's secrets. Cern was first headed by Victor Weisskopf, a brilliant Jewish physicist who fled Austria before the war and joined the Manhattan project at Los Alamos. He later became one of the most outspoken advocates for nuclear disarmament. Across the Atlantic, his contemporary, Robert Wilson, former head of the Manhattan project's experiment group, was to head Cern's greatest rival, known as Fermilab, another underground facility carved from the rock beneath Illinois buffalo country.

Inside a collider, subatomic particles are whipped up to within a hair's breadth of the speed of light before they are steered into head-on collisions that shower sub-atomic debris in all directions. Put enough grunt into the collisions and, conforming to Einstein's e=mc2, the energy released on impact will create new, sometimes heavier particles. Among them, with luck, will be the God particle.

Throughout the 90s, Cern scientists slammed particles together in their 17-mile circular accelerator and sifted through the sub-atomic wreckage for evidence of the Higgs particle. At first, they found strange signals. Some came and went with the rising of the moon, others more frequently. It was only after a lengthy investigation that they realised their multibillion machine was flexing with the tides of Lake Geneva and picking up stray currents from the TGV train, which came and went like clockwork from Geneva station down the road. Like Higgs himself, the particle remained elusive.

In America, physicists drew up plans to leapfrog the Europeans with an accelerator that would be 20 times more powerful than Cern's. To be built beneath the Texas boomtown of Waxahachie, the superlative-heavy "superconducting supercollider" (SSC) was proposed during a Reagan spending spree that was already hoping to protect America from Russian nukes with the "Star Wars" nuclear missile shield.

The man charged with selling the SSC to Reagan was Alvin Trivelpiece, then director of the office of energy research. Trivelpiece, an exceptionally astute physicist, had heard the president's sight was failing, and prepared his presentation on two large easels, which he dragged into the Oval Office. There, he likened the SSC's task to using rifle bullets to find billiard balls hidden in bales of hay. "I knew the one thing they would understand was guns," Trivelpiece told me.

The SSC had been sold to US taxpayers as the machine to find the Higgs and much more, but it ran into difficulties almost immediately. Mismanagement and ballooning costs saw the price tag leap to more than $12bn by 1993, and under Clinton Congress finally voted for building work on the collider to be scrapped. Nearly $2bn had already been spent. Under Waxahachie, more than 14 miles of tunnels drilled from the ground were abandoned. "I think Reagan was thoroughly confused as to whether this thing was going to help him zap the Commies," Higgs says.

Back in Geneva, the Cern scientists were about to face their own upheavals. In summer 2000, scientists working on the collider saw what looked like the first glimpse of the God particle. It was tantalising, but not enough to claim a discovery. They needed more time to prove it, but there was none. The collider was due to shut in a few months, to be replaced by a more powerful machine.

That September, at a restaurant on the island of Jeju, off the southern tip of the Korean peninsula, one of the odder events in the hunt for the Higgs particle was about to play out. Gordy Kane, now director of the Michigan Centre for Theoretical Physics, was discussing the glimpse of the God particle that had been reported at Cern. During the discussion that followed, a familiar voice cut in, offering to bet £100 that the Higgs particle would never be found, at Cern or anywhere else. It was Stephen Hawking. Kane accepted the wager. "It was a bit of a cheek," says Higgs, who believes Hawking was not familiar enough with the physics to back up his view.

At Cern, the scientists won a temporary reprieve. They crashed more particles and stared at more traces, but the glimpse never became anything more. In November 2000, the machine was switched off, triggering outrage among many who felt physics' most-wanted prize had been snatched from their grasp. Kane sent a cheque for £100 to Cambridge, but he expects his money back soon.

In an exchange of emails with me, Hawking said he is still willing to pay up if the particle is ever found, but thinks the hunt could be scuppered by mysterious, fleeting black holes created in Cern's accelerator. "I think there's a good chance that virtual black holes will make it impossible to observe the Higgs, but of course, if it is found, I will pay," he said. (On hearing this, Kane said he was willing to bet against black holes hampering the discovery.)

With the Cern collider out of action, the only place with a chance of finding the Higgs particle was America's Fermilab, and last December rumours hit the internet that they might have succeeded. So far, it appears to have been a false alarm, and the lab has yet to confirm the discovery.

Next May, Cern's new machine, the £4bn Large Hadron Collider, will switch on for the first time, again putting Europe and America in direct competition in the race for the particle. The LHC, as the new collider is known, will smash protons together with 14 times the energy of the Fermilab collider, and should churn out Higgs particles from the moment it is switched on.

For Higgs, the next year may become a blur. "I'm worried about the next few months, that there's going to be an outbreak of attention," he says. But he may also witness the discovery of the particle that bears his name, one that will change the course of physics. Ever understated, Higgs smiles at the thought. "It'll be a relief for them to find it," he says, collecting his things and preparing to leave. "If I'm wrong, I'll be rather sad." ·