Bill Murray is expected for lunch in a quiet backwater at the Cannes film festival. A limousine with tinted windows edges its way around a line of parked cars along the narrow street. A posse of film types assembles to greet the star and the car doors are opened with a flourish. Out step an unknown couple, surprised at the fuss. As they do so, a scooter splutters into view, sounding like a clapped-out lawnmower, with Murray's familiar figure perched on the saddle. He checks the location - a deadpan look on that pock-marked face - and parks. "I always like to have my life in my own hands," he says.

Murray, alone among the stars in town, has no entourage - no agent, business manager, lawyer or favourite hair and make-up person in tow. When you meet him, he has an old-fashioned politeness and you notice that he does not laugh much. But of the Hollywood stars who have made a business out of laughter on screen, he is one of the most enduring. Other comics who followed him into the movies from America's anarchic Saturday Night Live have gone on to greater riches - Jim Carrey and Mike Myers among them - but, at 55, Murray is recognised by his contemporaries as the king. Will Ferrell, the latest SNL recruit to films, can recall each twist of Murray's act. "He would come on as a guest and I'd watch him from the sidelines, thinking, 'How can he be so cool?'" George Clooney admits to a tongue-tied moment the previous year at the Venice film festival: "I could not string two words together," he says. "He's the best comic actor in the world by a mile."

Murray's confidence, though, is surprisingly fragile. He'd had a terrific audience reaction to the previous night's premiere of his latest film, Broken Flowers - indeed, it was on the strength of this that he'd agreed to meet me. "I'm happy," he says. And he does look bright enough, in an unfussy short-sleeved blue check shirt - no rings, no actorish bracelets, not even a watch. "Yes, I am relaxed," he declares. His creased, lopsided face, topped with thinning, grey hair, is so inscrutable it would be impossible to judge his mood.

At the premiere, he says, he'd been uncertain and edgy. He turned up, reluctantly, to walk along the red carpet, wave to the crowd, and sit beside director Jim Jarmusch. "I had been told that if they don't like what they see, the room is not just freezing, but downright nasty," he says. "Jim said that he'd presented a movie a few years before to 2,500 people in the same auditorium and, after the credits ended, there was total silence. A voice from high up above said, 'Jim, it's shit.' Then the booing started. I thought about that for every minute I sat there watching, despite the audience's laughter."

Once the applause started, it went on for five minutes. Broken Flowers - in which Murray gives an even quieter performance than he did in Lost In Translation - went on to win the Grand Prix, regarded as the festival's runner-up prize.

Murray plays a successful, middle-aged man called Don Johnston - and yes, there are several in-jokes about Don Johnson of Miami Vice in the script. He is dumped by his latest girlfriend (Julie Delpy) and then receives a mysterious letter, in pink handwriting, from an anonymous former lover, informing him he has a 19-year-old son. Johnston begins to crisscross America in search of clues from four old flames: Laura (Sharon Stone), Dora (Frances Conroy), Carmen (Jessica Lange) and Penny (Tilda Swinton). He visits each, unannounced, on a road trip into his past. Stone's character is a widow with a seductive daughter, Lolita, who parades naked around the family home. Lange's Carmen is off men for good and has a jealous female companion (Chloë Sevigny).

"Just the very thought of someone my age going to visit old girlfriends had instant appeal," says Murray. "Even women think, 'That would be interesting.' Not comfortable, but interesting. It is not a comfortable film at any point."

What gave it an added frisson was that Jarmusch, an independent director, was not under the cosh of any film studio. He told his actors they could play their first scene with Murray any way they wanted. "Tilda Swinton's first words to me were, 'What the fuck do you want, Donny?' " Murray recalls. "They kept that line in - it really woke me up."

There are those who say Murray could do with a wake-up call - something to shake him out of his blank-faced calm. The New Yorker carried a cartoon of a board meeting, with members realising one of their number is dead at the table. The caption reads: "I thought he was just doing his Bill Murray impression." He's not one to pretend to emotions he does not feel, which can itself be revealing. At last year's Oscars, when he was beaten to a best actor award for his performance in Lost In Translation, he alone among the other four nominees didn't applaud Sean Penn's victory. It was refreshing to see an actor refusing to fix a false smile and cheer a rival's triumph. What you see with Murray is what you get. "Pissed off?" he says, when I mention the Oscars. "You bet I was. I don't approve of award ceremonies, so I was wondering what had persuaded me to attend that one. I was pissed at myself."

Murray has made more than 40 movies and there's still no big prize in sight. Comedy, however well done, is at the bottom of the barrel when it comes to awards. He knows it. So do Steve Martin, Jim Carrey and Mike Myers. "Even Charlie Chaplin knew it," Murray says. "Laughter and the lighter moments of life always seem easy to deliver. I don't expect those giving out the awards to understand."

Murray is the fifth of nine children from small-town Wilmette, Illinois, who spent much of his spare time in his teens on a golf course, caddying for rich local businessmen. "It was my first glimpse of comedy. When you see grown men near to tears because they've missed hitting a little white ball into a hole from three feet, it makes you laugh." His blue eyes light up for a second at the memory, but his mouth doesn't break into a smile. His education, he says, was virtually a waste of time. "I was on the golf course rather than being in lessons. I couldn't really think of anything that interested me."

Things did not improve after school. He was arrested for possession of marijuana at Regis College, Denver. "I sort of dropped to one side. I could not think of my way through to any sort of long-term career." The way he tells it - and Murray doesn't tell it very clearly - he flailed around, like an octopus with boxing gloves, in the hope of hitting on something useful. One thing he had realised was that he could be funny. He brightens when recalling how his humour appealed to his fellow 10-year-olds at St Joseph's school in Illinois. "I went to the grade school reunion last year, which was a hoot," he says. "You spend eight years in grade school with people and they knew you as a kid - they know how you really are. I made an extraordinary effort to be there, because I was working on a movie. I had to work extra-long days to get the day off to go. But it was worth it. The kids laughed at me at school and they still laugh today. I got the wish to play to an audience from them."

He applied, years later, to join the Second City comedy troupe in his nearest city, Chicago. "I was accepted and got a foot in the door," he says. That door opened wider when he joined Dan Aykroyd, Gilda Radner and John Belushi on Saturday Night Live. "I did not start learning until then," he says. "I made a lot of mistakes and realised I had to let them go. Don't think about your errors or failures, otherwise you'll never do a thing."

Aykroyd, the only survivor of the original SNL line-up, recalls, "Bill would try anything - and get away with anything. He always looked as if he didn't particularly care, but he cared very much about what he did and how he did it."

His early movies, beginning with Meatballs (1979), directed by Ivan Reitman, seemed like trial and error. Murray had already fashioned his underwhelmed look, but he didn't shine in the country club movie Caddyshack (1980) nor the army comedy Stripes (1981). Then came Tootsie (1982), playing Dustin Hoffman's roommate. He ad-libbed his way through the script. "They kept on saying, 'Just react.' So I would come up with lines like 'That is one nutty hospital' or 'I'm just afraid you are going to burn in hell for all this' ... Then they would write these down as scenes and say, after a few days, 'Come up with something else.' It was like that through the movie."

He also learned about power play on a film set, witnessing the constant arguments between Hoffman and director Sydney Pollack. "Dustin would throw a fit, and the crew just stood back and watched," he says. "He's a perfectionist. These things explode, but it is always about getting the right sort of film. Sydney would have a go back and they'd be like these two prize fighters, with veins bulging in their foreheads. I still felt like the junior guy in movies, so I tried to lighten the mood." That took the form of Murray pretending to have a fit himself. "Everyone knew I was kidding," he says, "but it helped defuse one or two situations."

It wasn't until Ghostbusters (1984), in which Murray teamed up again with Aykroyd, that he was the star of a box-office hit. It allowed him to become a leading man in a succession of films through the 1980s, including The Razor's Edge, Little Shop Of Horrors and Scrooged. Ghostbusters II helped his bank balance more than his acting profile, but in his 1991 hit What About Bob?, playing a patient pursuing his psychiatrist (Richard Dreyfuss) on holiday, Murray was back to his comedic best. His role as a TV weatherman reliving the same day in Groundhog Day in 1993 and finally getting things right with Andie MacDowell crowned what appeared to be a comedy run with a golden touch.

On the occasion I encountered Murray on a film set - he was making Larger Than Life in Los Angeles in 1996 - his pervasive influence was obvious, apparently in a benign way. A country band suddenly turned up to play over lunch. Murray had seen them in action the previous night and wanted them to share their talents with the cast and crew. The tensions, where all concerned were hugely anxious to produce a hit, were eased and the crew perked up. (The film was still not a great success, however.)

Four years ago, he was in hot water on the set of the high-profile Charlie's Angels, in which he played the Angels' chief, John Bosley, with a glamorous line-up of Cameron Diaz, Drew Barrymore and Lucy Liu. He made a chance remark to Liu about the quality of the script. "She got furious with me because she thought it was a personal assault," he says. "We went at each other for about 15 or 20 minutes. We made our peace, but that row has been reported again and again as something really significant. It happens all the time on film sets when you are arguing about the work."

Murray did not feature in the sequel. He was getting a reputation as not the easiest man to work with, and there was churlish talk of younger, funnier comics emerging to overtake him. By the late 1990s he was in need of a box-office success and quality, and he found them both with three independent, or at least independent-minded, film-makers: Wes Anderson, Jarmusch and Sofia Coppola. With them, he reinvented himself.

For Anderson, he was a somewhat sinister industrialist in Rushmore, Gwyneth Paltrow's uptight husband in The Royal Tenenbaums, and, most recently, nonchalantly funny as a washed-up marine explorer in The Life Aquatic With Steve Zissou, appearing much of the time in a wetsuit and capitalising on the pathos and sense of expected failure that have always been enjoyable elements in his performances. Jarmusch roped Murray in for a turn in his portmanteau movie Coffee And Cigarettes (2003), and eventually wrote Broken Flowers specially for him.

In between came Lost In Translation. "I behaved very well on that movie," Murray says. "It is starting to bug me, because they take me for granted. I don't think I am doing myself any favours, because I am getting a reputation for being a pushover." Or being a muse. Coppola always had him in mind for the role of ageing film star Bob Harris, in Tokyo to make an advert for Japanese whisky. It was almost an echo of Murray's own position when he told a neglected wife (played by Scarlett Johansson), "I am here being paid $2m for a commercial when I could be doing a play."

There were many threads in the film he recognised from his own life. "You are always away from home, as a film actor. Look at me now. You can be stuck in a hotel, several thousand miles away in a whole different time zone, and it is never glamorous. You can't sleep, you put on the television in the middle of the night when you can't understand a word, and you make phone calls back home which don't really give you the comfort they should."

Murray's second wife, Jennifer Butler, is at home in Los Angeles. She rarely travels with him because they have three young sons - Jackson, 11, Cal, nine and Cooper, eight. He has two more boys, Homer, 22, and 19-year-old Luke, with his first wife, Margaret Kelly.

"I know what it's like to be that stranger's voice calling in," he admits. "It happens in acting and it happens in business. Those who are living together all the time and can guarantee seeing each other every night or weekend probably don't know what I am talking about. There is also that little-discussed subject - loneliness. That is a great taboo, isn't it? No one really wants to admit they are lonely, and it is never really addressed very much between friends and family. But I have felt lonely many times in my life.

"The oddest feeling of all was after the death of my mother, Lucille, 10 years ago. My father had already died, but I always had some attachment to our big family while she was alive. She was from the previous generation, which kept us all going. She was always interested in all of us and would pass on all the news from various uncles and aunts. But when she died, I felt bereft. It seems strange to say now that I felt so lonely, yet I did. It was as if her passing put me into the same category of a kid who was an orphan. Crazy. And this is from a man in his 40s. Truth be told, I feel like shedding a tear here and now just talking about it."

You wouldn't have known. Murray's face does not change much. An occasional frown, perhaps, or a tightness around the eyes, an overall impression that he would rather not be here talking about himself. But he has good reason to be pleased with himself. He's done his best work since he's reached middle age. Despite his hang-dog implacability, he's not a character actor in the manner of Danny DeVito. He has never been traditionally handsome in the way of his fellow fiftysomethings Richard Gere and Kevin Costner or the more solid Tom Hanks (actually 49), but he is a bona fide leading man - among other things, he's a very good physical performer and he has the charisma that was on full display in Lost In Translation. It's possible he's not sure where he fits in the Hollywood pecking order.

He's clear on one thing, though. "Whenever I think of the high salaries we are paid as film actors, I think it is for the travel, the time away, and any trouble you get into through being well known. It's not for the acting, that's for sure."

He's also clearly unnerved by the inevitable gaps between working. When he's not filming, he says, "I do absolutely nothing. I go home and stay there. I wash and scrub up each day, and that's it. One month I actually grew a moustache, just so I could say that I'd done something. I am years behind on reading or seeing movies. I find myself watching sports on television or riding a stationary bicycle. Once I break into a sweat, I get off it. But I can only take so much TV, because there is so much advice. I find people will preach about virtually anything - your diet, how to live your life, how to improve your golf. The lot. I have always had a thing against the Mister Know-It-Alls."

Murray says his goodbyes, rifles through his pockets for the key to his scooter and eases into the saddle. He revs up and he's off on his own way again.

· Broken Flowers is on general release