Last year, Kevin Kline spent a fortnight in Las Vegas shooting his latest movie. It was his first trip to that infamous mecca of sex and excess. "And my last," proclaims the 66-year-old, voice rolling, theatrical. "Everyone looked so miserable. I would see these couples pushing babies in strollers through the casinos like zombies. It was horrible." He shakes his head, sorrowful.

"And I just couldn't bear the constant noise. I would step outside the hotel to get some fresh air and it would be 'boom, boom, boom'" – he does an impressive impression of pounding bassline – "disco music blaring on the kerb, by the pool, everywhere. That is not my idea of tranquillity."

This, I suspect, is more like it. The Four Seasons hotel in New York, a short hop from Kline's Upper East Side home. It is two days before Christmas and we are taking high tea in an almost deserted lobby bar. Among the pots of giant bamboo and plates of tiny pastries, Kline – tall, lean, dapper in crisp shirt, waistcoat and woollen suit, with neatly clipped moustache – seems very much at ease.

Last Vegas, the film that forced him to Sin City, plays out a little like a geriatric Hangover. Kline, Michael Douglas (69), Robert De Niro (70) and Morgan Freeman (76) play a group of boyhood pals now dealing with bereavement, dodgy tickers and prostate problems. Kline had not worked before with any of them, he says, though he did see Freeman do "the definitive Coriolanus in Central Park". The film has the foursome gamely embark on a stag weekend for the perma-tanned commitment-phobic Billy (Douglas), who is finally heading up the aisle, albeit with a woman less than half his age.

For the most part, it is broad comedy, but it also has a stab at exploring the challenges facing friendship and romance at a time of life when Hollywood, at least, would suspect you were past it.

"Our whole culture is youth-obsessed, so films reflect that," says Kline. "It's sad for us older gents, but that's how it is. They want you to play the father, or the professor, or the uncle, or the old man next door. Slim pickings," he shrugs. "And the older you get, the slimmer they become."

Heavily improvised in parts, Last Vegas gives Kline a chance to flex the comedy chops that won him an Oscar for his inept ex-CIA agent in A Fish Called Wanda. The legacy of that role – away from Broadway at least – means many forget that he was once dubbed "the American Olivier" by New York Times theatre critic Frank Rich, and boasts a heavyweight stage CV, including Richard III, Henry IV, multiple Hamlets and King Lears and a couple of Tonys. This has come at the expense of a major Hollywood career: he has shied from the lure of LA, instead signing on for quirky indie fare such as Grand Canyon and Life As a House. Was he never tempted to go west? "No. In fact my agent told me: 'Stay in New York, there's much more cachet.'"

Kevin Kline (right) plays King Lear in New York in 2007. Photograph: Sara Krulwich/New York Times/Redux/eyevine

He leans forward and spreads marmalade on his scones. "I did theatre once in LA – it was a ghastly experience. It was like the audience was just waiting for the commercials." Burglars would struggle to locate his awards at home. "It's not as if they are hidden; they're just … camouflaged. I have one photo of myself in my fat suit as Falstaff, which my best friend paid a lot for, so I felt had to hang it. But otherwise, I don't like putting up posters or pictures, or flagrantly displaying awards. There's something very … midwestern and undignified about it."

Kline was raised in St Louis, Missouri – very much the midwest – where his father, Robert, owned a record shop and sang in amateur operas, though his mother, Margaret, was the one with the "theatrical, dramatic character". While he plays up his luddite credentials ("I don't tweet; I barely email"), in reality, he is far from the pitiful character he plays in Last Vegas, who has retired early and relocated to a creaking senior citizens' compound in Florida. "Do people honestly live like that? Do they actually move to retirement communities at 55?" he asks. "It's like giving up."

Kevin Kline with his wife, Phoebe Cates. Photograph: Kevin Mazur/WireImage

Certain themes in the film were closer to home, such as the age-gap relationship; Kline has been married for almost 25 years to former teen model and actor Phoebe Cates, who is 16 years his junior. He puts it down to biological determinism. "I was 40-ish, when I finally stopped obsessing about acting and thought it would be nice to have a life and actually got married," he says. "If I had married someone my own age, the possibility of progeny would have been very slim. I didn't consciously say: 'Hey, we could make babies; want to get married?' It just happened that way."

Indeed, Cates – who had starred in Fast Times at Ridgemont High and Gremlins – gave up her career to raise their children, Owen, now 22 and an aspiring film director, and Greta, 19, a music student and successful singer-songwriter. "That was one of the many things I loved about her," Kline says. "She loved acting while she was doing it, but she was not driven [he goes grand, declamatory; cocks an eyebrow] by it, as I had been."

Overwhelming though it became, his desire to act surfaced relatively late. After an all-boys Catholic education from Benedictine monks, he began studying music at Indiana University, aspiring to become a classical pianist. However, after joining an off-site drama group, he caught the bug and switched to acting. "Being a pianist is very solitary," he says. "Whereas I really enjoy the social interaction, the community feeling of acting." He was accepted into the founding drama class at Juilliard performing arts conservatory, after which, along with his then-girlfriend Patti LuPone, he travelled the US as part of the City Center Acting Company.

"I honestly thought I'd be in repertory theatre for ever," he says. "Films just didn't really occur. You had to work off-off-Broadway and then off-Broadway; you had to pay your dues. It's a gross generalisation to say this current generation is too entitled, but it might also be true." He finally made his screen debut – opposite Meryl Streep – in Sophie's Choice in 1982.

But beyond his dues paid, his camouflaged trophies and his New York cachet, Kline is honest enough to admit that, so far, his dance card is empty for 2014. Does that trouble him? "I prefer to work," he concedes. "I like some structure to my day. But I am more and more determined, with every passing year, to live life, as opposed to 'having a career' that is in some way stellar.

"And I always have the theatres,' he adds, cheerily, through a mouthful of scone. "There's always King Lear."

• Last Vegas is released in the UK on Friday 3 January.