In a deserted room above a London pub, there's a man sitting opposite me who looks and sounds a bit like the person who has been known as Johnny Vegas for nearly 20 years. But Vegas – who projected a drunk persona, part joyful anarchy, part pitiful self-loathing – is not here today. Instead, Michael Pennington has shown up and he's completely different: gentle, softly spoken, telling me with a smile: "Johnny has been in a box for a couple of years."

In his new book, Becoming Johnny Vegas, Pennington traces the birth of the alter ego who made him a comedy star, culminating in his triumphant appearance at the 1997 Edinburgh festival, where he won the critics' award and was the first newcomer ever to be nominated for the Perrier.

For years, he had believed his own version of events – that he had created the character after becoming interested in comedy in his early 20s, while studying pottery at university. But writing the book, and publishing it now at the age of 42, made him realise that Vegas had been around since he was a young, unhappy teenager. The memoir almost reads like a novel, ending with Vegas having all but consumed his creator. "My biggest fear about the book was being sectioned once people read it," he says. He is worried about his mum reading it: not because of all the masturbation scenes, but because he doesn't want her thinking: "I didn't know you were that unhappy."

The key moment was going to a seminary at the age of 11, to begin his training to become a priest. He believes he would have been much happier, more emotionally balanced, had he not gone. "It's one of my few real regrets," he says. "I think people would be very unforgiving if I said I'd regretted how my life turned out – because I've had a good life and a successful career. But for me as Michael Pennington, and finding myself again now, I regret not knowing what life would have been like had [the seminary] not come along."

It wasn't only the frightening advances of an older boy one night – but the experience of the school as a whole. He had gone from a close and loving family in St Helens, Merseyside, to this large institution – with cold showers and mice – that forbade any questioning of one's faith (he did have questions, and still does). It was also, he says, "the experience of suddenly being earmarked as something special, someone who was going to achieve something on behalf of so many other people [in his deeply Catholic community] and the pressure that came with that."

Although Pennington left after just four terms, he struggled to settle back into school. "I felt I no longer fitted in with children my age. That was a very lonely time, not knowing where my place was in the world." Around the same time, Johnny Vegas became a voice in his head, an imaginary friend, showing up with witty retorts when he was bullied in the school playground – though these would never make it out of his mouth. His actual response to bullies was telling them he would pray for them.

Soon after, Pennington developed hypochondria, characterised by yet another voice in his head telling him every day he was about to die. The anarchic spirit of Johnny Vegas eventually killed that off, but there were downsides. "When Johnny did away with the hypochondria, it was suddenly like I had no fear of death. There was a real carelessness towards my wellbeing."

Reading this on mobile? Click here to listen to audio

It didn't help that Pennington could only summon up Vegas through drink, meaning heavy boozing became an essential part of his standup. He had started his comedy career at university and, before every gig, he would have to glug down a carefully calculated amount of alcohol. As he writes: "Too pissed, no nerves; no nerves, no energy to keep him on his feet out there." He only went on sober once, at a comedy contest in 1995. "It was a disaster. Whatever you think of Johnny, he was fearless. He was a natural on stage, and I never was." He says standup felt like "an out of body experience – something I could never take credit for."

While this sounds modest, I wonder if the Vegas persona isn't sometimes used as a cop-out for his creator's failures. He seems to blame "him" for "trampling over" his relationships with family, friends, and first marriage. When I ask if Vegas ever went too far on stage, thinking of the 2008 show where he was criticised for groping a young woman from the audience, Pennington is quick to defend not himself but Vegas (his lawyers complained to the Guardian, which had run a piece written by a woman who was at the gig, saying that the person in question had told them she went along with the joke willingly, and did not feel intimidated). "Suddenly [people were saying] what you've done was wrong, and it came as a real shock to be accused of something like that. Johnny was as needy as me. But he wasn't a woman-hater. He didn't carry resentment towards women that would lead him to do something like that." But that night was, he thinks, "the beginning of the end of doing standup. I think even Johnny was conscious then of what he could do and couldn't."

Creatively, the neutering of Vegas had begun earlier. "The problem with Johnny was he was now part of the establishment – you're doing panel shows and you can't really go out and rally against the very people you're sitting next to. TV was a massive dilution of the original standup, and it kind of required a percentage of Michael to come and keep control, otherwise Johnny wouldn't be invited back."

After the birth of his son in 2004, Pennington also started to tone down the grip Vegas had on him off stage: he started losing weight and stopped drinking so much. "Johnny was a reaction to so many things, but I had a very loving upbringing, and a very good relationship with my parents, and my son didn't deserve Johnny as a dad. I didn't want my son one day thinking that he'd come second-best to someone who had chased an early grave and notoriety."

Just a couple of months after the birth of his son, Pennington and his wife split in what would become an ugly and public divorce. In 2011, he got married again, though is still not entirely settled: he splits his time between St Helens, where he lives; London, where his son lives; and Dublin, where his wife lives. Still, he found he was a happier person – a state hardly conducive to conjuring up a raging, confrontational drunk. The 2005 Channel 4 chatshow 18 Stone of Idiot, fronted by Vegas, was "the last push for Johnny to prove he could do TV by himself and that was …" He stops and grimaces. It was a pretty awful show.

The work he has done since – under the name Vegas, but without much of the unruliness – has been more satisfying. His sitcom Ideal ran for seven series, he was praised for roles in BBC adaptations of Bleak House and A Midsummer Night's Dream, and took on Chekhov for Sky Arts, although he says he didn't find acting easy. "I never had the confidence that I had in standup [but] I'm not going to start drinking before I go to work if I'm on a film set. I didn't want a whole confidence system based on drink." Recently he has started directing. "I love it. If you write, produce and direct, you own things and see them through to the end."

Would he ever do standup under his real name? "I would. But it would be hard going back up and being a sort of …" He pauses. "Being a not particularly funny bloke. I wonder if I can still retain some elements of Johnny without letting him out the bottle again."

Last month, Pennington was interviewed about his book on stage by the writer Frank Cottrell Boyce at the Edinburgh festival. "It was really bizarre. At the end, he said, 'Ladies and gentlemen, Michael Pennington.' And people stood up and applauded. Michael Pennington has never had a round of applause. It felt like the first step to claiming him back."