On 7 August 1961, in the Rio Cafe in run-down Notting Hill, west London, Aloysius “Lucky” Gordon, a Jamaican jazz singer and hustler, met Christine Keeler for the first time. It was an encounter that would unravel a scandal and help to secure the downfall of the Conservatives at the 1964 election, ushering in the Labour administration of Harold Wilson. Gordon, who has died aged 85, was affected by its consequences for the rest of his life.

He was selling marijuana, and Keeler was looking to buy. With her were two men: Stephen Ward, the mysterious society osteopath, a pimp-like mentor to the soon-to-be-notorious Keeler, and John Profumo, secretary of state for war and Keeler’s lover, who handed the young woman cash for the drug.

Gordon was immediately besotted by the beautiful 19-year-old model. Soon he also became her lover, but their fractious relationship quickly disintegrated in violent encounters. In the second half of 1962 Keeler sought refuge from Gordon’s temper by hooking up with Johnny Edgecombe, an Antiguan shebeen owner. That led to a confrontation between Edgecombe and Gordon in October of that year outside the Flamingo Club in central London, in which Edgecombe slashed Gordon’s face with a knife, the wound requiring 17 stitches.

Lucky Gordon rehearsing at the Stork Club in London in 1963. Photograph: John Twine/ANL/Rex/Shutterstock

Then in December came the incident that brought to light the tawdry world Ward had constructed: Edgecombe fired shots at the door lock of Ward’s house in central London, behind which were sheltering Keeler, her friend Mandy Rice-Davies and Gordon. (Other accounts denied both that Gordon was there and that Profumo had been at the Rio Cafe, but Gordon was insistent to me on both points.) The police and press investigations that followed the shooting and the conjunction of names that emerged from it, including that of Yevgeny Ivanov, a senior naval attache at the Soviet embassy with whom Keeler was also sharing her bed, led to the uncovering of what became known as the Profumo affair.

After Profumo’s relationship with Keeler was alleged, he lied to the House of Commons in March 1963, claiming there had been “no impropriety whatever”, but 10 weeks later he had to confess and resign. The fallout from the scandal, which included the imprisonment of Keeler and the suicide of Ward, undermined the authority and health of the prime minister, Harold Macmillan, who resigned that October. After an election was called by his successor, Sir Alec Douglas-Home, Labour took power under Wilson.

There were repercussions from the affair for Gordon, too. He was sentenced to three years in prison in June 1963, after Keeler accused him of assault. But Keeler then withdrew her accusation, and he was released from prison the following month after an appeal; she was jailed for perjury.

On his release, Gordon discovered that the founder of Island Records in Jamaica, Chris Blackwell, had issued a comedy album about the Profumo affair. Gordon tried to hustle money from Blackwell and, while the entrepreneur was immune to his efforts, he did offer Gordon a job as a cook at the Island Records studio in Notting Hill. Spirited, witty, but never without an air of menace – or a bag of ganja – Gordon proved an asset to Blackwell, who noted that “Lucky’s energy was very important in helping to establish the right vibe at the studio”. In 1977, when Bob Marley was based in London, Gordon moved into his Chelsea house to prepare the food.

Lucky Gordon at the Old Bailey in London in 1963 after being accused of assaulting his former girlfriend Christine Keeler. Photograph: John Franks/Getty Images

Born in Kingston, Jamaica, Lucky earned his nickname from a small pools win by his mother, a cook, on the day he was born. As an adventurous but impoverished teenager, he would stow away on ships sailing out of Kingston. It was thus that he arrived in Scotland in 1948, only 17 years old. Threatened with deportation, he opted to become a miner, but endured the job for only a few days before disappearing to London.

In the capital he found work as a cook. But he also developed a nightclub act, modelling himself on the American jazz singer King Pleasure, and made money from nefarious activities including the art of smash and grab, which had come into vogue during the 1950s. Gordon boasted about the celebrated West End jewellers he had robbed, describing how he had once lain down on the road beneath a car until the police left.

A week before the 1958 Notting Hill race riots, Gordon was arrested in one of the area’s illegal gambling dens and beaten so badly that he lay in a coma for 13 days; he was jailed for nine months for having “attacked” the arresting officers. Gordon was no stranger to prison after that point: he was convicted in 1960 over an assault in Denmark on a 19-year-old woman, and a stint forging cheques brought him a spell in Wandsworth prison, where his friendship with Reggie Kray earned him favours.

He continued working as a cook (and occasional guest vocalist) for Island records in London until Blackwell sold the label in the early 90s, after which Gordon adopted a more leisurely lifestyle. He could always be found in one of the betting shops on Portobello Road.

Later in life, as his body failed him, Gordon became more peaceable, no longer threatening to kill anyone who offended him. But Keeler seemed rarely from his mind. “I loved her, I loved her,” he would utter mournfully.

He is survived by his brother, Syco, and by a daughter, April.

• Aloysius “Lucky” Gordon, singer and cook, born July 5 1931; died 15 March 2017