The sublime tenor voice of the American soul singer Howard Tate, who has died aged 72, lit up a string of immaculate recordings in the late 1960s. He is best known as the singer of Get It While You Can, an eloquent ballad cut in 1966 that failed to chart on release, but later became a hedonist anthem when Janis Joplin recorded it on her 1970 album Pearl.

Tate never enjoyed much good fortune in the music industry and stopped performing for more than 20 years. Yet his remarkable range, expressiveness and artistry meant fans of black American music remained in thrall to his recordings and when, after attempts to discover his whereabouts, he was found alive and well in 2001, there was much celebration – not least from Tate, who discovered that he by then possessed a loyal European fanclub.

Tate was born in Eberton, a small town near Macon, Georgia, and raised in Philadelphia. His father was a Baptist minister who encouraged his son to sing in church. At the age of 10 he formed a gospel trio with his cousins. A visiting gospel group, the Gainors, were so impressed by Tate's singing that they invited him to join. They performed at churches all over the Philadelphia area and, in 1955, a Mercury Records scout signed the group, on the condition they recorded doo-wop. The resulting 45s did not enjoy any real success and, in 1960, Tate leapt at the offer to be a vocalist for the pianist and organist Bill Doggett, who had scored a huge US hit in 1956 with Honky Tonk.

Tate stayed with Doggett for three years before returning to Philadelphia. There, he found that the Gainors had changed their name to Garnet Mimms and the Enchanters and scored a huge hit with Cry Baby (1963), a stunning slice of emotionally laden, orchestrated soul music written and produced by the supremely gifted Jerry Ragovoy.

Ragovoy heard a recording of Tate, and encouraged him to move to New York. His first single, Half a Man, appeared on Ragovoy's Utopia label and failed to chart. Ragovoy dug deeper, signing Tate to Verve Records and providing him with strong songs at their next recording session in November 1964. Ain't Nobody Home hit No 12 on the R&B charts and Tate went from mixing mortar on a building site to joining Sam Cooke and Wilson Pickett, two of the hottest soul stars then working, on coast-to-coast tours.

Tate scored a moderate hit on the R&B charts with Look at Granny Run Run (1966), yet his third single, Get It While You Can (1967), was a flop. Many listeners, including Joplin, acknowledged the recording as a masterpiece. Tate would later blame Verve, a jazz label, for failing to understand how to market R&B.

Stop, another magnificent Ragovoy-Tate pairing, was a minor hit in 1968 and was later recorded by Jimi Hendrix but Tate, frustrated by his lack of success, split from Ragovoy. In 1970 he recorded an album, Howard Tate's Reaction, for Turntable, but found out that the label was a front for the mafia.

Reuniting with Ragovoy, he signed to Atlantic Records and cut the album Howard Tate (1972). Atlantic also failed to provide Tate with a breakthrough and he and Ragovoy left for Epic, where they recorded one single. After that failed to chart, Tate was consigned to working the chitlin' circuit – black nightclubs across the US south – where each night he would be one of several singers. By 1980 he had left the music business altogether and a few years later was a homeless drug addict.

In 1994 he experienced a religious conversion that helped him clean up. He became a preacher, working with homeless addicts, unaware that soul fans were searching for him and that his classic Verve recordings had been reissued on CD. The DJ Phil Casden had long been begging listeners for information on Tate and in 2000, the veteran R&B musician Ron Kennedy was able to put them in touch. Casden announced to the world that Tate was alive and helped him win the royalties from his Verve days.

Ragovoy then got in contact, suggesting that he and Tate should work together again. The resulting album, Rediscovered (2003), won Tate critical garlands and plenty of work – a memorable concert at Madame JoJo's in London found him singing with remarkable range and power.

He is survived by three children.

• Howard Tate, singer, born 13 August 1939; died 2 December 2011