Charles Van Doren, perhaps the most infamous TV game show contestant ever -- and the subject of an Oscar-nominated movie -- died Tuesday at his Connecticut retirement home ... according to his family.

Charles famously admitted cheating on the NBC game show, "Twenty-One," back in the 1950s. He actually testified before Congressional investigators and said the games were rigged. It was a huge scandal because prime time game shows were enormously popular back then ... drawing up to 50 million viewers.

Robert Redford directed the 1994 movie, "Quiz Show" -- with Ralph Fiennes playing Van Doren -- about the cheating scandal. The movie earned 4 Oscar nominations and 4 Golden Globes nominations.

Van Doren won $129,000 -- which would be more than $1 million today -- but later told Congress he had been given the questions and answers in advance. He also said he was coached up on how to create more drama as he pondered his answers on the show.

Although he was from a very prominent and wealthy Manhattan family ... Van Doren was disgraced after the cheating scandal. He went on to become an editor for Encyclopedia Britannica and wrote a few books.

He'd been living in the retirement community for several years before his death.