The Social Security Administration is telling sisters in Oregon and California to pay back approximately $100,000 in survivor benefits their family received after their father vanished and was presumed dead almost 50 years ago. In fact the man, funeral director Douglas Grensted, had faked his death and run off with his mistress.

Grensted lived for decades under the name of Richard Morley, having conned Social Security into giving him a fraudulent Social Security card by stealing the identity of one of the people he buried. He confessed the deception in a letter to the administration shortly before his death in Arizona in 2015, according to the NBC affiliate in Portland, Ore., which first reported the story. That revelation prompted the administration to demand that his wife, Barbara Grensted, pay back $87,000 in survivor benefits she received from the government after he was declared dead by a court in 1976. It is also seeking $10,000 paid to his daughter, Beth Grensted, and $12,000 to his other daughter, Lynne Grensted Thurston.

An administrative court judge, citing the highly unusual circumstances of the case, initially limited Barbara's repayments to just $10 a month until her death, at which time the remaining amount would be forgiven. The judge reversed course after Barbara Grensted's death in September at age 89, just two months after the prior ruling, and said that her surviving daughters, now in their 60s, were on the hook for the rest. They are appealing the judge's ruling.

The daughters contend their mother's death was at least partly due to shock of the case. “All this time we thought my dad was a victim, that something happened to him. And just getting hit with the whole thing of he’s been alive, we were the victims!" Thurston told the Portland NBC affiliate.

Santa Cruz attorney Glen Olives, who is representing the Grensted sisters, says his clients earnestly believed their father vanished on a hunting trip in 1968 and had been dead ever since then. So did the Social Security Administration, he adds.

"In 1976 Barbara went to court with another attorney and got a ruling from a superior court that Doug was in fact presumed to be dead. In 1977 she finally got a letter from the Social Security Administration saying we are no longer going to dispute the fact that Doug was dead," Olives told the Washington Examiner, noting that the 1968 search involved both the FBI and the National Guard.

Olives also claims that the eventual confession that the Social Security Administration used to to bring the overpayment claim to the surviving family members was never dated or properly authenticated.

The Social Security Administration has argued that those claims are irrelevant to the case. "You would think that after 10, 20, 30, 40, 50 years that a statute of limitations might set in," Olives said. "Social Security does not recognize a statute of limitations."

The Social Security Office in Santa Cruz, Calif., which has been handling the case relating to Beth Grensted, did not respond to a request for comment. A spokesman for the SSA office in Salem, Ore., which is handling case relating to Thurston, could not be reached for comment.