Imagine keeping up with your Wells Fargo car loan payment every month -- but having your vehicle repossessed anyway. That's exactly what Samir Hanef said happened to him.

"My car was held as extortion and I was forced to pay for Wells Fargo's mistake," Hanef told CNNMoney.

"The stress and anxiety ... are truly indescribable," said the clinical social worker from Durham, North Carolina.

Hanef is a victim in the latest Wells Fargo scandal. He is one of up to 570,000 auto loan borrowers that Wells Fargo has said it may have enrolled and charged for car insurance without their knowledge. Wells Fargo has admitted that as many as 20,000 of those customers may have defaulted on their car loans or had their vehicles repossessed in part due to these unnecessary insurance costs.

Wells Fargo told CNNMoney that it plans to work directly with Hanef to give him a refund.

"We are truly sorry for any inconvenience this caused our customers," a Wells Fargo spokeswoman said in an email to CNNMoney.

Related: Wells Fargo may have forced 570,000 into unneeded auto insurance

Unlike most big banks, Wells Fargo's auto loan contracts allowed the lender to obtain collateral protection insurance on a customer's behalf if they failed to buy liability coverage themselves. Wells Fargo conceded that it bought insurance for some customers -- and charged them for it -- even when they had their own.

Lenders can impose collateral protection insurance on borrowers if their normal coverage lapses. It's designed to protect from a financial loss in the event the borrower gets hit with an insurance claim, such as for an accident, and doesn't have any coverage.

Wells Fargo told CNNMoney that it has discontinued this insurance program after "finding errors" that hurt some customers.

Most big banks don't force auto insurance on borrowers who don't have it. JPMorgan Chase (JPM), Bank of America (BAC), PNC Financial (PNC) and Santander Consumer USA told CNNMoney that they do not follow this practice.

This week, a New York regulator subpoenaed two Wells Fargo divisions, demanding the bank turn over loan agreements and other documents related to the auto insurance charges, a source told CNNMoney.

Related: Wells Fargo hit with subpoenas over auto insurance scandal

Wells Fargo's auto insurance scandal comes nearly a year after the bank said it had created about 2 million potentially unauthorized checking and credit card accounts.

Hanef, a Wells Fargo customer for 23 years, said he took out a Wells Fargo auto loan in June 2014 to purchase a used Honda Civic. To be safe, Hanef said he always paid $300 per month, even though the bill was only $280.

The problem was that starting around March 2016, Wells Fargo (WFC) raised his monthly bill to $374 because it added the collateral protection insurance. Hanef insists he had auto insurance, so there was no need for Wells Fargo to charge him for more.

"My insurance never lapsed," he said, echoing the claims of other customers. Wells Fargo told Hanef that it sent him notices about the insurance charges, but he never saw them.

Hanef said that he didn't notice the insurance charges added to the monthly invoices sent to his home. He admits he stopped looking at the bill closely because for almost two years the amount due never changed.

"It always stayed the same," Hanef said.

He didn't know there was a problem until it was too late. Hanef's Honda Civic was repossessed on December 27, 2016, according to a letter sent by Wells Fargo to Hanef and viewed by CNNMoney. Losing his car meant that Hanef missed work and inconvenienced his patients, many of whom have mental health and substance abuse problems.

Before he got his car back on January 10, 2017, Hanef said he was forced to pay the repo fee and get current on the loan -- or risk losing the car for good.

"They told me the car would be sold on auction if I didn't pay the ransom," Hanef said.

Wells Fargo wrote Hanef a letter on January 31 saying it removed the insurance surcharge after he proved he had insurance. But the bank refused to refund the repo fee because it said he would have been late on his payments in any case.

The worst part? Hanef said his credit score has been "decimated" by the Wells Fargo experience, dropping more than 100 points. Now he's not able to refinance his house.

Related: Wells Fargo ordered to rehire whistleblower

Wells Fargo said last week it's "extremely sorry" for the auto insurance debacle. The bank promised to pay $80 million in refunds and account adjustments. Wells Fargo said it will also work with credit bureaus to correct errors in customers' credit records.

Hanef is considering joining a class action lawsuit launched this week by the law firm Keller Rohrback over the auto insurance scandal. The firm reached a $142 million preliminary class action settlement with Wells Fargo earlier this year over the fake account scandal.

Senator Elizabeth Warren and some of her Democrat colleagues want to grill Wells Fargo CEO Tim Sloan and Chairman Stephen Sanger about the recent scandals. They wrote a letter on Tuesday to Mike Crapo, the Republican chairman of the Senate banking committee, requesting a September hearing. Crapo's committee did not respond to a request for comment from CNNMoney.