Americans went on a massive shopping spree for cars and trucks in recent years. Many paid for their vehicles by taking out a loan.

A record 107 million Americans have auto loan debt, according to data released this week by the Federal Reserve Bank of New York. That's about 43% of the entire adult population in the US.

It's an eye popping number. Auto loans have been growing rapidly. In early 2012, only 80 million Americans had car loans. In fact, more Americans had home mortgages than auto loans in 2012. But all that has changed.

Today the number of auto loans far outpaces home loans. It helps explain how car makers have had year after year of record sales. Car sales notched another all-time high in 2016, though lately the buying frenzy seems to be over.

Related: Auto sales are slowing, and upheaval is next

6 million Americans are behind on their car payments

Americans don't need to buy new cars every year, and an alarming number of people can't afford to pay their car loans. There are 6 million people who are 90 days or more behind on their car payments, according to the data. That puts them in danger of having their car or truck repossessed.

Beth Yeager has seen the ugly side of the auto loan boom first-hand. She helps run the Pathway of Hope program for the poor at the Salvation Army in Louisville, Kentucky. A lot of single moms with kids end up in her office in a bind. They can't pay their auto loan, but if they lose their car, they often can't get to work.

"This happens to poor people over and over again," Yeager told CNNMoney, the frustration evident in her voice. She loathes car lots with the big banners that say things like "Buy here. Pay here. Easy credit."

Many of the low-income people she works with don't understand credit. They have never had it before. They are so thrilled they qualify for a car that they sign the paperwork, even though the interest rates can be astronomical. She routinely sees interest rates of over 20%.

"Once a car repossession goes on someone's credit report, it impacts not only their ability to get another car, but to get affordable housing," Yeager says.

Related: Household debt tops 2008 peak

There's a 'human side' to loans going bad

After the financial crisis, home mortgages became harder to get, but auto loans were easy. So-called "sub-prime" auto loans were routinely given to borrowers without good credit scores. Low quality loans spiked to pre-crisis levels in 2015 and 2016, according to Fed data.

"There's a human side to all of these loans going bad," Yeager warns.

A mother of three young children came to her for aid earlier this year. Her youngest son was born with complications and needed heart surgery. She lost her job while caring for her child.

Yeager says the mom was confronted with the decision of whether to "keep paying food and rent for the children" or "pay for the car."

The woman stopped paying for her car and it was repossessed. Yeager tried everything, even Goodwill's "cars to work" program, but the black mark on her credit blocked her from help. She walked to work for months before purchasing a "crappy" car for $1,000 that breaks down often. Yeager is still trying to figure out what else to do.

--Have you had trouble paying your car loan? CNNMoney wants to hear your story. Email heather.long@cnn.com