Bestselling author James Patterson spoke with CBSN's "Red & Blue" on Tuesday about Jeffrey Epstein's arrest and sex trafficking indictment, criticizing the justice system that allowed Epstein to cut a deal 11 years ago and describing the mysterious financier's abuse of young girls as "an obsession."

Federal prosecutors in New York charged wealthy financier Jeffrey Epstein on Monday with sex trafficking charges, alleging he abused dozens of underage girls as young as 14 over a number of years. Epstein pleaded not guilty.

"This story is about justice breaking down, the justice system breaking down and about how could we since 2006 not pay attention to this story," said Patterson, the author of the 2016 book "Filthy Rich: The Billionaire's Sex Scandal – The Shocking True Story of Jeffrey Epstein."

Get Breaking News Delivered to Your Inbox

"What got me into this was a lot of people had been so hurt by the original situation in 2006, when he kinda got off for 13 months," Patterson said, referring to the non-prosecution agreement Epstein signed for his crimes in 2008.

"When you see the pattern that goes from 2003 or so or maybe earlier ... that's a long time to be obsessed with young girls," Patterson said. "It is an obsession and it is a sickness."

Jeffrey Epstein mug shot Palm Beach Co. Sheriff's Dept.

Patterson worked with collaborator Tim Malloy on the book, and lived in the same Palm Beach neighborhood as Epstein before and during the writing. Even after investigating Epstein, Patterson still seemed stunned by what he uncovered.

"It's so bizarre and insane that somebody would be bringing these young girls to their house, sometimes a couple times a day. Nobody would believe this," he told CBSN's Tanya Rivero.

The indictment alleges that Epstein recruited girls between 2002 and 2005 to "engage in sex acts with him, after which he would give the victims hundreds of dollars in cash. He faces up to 45 years in prison.

Epstein is reportedly a financier who has managed $15 billion worth of personal fortunes for individual clients. Profiles in Vanity Fair and New York Magazine describe him as an "international moneyman of mystery" who is a member the Council on Foreign Relations and the Institute of International Education. In 2003, he donated $6.5 million to Harvard University to set up a program dedicated to the study of evolution, according to a Harvard source familiar with the donation. His net-worth is not known, though his mansion on the Upper East Side of Manhattan is estimated to be worth $77 million.

Epstein taught at the prestigious Dalton School in New York in the 1970s, and was hired by Donald Barr, the headmaster and father of current Attorney General William Barr. Epstein worked at the Bear Stearns investment bank in the early 1980s, before stepping away to begin private wealth management.

"Here's one of the weird things about this, he never graduated from college and yet he taught at the Dalton School," Patterson said. "How did he get in there? A total mystery person. Where did the money come from?"

His alleged crimes have cast a shadow over this careful manicured mythical image of money and philanthropy.

"It's Gatsby," Patterson said. "He liked to be around famous people and he liked to throw parties. Epstein liked stars."

In his 2016 book on Epstein, Patterson writes, "There never was any doubt that Jeffrey Epstein was guilty. He admitted as much in the non prosecution agreement he agreed to sign in 2007. The question is, what exactly was he guilty of?"

Correction: An earlier version of this story reported Epstein's gift to Harvard University as $30 million, but that number has been corrected after communication with a Harvard source familiar with the donation.

Correction: Despite Epstein's foundation website claiming membership, the New York Academy of Sciences has no record of Mr. Epstein's ever having been a Member.