Lexus is joining a growing list of brands which have pulled their ads from Fox News TV show "The O'Reilly Factor" amid allegations of sexual harassment by host Bill O'Reilly, which he has denied.

A New York Times investigation reported on Saturday that he or Fox paid about $13 million to five women since 2002 to settle cases where they alleged inappropriate behavior by O'Reilly.

Fox said in a statement to CNBC that it is working with ad buyers to address their concerns about the show.

"At this time, the ad buys of those clients have been re-expressed into other FNC programs," Paul Rittenberg, executive vice president of advertising sales at Fox News.

Esurance said in a statement that it is also suspending its ads on O'Reilly's show.

BMW, Hyundai and Mercedes previously announced they were pulling their ads as well. BMW said in an emailed statement today: "In light of the recent New York Times investigation, BMW of North America has suspended its advertising with "The O'Reilly Factor."

On Monday, Mercedes-Benz pulled ads from the show because of the "controversy," calling the allegations "disturbing."

"We had advertising running on The O'Reilly Factor (we run on most major cable news shows) and it has been reassigned in the midst of this controversy," Mercedes-Benz USA said in an emailed statement.

"While it's hard to tell what the facts are, the allegations are disturbing. Given the importance of women in every aspect of our business, we don't feel this is a good environment in which to advertise our products right now," it added.

Hyundai has also pulled its advertising, saying in an emailed statement: "Hyundai currently has no advertising running on The O'Reilly Factor. We had upcoming advertising spots on the show but are reallocating them due to the recent and disturbing allegations.

"As a company we seek to partner with companies and programming that share our values of inclusion and diversity. We will continue to monitor and evaluate the situation as we plan future advertising decisions."

Add online marketing company Constant Contact to the list of those removing ties from O'Reilly, too.

The company told CNBC in a statement on Tuesday: "Based on the recent allegations and our strong commitment to inclusion, respect and tolerance in the workplace, we have decided to pull Constant Contacts ads from The O'Reilly Factor."

A representative for Credit Karma told CNBC it will no longer advertise on the show, and has asked for its ads to be pulled.

Pharmaceutical firm GlaxoSmithKline then announced it has "temporarily put a hold on spots running on The O'Reilly Factor" while the company assesses this situation. "We are continually reviewing our advertising to ensure it is conducted in a responsible manner aligned with our values," GSK said in a statement.

And then drugmaker Sanofi's U.S. division said Tuesday it would be reallocating its current advertisements, writing in a statement: " ... allegations made against Bill O'Reilly are matters that we take seriously and will continue to monitor. We do not endorse the behavior or opinions of program hosts or the content."

A statement from O'Reilly given to the New York Times and posted on his website said: "Just like other prominent and controversial people, I'm vulnerable to lawsuits from individuals who want me to pay them to avoid negative publicity. In my more than 20 years at Fox News Channel, no one has ever filed a complaint about me with the Human Resources Department, even on the anonymous hotline.

"But most importantly, I'm a father who cares deeply for my children and who would do anything to avoid hurting them in any way. And so I have put to rest any controversies to spare my children.

"The worst part of my job is being a target for those who would harm me and my employer, the Fox News Channel. Those of us in the arena are constantly at risk, as are our families and children. My primary efforts will continue to be to put forth an honest TV program and to protect those close to me."

When contacted, 21st Century Fox said it was not commenting beyond O'Reilly's statement, and referred CNBC to its statement to the New York Times, which read:

"21st Century Fox takes matters of workplace behavior very seriously. Notwithstanding the fact that no current or former Fox News employee ever took advantage of the 21st Century Fox hotline to raise a concern about Bill O'Reilly, even anonymously, we have looked into these matters over the last few months and discussed them with Mr. O'Reilly.

"While he denies the merits of these claims, Mr. O'Reilly has resolved those he regarded as his personal responsibility. Mr. O'Reilly is fully committed to supporting our efforts to improve the environment for all our employees at Fox News."

Follow CNBC International on Twitter and Facebook.

—CNBC's Ari Levy, Lauren Thomas, Christine Wang and Julia Boorstin contributed to this report.