Hobby Lobby will return more than 5,000 ancient Iraqi artifacts allegedly acquired illegally and pay $3 million to settle a federal lawsuit brought by the Department of Justice.

According a Justice Department press release Wednesday, Hobby Lobby, the Oklahoma-based arts and crafts chain, began to collect "historically significant manuscripts, antiquities and other cultural materials" around 2009.

In 2010, Hobby Lobby President Steve Green personally executed a deal to purchase over 5,500 artifacts, comprised of cuneiform tablets and bricks, clay bullae and cylinder seals, for $1.6 million.

Green said that Hobby Lobby imported the artifacts as part of its "passion for the Bible." Green is also chairman of the board of the Museum of the Bible, set to open in Washington, D.C., later this year, which is collecting biblical artifacts.

Just before the agreement, a lawyer warned Hobby Lobby that "the acquisition of cultural property likely from Iraq, including cuneiform tablets and cylinder seals, carries a risk that such objects may have been looted from archaeological sites in Iraq," according to the Justice Department.

The Justice Department says the artifacts originated in Iraq and were smuggled into the U.S. through the United Arab Emirates and Israel.

Hobby Lobby, in addition to paying a fine and returning the artifacts, agreed to strengthen its practices regarding purchase of "cultural property." It will also "submit quarterly reports to the government on any cultural property acquisitions for the next eighteen months."

In a statement, Green said Hobby Lobby "should have exercised more oversight and carefully questioned how the acquisitions were handled."

He said Hobby Lobby "was new to the world of acquiring these items, and did not fully appreciate the complexities of the acquisitions process."

"This resulted in some regrettable mistakes," Green added.

Hobby Lobby is best known for winning a Supreme Court case in 2014 that challenged the Obama administration's contraception mandate.