AFGE later sent a press release calling on Ferguson to remove the items and publicly apologize.

“It is beyond uncomfortable to be told to sit by such disdainful language and materials, and we don’t understand why in this day and age the Congressman has them in the first place,” said Shekina Givens, president of the union that represents TSA workers in Atlanta.

AFGE said Miller reached out to Ferguson’s staff requesting the materials be removed. Miller later told CNN he received a call from Chief of Staff Bobby Saparow on Tuesday morning apologizing and confirming they were taken down.

In a statement released Wednesday, Ferguson said he had been unaware of the book.

“The office was decorated by staff and the book in question was underneath a box of military challenge coins. I did not even know it was there,” Ferguson said. “When my staff learned about it, they removed it and apologized to the individual who was upset by it.”

Ferguson later told CNN he was "certainly as offended by the remarks in that book as anybody would be, and that's why it's no longer in the office."

A second-term House member, Ferguson is seen as an up-and-coming member of Georgia's congressional delegation. The former West Point mayor was recently tapped as the House GOP's No. 2 vote counter and appointed to the powerful Ways and Means Committee.

He attracted similar negative attention last year when his office unknowingly tweeted a D-Day photo that depicted Nazi soldiers rather than American troops.

News of the book traveled fast to the Georgia Capitol, where state Sen. Elena Parent, D-Atlanta, urged Ferguson to publicly apologize in a floor speech Wednesday morning.

“I would like to encourage him to set the record straight and apologize to all of his constituents and the state of Georgia – and especially his African-American constituents,” Parent said. “He needs to apologize for having such a racist item displayed.”

Staff writer Maya T. Prabhu contributed to this article.