Ben Carson. Drew Angerer/Getty Images

A top aide to retired neurosurgeon Ben Carson told Business Insider on Tuesday that Carson was "seriously considering" what he characterized as an offer from President-elect Donald Trump to be the secretary of the Department of Housing and Urban Development.

Armstrong Williams, Carson's business manager and close confidant, added that the job was "very attractive" to Carson.

"HUD is a place that has an impact on something that Dr. Carson cares tremendously about," Williams told Business Insider. "The inner city and urban America. Dr. Carson really has a passion for those issues where you could really make a difference and impact America's mayors. And while he would like to remain in private life, the president has made an offer that he will really seriously consider, and that's what he's doing."

Carson had been rumored to also be in consideration for the top positions in the departments of Education and Health and Human Services.

Williams said "all of them are very important in this country," but HUD would be a department that ties closely to "a strong passion that he has."

"It doesn't matter what agency, Dr. Carson doesn't rank one over the other," Williams said. "All are important. But he also understands that, with housing and urban development and what you can do in urban America, in these inner cities, it's very attractive to him. It's a strong passion that he has."

During a Tuesday interview with Fox News, Carson said HUD secretary "was one of the offers that's on the table."

"Our inner cities are in terrible shape, and they definitely need some real attention," he said.

Trump tweeted earlier Tuesday he was "seriously considering" Carson to be his HUD secretary, even though Carson had repeatedly said he was not looking to serve inside a Trump administration.

"I am seriously considering Dr. Ben Carson as the head of HUD," Trump wrote. "I've gotten to know him well — he's a greatly talented person who loves people!"

Carson met with Trump at Trump Tower on Tuesday.

Last week, The Hill reported that Williams said Carson wouldn't join the administration and would instead be an unofficial adviser after reports surfaced that Carson rejected an offer to be HHS secretary.

"Dr. Carson was never offered a specific position, but everything was open to him," Williams told The Hill. "Dr. Carson feels he has no government experience; he's never run a federal agency. The last thing he would want to do was take a position that could cripple the presidency."

Carson subsequently published a Facebook post disputing what Williams attributed as his reason for not seeking a position. And speaking with Business Insider on Tuesday, Williams said Carson never completely ruled out serving in a Trump administration.

"Only on-record interview last week was with Robert Costa of The Washington Post, and he was very clear that his preference was to not take a position, to stay outside," Williams said. "He thought he would be more effective. But if the president did ask, he would have to strongly consider the president's request. And the president has made the request and Dr. Carson is strongly considering."

In an interview with Business Insider just before the election, Carson said, "I don't want to be a part of the administration."

"Not that I have anything against it," he said. "Just that I think my voice will actually be more valuable outside the administration. There are so many issues that affect our country right now, and we can't lose sight of them. So winning the election is really just step one."

"I'll continue to write, continue to speak publicly, and work on helping to focus us as a nation on what's really important," he said.

In a Monday Facebook post, however, Carson seemed to signal a shift in his thinking, though he wrote "there is no reversal of my position in terms of working with the Trump administration."

"I have always made it clear that I preferred to work outside of the government as an advisor, but if called upon, I would serve inside of the government," Carson wrote. "I believe it is important to have voices that are outside of the administration combating media bias and the divisiveness that has infected our country."

Costa tweeted shortly after Trump's tweet that Trump "has been relentless" in pushing Carson to reconsider his position from last week, when he said he was not interested in a Cabinet position.