Delhi (CNN) President Donald Trump's acting chief of staff Mick Mulvaney did not travel with him to India as scheduled at the last minute, according to four people who say he has a cold and was advised not to go.

A senior administration official who briefed reporters Friday originally listed Mulvaney along with 11 others who were expected to travel with the President for his two-day visit to the world's most populous democracy, but Mulvaney did not board Air Force One with the others Sunday, according to two people familiar with what happened.

Several aides found out Sunday that Mulvaney would no longer be traveling to India. According to one person close to Mulvaney, White House doctors advised against having him travel in such close proximity to the President and first lady for the two-day trip

Mulvaney also did not travel with Trump on his four-day swing out West last week either, which aides took as a potential sign he likely wouldn't remain in the job much longer because both of his predecessors stopped traveling with the President near the end of their tenures, too.

Instead he was in London, where during a question-and-answer session at Oxford Union, Mulvaney said: "I disagree with the President every single day. You just don't hear about it -- that's not my job."

Read More