(CNN) President Donald Trump's unannounced visit to Walter Reed National Military Medical Center last weekend spurred speculation about his health from the public, and from doctors. In a statement late Monday night, the White House doctor said that the President underwent "a routine, planned" checkup and attributed "scheduling uncertainties" for keeping the trip off the record.

Trump himself addressed the visit during a Cabinet meeting on Tuesday, saying, "I went for a physical on Saturday" because he had extra time. It was the first time he had been seen in public since that weekend's visit.

Visited a great family of a young man under major surgery at the amazing Walter Reed Medical Center. Those are truly some of the best doctors anywhere in the world. Also began phase one of my yearly physical. Everything very good (great!). Will complete next year.

We know that Trump is 73 years old, has heart disease and is clinically obese. For any man of that age and medical history, an unexpected visit to the hospital is concerning.

Over the past week, I have spoken to doctors who've previously worked at the White House and those who are currently in touch with the White House. They all say that what happened last weekend is unusual: an unscheduled hospital visit for what was characterized as very routine testing -- testing that could have been done at the White House.

A surprise visit to Walter Reed

Given that the White House had previously given plenty of advance notice about the President's past physical exams, last weekend's visit to Walter Reed reportedly took everyone by surprise, including much of the staff at the hospital itself. Whenever the President is planning a visit to Walter Reed, an institution-wide notice goes out, making staff aware of certain road and corridor closings. According to a person familiar with the matter, that didn't happen last weekend.

Also striking: the fact that the president's physician, Dr. Sean Conley, rode with Trump in the presidential motorcade. Typically, the doctor rides separately from the President for security reasons. A former White House doctor told me it had never happened during their time there.

Typically, the president's physician rides separately for security reasons.

I reached out to the White House to get further clarification on what testing or procedures had been done at Walter Reed hospital.

In return, we received on Monday evening a memo from Conley , who wrote "Despite some of the speculation, The President has not had any chest pain, nor was he evaluated or treated for any urgent or acute issues. Specifically, he did not undergo any specialized cardiac or neurologic evaluations."

"The President has not had any chest pain, nor was he evaluated or treated for any urgent or acute issues," according to a memo from Dr. Sean Conley, physician to the President.

Despite Conley's memo, there are reasons why questions continue about President Trump's unannounced visit to Walter Reed.

For starters, all the tests Conley described could've been performed at the White House instead of the hospital. Many blood tests require the patient to fast overnight and are thus performed first thing in the morning -- not in the middle of the afternoon, as apparently happened with the President.

And remember, the President had these tests just nine months ago. One of the reasons doctors wait a year to order labs for a routine physical is to better assess the impact of medication and lifestyle changes over a consistent interval of time. There is no benefit to drawing the blood early, unless there is a concern about something.

Finally, there is no such thing as a phased physical exam, as Trump had described it in his tweet from last weekend.

In Conley's memo, he described the visit as a "routine, planned interim checkup," not a "physical exam." Dr. Jennifer Peña, who served as a physician to Vice President Mike Pence until 2018, told CNN's Jeremy Diamond that these two characterizations are significantly different.

"Routine annual is where we do a comprehensive history and physical exam, with any necessary labs and studies," she said, while noting that an "interim checkup" suggests a "follow up" visit for a condition or medication that is being monitored.

As both a physician and a reporter who has covered four administrations, none of this adds up, and it raises the question: What do we really know about President Trump's health?

An astonishing bill of health

Take a look back at what we've been told about Trump's health over the years.

I've reported in the past about doctors who seemed to cater to Trump's whims. One doctor who treated the President signed his name on a letter he later said Trump dictated. Another doctor said that he might live to be 200 years old.

When Trump was running for office in 2015, his campaign issued a letter from Trump's then-personal physician , Dr. Harold Bornstein, who wrote "His physical strength and stamina are extraordinary." Bornstein added, "If elected, Mr. Trump, I can state unequivocally, will be the healthiest individual ever elected to the presidency."

A letter from Dr. Harold Bornstein, Trump's personal physician, was released by the Trump campaign in 2015. Bornstein later said Trump "dictated that whole letter."

The letter stated that Trump "has had no significant medical problems" and called the candidate's blood pressure and lab results "astonishingly excellent." Bornstein wrote that his blood pressure was slightly better than average at 110/65 and that Trump had never had any form of cancer and only one operation in his entire life, when he was 10.

The hyperbolic letter immediately raised flags. It wasn't a standard medical assessment of a patient's health, which is an objective record. Aside from the flowery writing, it was notable that there was very little medical history described aside from a note that he took a daily aspirin and a low-dose statin.

It was hard not to notice the similarities between the over-the-top language used in the letter and the language that Trump was fond of using. Just over two years later, Bornstein alleged that very letter praising the candidate as a vision of health had been dictated by Trump himself. "He dictated that whole letter. I didn't write that letter," Bornstein told CNN . "I just made it up as I went along."

A history-making news conference

At the age of 70, President Trump was the oldest president sworn into office in American history and he used his first physical exam in 2018 as an opportunity to quiet critics who questioned both his physical and mental well-being.

The results of his exam were revealed during a news conference with White House physician Dr. Ronny Jackson on January 16, 2018.

I don't recall ever covering a press conference where the president's routine physical exam was the topic of discussion. The only other known White House physician to address the public on the results of the president's physical was Dr. William Lukash, President Gerald Ford's physician

Jackson said that President Trump asked that he be administered a cognitive exam, which is not a routine part of a physical. Jackson, who had also been the physician to President Barack Obama, said it was the first time he was aware of any president taking any sort of cognitive test.

"He actively asked me to include that in it, so we did," Jackson told reporters, including myself. Jackson said Trump received a 30 out of 30 on the Montreal Cognitive Assessment , a quick screening for mild cognitive impairment and early Alzheimer's disease but not a diagnostic tool.

During the conference , we learned that Trump's blood pressure was normal, but both his weight, 239 pounds, and his total cholesterol level, 223, were high.

In the official readout of the physical exam , the White House did not reveal that Trump also had a coronary calcium scan done. This test uses a CT (computed tomography) to look for the buildup of calcium plaques on the walls of the coronary arteries, which supply blood to the heart.

JUST WATCHED Gupta questions Dr. Ronny Jackson on Trump's health (2018) Replay More Videos ... MUST WATCH Gupta questions Dr. Ronny Jackson on Trump's health (2018) 00:20

It was only after I questioned Jackson that he revealed the President had undergone the heart scan and scored 133, indicating that Trump had a common form of heart disease. According to the Mayo Clinic , "A score of 100 to 300 means moderate plaque deposits. It's associated with a relatively high risk of heart attack or other heart disease over the next three to five years."

Previous medical records indicate that in 2009 his coronary calcium score was 34, and in 2013 it was 98.

Jackson was prescriptive. "We'd like to get the LDL down below 120, so that's what we'll be shooting for," he said. He added that he would like to see the President lose 10 to 15 pounds. He also increased the dosage of his statin.

But despite all of that, Jackson, like Bornstein before him, was also extraordinarily effusive about the President's health. "It's called genetics. I don't know. Some people have just great genes." Jackson added, "I told the President that if he had a healthier diet over the last 20 years, he might live to be 200 years old."

Jackson eventually became embroiled in political controversy over his nomination later that same year to lead the US Department of Veterans Affairs. Jackson is now considering a run for a Texas Congressional seat.

That is when Dr. Sean Conley, a doctor of osteopathy , took over the responsibilities of overseeing the President's health.

The President's 2019 physical

President Trump's physicals in early 2018 and early 2019 were performed by different doctors.

President Trump's most recent full physical examination took place in February 2019, and it revealed that the 6-foot-3-inch Trump weighed 243 pounds, 4 more pounds than the previous year. It put his body mass index, or BMI, at 30.4, making him clinically obese by the guidelines from the National Institutes of Health

The results released by Conley also noted that the President had increased his dose of rosuvastatin , a medication used to treat high cholesterol, to 40 mg per day. His cholesterol levels showed a total cholesterol of 196 -- HDL of 58, and LDL of 122 -- a decrease from the year before.

Trump's blood pressure measured 118/80 and the President had been vaccinated for shingles and pneumococcal diseases, such as meningitis and pneumonia.

Conley said the President was seen by 11 different specialists at Walter Reed, that the physical took "approximately four hours" and did not require any sedation or anesthesia.

Why doctors are worried about Trump

So, the question remains: Why was there a need for President Trump to be at a hospital on a Saturday afternoon? Any doctor who heard about a man with a history of heart disease and clinical obesity making an unannounced visit to a hospital would be worried the patient may be having symptoms that warrant that visit.

That is the reason why so many in the medical community remain concerned. Both a former White House doctor and a doctor currently in contact with the White House have raised their unease with me. "It's concerning to me. It doesn't make a lot of sense, you know, to do that kind of testing at Walter Reed without, really without provocation," said Dr. Jonathan Reiner, cardiologist to former Vice President Dick Cheney. Reiner is authorized by the White House to speak about last weekend's incident but is not privy to all of Trump's medical records. He and the other doctor I spoke with expressed concern that a full picture of the President's health is not being revealed.

Bornstein, Jackson and Conley have all shared information about President Donald Trump's health with the public.

Even basic information such as the President's height and weight weren't disclosed this time, let alone the results of any more sophisticated testing, once again leaving an incomplete and opaque picture of his health. Rather than detail what tests were performed last weekend, Conley emphasized what wasn't performed on Trump in his memo . "Specifically, he did not undergo and specialized cardiac or neurologic evaluations," wrote Conley.

The health of the President has far-reaching implications. The President has no obligation to share any of his health history with the public, but we should be confident that the president of the United States is receiving the best care possible.

On Friday, the president called into Fox and Friends and said the hospital visit was done at the suggestion of his doctor. "The doctor calls, "Sir, would you like to go out because I have three hours. Would you like to go out to Walter Reed and do your first part of the physical?"

Get CNN Health's weekly newsletter Sign up here to get The Results Are In with Dr. Sanjay Gupta every Tuesday from the CNN Health team.

Regardless of what prompted the trip to Walter Reed, the President's surprise visit to the hospital sends an important message. Get checked out if you are having unusual symptoms. While we don't know what symptoms, if any, pushed Trump to go to the hospital last weekend, it appears as if they have resolved and were unlikely severe or requiring intensive therapy. Conley wrote that the visit was just a little more than an hour of exams, labs and discussions and Trump has been in the public eye since then.

The greater concern here isn't whether the White House medical team is being honest with the public. It's whether the team is being honest with the President himself. If they are simply following the patient's orders instead of acting independently, they are treating the President's whims as much as they are treating his health.