WASHINGTON — President Trump, fresh off replacing his secretary of state and C.I.A. director, is considering firing his secretary of veterans affairs and installing Energy Secretary Rick Perry in the post, according to two people close to the White House.

Mr. Trump did not make a formal offer to Mr. Perry when the two men met on Monday. But the people said the president has grown impatient with the department’s current secretary, Dr. David Shulkin, and may want to replace him with someone already in his cabinet.

It was unclear if Mr. Perry, who was an Air Force pilot before entering politics, would accept the change in position if Mr. Trump offered it, or if Mr. Trump had a successor in mind to lead the Energy Department.

The White House did not immediately respond to requests for comment, nor did Dr. Shulkin.

The conversation follows weeks of bitter infighting at the Veterans Affairs Department, where Dr. Shulkin has faced off in a rare public spat with a prominent group of Trump administration appointees who want to see him removed from office. The dispute goes beyond personality to a struggle over how far and how fast to privatize health care under the V.A. system.