President Trump told the White House counsel earlier this year that he wanted to direct the Justice Department to prosecute his political enemies, including his 2016 election rival Hillary Clinton and fired FBI Director James Comey, according to a new report Tuesday.

But Don McGahn, who was the White House lawyer at the time, told the president that he had no authority to order the prosecution of Clinton and Comey and warned Trump that doing so could lead to his possible impeachment, the New York Times reports.

Following the request, White House lawyers wrote a memo strongly cautioning Trump against ordering the Justice Department to investigate anyone. In the memo, the lawyers said such an action could potentially be seen as an abuse of power by Congress and voters, which could lead to his impeachment or being voted out of office.

It’s not clear whether Trump read the memo or continued to press his aides further about prosecuting Clinton and Comey, the report notes.

“Mr. McGahn will not comment on his legal advice to the president,” said McGahn’s lawyer, William Burck, told the Times. “Like any client, the president is entitled to confidentiality. Mr. McGahn would point out, though, that the president never, to his knowledge, ordered that anyone prosecute Hillary Clinton or James Comey.”

During the 2016 presidential campaign, Trump threatened to prosecute Clinton if he was elected president. Trump has repeatedly criticized Clinton over her use of a private email server while she was secretary of state, as well as her role in the controversial “Uranium One” deal.

“If I win, I am going to instruct my attorney general to get a special prosecutor to look into your situation,” Trump told Clinton during a debate, later telling her she “would be in jail” if he was in charge of the country.

Trump has accused Comey of leaking classified information after the memos Comey wrote about his interactions with the president were shared with the media. The memos reportedly contained no classified information.

Vox reported earlier this month that Acting Attorney General Matthew Whitaker had given Trump advice about how the White House might be able to convince the Justice Department to appoint a special counsel to investigate Clinton.