When Kurt Volker agreed to work for the Trump administration in 2017, he told colleagues he hoped to navigate the president’s mercurial nature and his evident attachment to Vladimir Putin, and still pursue a traditional US policy of upholding Ukrainian independence and pushing back against Moscow.

Many were sceptical and predicted a clash between the two approaches, but even they did not expect the spectacular collision of the past few weeks.

In the end Volker, as special envoy on Ukraine, did not collide with Donald Trump ideologically, but found himself at the wrong end of the president’s personal agenda.

Volker resigned last week and now finds himself giving testimony to Congress as the first witness in the rapidly evolving impeachment scandal. By agreeing to set up a meeting between Trump’s personal lawyer, Rudy Giuliani, and an Ukrainian presidential aide, he thought he could stop Trump cutting off ties with the new government of Volodymyr Zelenskiy and keep US military aid flowing,

It was a calculated gambit, but it did not come off. Volker is now fighting to defend his reputation against the impression – enthusiastically conveyed by Giuliani – that Volker was an integral part of Trump’s parallel policy, which centered on digging for compromising material on former vice-president Joe Biden and his son Hunter.

“He really tried to have his cake and eat it, thinking he could operate inside the administration with one foot outside, that would give him some flexibility. But those two things were incompatible,” a former official said.

A dapper, mild-mannered former ambassador to Nato, Volker’s role as special envoy was unpaid and part time. He remained head of the McCain Institute for International Leadership at Arizona State University. The name alone would have normally been enough for Trump to veto his appointment. The institute is named after the late Arizona senator, John McCain, a traditional Republican foreign policy hawk who Trump despised, even after his death from cancer.

However, Volker was hired by former secretary of state Rex Tillerson at a time when Tillerson still had some autonomy and the appointment appears to have gone below Trump’s personal radar.

“Volker was honestly trying to see if he could help move things along in terms of peace negotiations to get the Russians to move on Ukraine,” said Angela Stent, a former national intelligence officer for Russia and Eurasia, now at Georgetown University, who has known Volker for many years. “He never signed a never-Trump letter, because I think precisely he didn’t want to be the position where if somebody needed his expertise, he couldn’t give it.

“Obviously, I’m sure he would never have believed at the beginning that it would have involved brokering meetings for the president’s personal lawyer with individuals in Ukraine.”

Steven Pifer, a former ambassador to Ukraine, said: “Giuliani playing on the sidelines made the job much more difficult, with two policies, one in the US national interest, and the other aimed at promoting Trump’s re-election prospects.”

Further complicating the picture, Volker had remained a senior international adviser to a Washington lobbying group, BGR. BGR and the McCain Institute had financial ties to Raytheon Co, which makes the Javelin anti-tank missile that the US supplied to Ukraine. BGR has said he had recused himself from any issues related to the company’s work on Ukraine.

Ultimately it was not conflict of interest questions that brought Volker down, but the incompatibility of seeking to pursue conventional Republican foreign policy in the midst of the chaos of the Trump administration, where at least two policies were being pursued on Ukraine.

“You had Kurt Volker going everywhere saying we have to be tough on Russia and help the Ukrainians, and the president saying we should have nothing to do with the Ukrainians [because] they are so corrupt,” a European diplomat in Washington said. “This is a bigger issue. There are traditional diplomats who are considering the role of their country in a traditional way, and the higher levels of the administration that are not really engaged.