The battle of words between Germany and the European Central Bank (ECB) is heating up once again and is managing to affect the exchange rate of the euro zone's single currency.

ECB President Mario Draghi said Thursday morning that its current monetary policy stance is adequate and there is no need to start tightening. The euro moved lower against the dollar on his remarks. The currency had been trading slightly up on the day, but fell after Draghi's comments, hitting a three-week low of $1.0629.

But, Bundesbank President Jens Weidmann, asked later on Thursday for an unwinding of stimulus. He told a conference in Germany that the ECB should be careful to avoid its policies making more harm than good. "I could absolutely envisage a less expansionary policy," he said.

"Given that EONIA futures (Euro OverNight Index Average, which is an average of all overnight unsecured lending transactions in Europe) earlier this week had completely unwound their March spike (on the view the ECB could tighten before ending PSPP (bond buying program)) the balance of risks suggests something hawkish could be found in the minutes or Weidmann's speech today, which could lift the EUR. We have slight bias in EUR/$ to 1.0700 perhaps 1.0740 today," ING said in a note on Thursday morning.

This battle over the ECB's monetary policy has been a long one. The German Finance Minister Wolfgang Schaeuble has also stepped into the debate, asking central banks to gradually exit their loose monetary stances. Whereas German officials argue that negative rates and ultra-loose monetary policy are hurting banks' balance sheets, the ECB claims that its action is entirely linked with inflation and that is still below the central bank's target.