On Monday the U.K. Treasury redeemed the outstanding £1.9 billion ($2.8 billion) of debt from the "War Loan", which was originally taken out in 1917 in order to finance the country' huge debts incurred in fighting the four-year war.

Almost 100 years after the debt was was issued, the U.K. government has finally finished paying for World War I.

Originally issued with a coupon of 5 percent, the bond was converted into a lower coupon issue of 3.5 percent in 1932, after the UK's debt mountain became unsustainably high. The Debt Management Office estimates that Britain has paid some £5.5 billion in total interest on the 5 percent and 3.5 percent war loans since 1917.

"In 1914, the national debt was £650 million, but by the end of the war it was around £7 billion and it was financed by bond issuance," said head of retail fixed income at M&G Investments, Jim Leaviss.

"Astonishingly, there are still 38,000 holders of war loan that have under £100 of bonds outstanding. So getting rid of it does tidy up the UK finances, but probably only saves the government around £15 million a year," he said.

The final redemption of the War Loan is part of a wider strategy initiated by the U.K. Chancellor of Exchequer, George Osborne, to get rid of all six undated or "perpetual" bonds without maturity dates in the U.K. government's bond portfolio.

When Osborne announced the move at the end of last year, he said the U.K. government would now be able to refinance this debt with new bonds, benefiting from today's very low interest rate environment.

Read MoreWhy UK has to make business borrowing easier

At the same time, the redemption demonstrated the current government's "fiscal credibility" and is a "good deal for this generation of taxpayers", Osborne said.

"I think it is a good deal for the tax payer. War Loan was a very expensive bond to have outstanding. Its yield was higher than other long-dated bonds that the UK could issue, partly because it wasn't very liquid," Leaviss said.

"On fiscal credibility, I think I could argue. After all since this government has come to power, the U.K. has lost its prized AAA rating. Its deficits have not improved by as much as that in the euro zone or in the U.S. and the government has missed its debt reduction targets as well," he added.