(CNN) The Internal Revenue Service accumulated a backlog of 5 million unsorted pieces of mail during the five-week government shutdown, according to an agency watchdog report released Tuesday.

The National Taxpayer Advocate, an independent organization within the IRS, delivered an annual report to Congress that painted a dismal picture of the federal tax collector as already stretched thin trying to implement changes associated with the 2017 tax overhaul just as the shutdown hit.

Phone lines to reach the IRS for taxpayers wishing to make payment arrangement was "abysmal," the report said, with average wait times of 93 minutes. Even after the shutdown was over, 93.3% of taxpayers calling to make payment arrangements were unable to speak to a live agent, according to the report.

For the first week of the filing season, after the shutdown ended, there was a "shocking" decrease in service levels on the agency's phone lines, National Taxpayer Advocate Nina Olson wrote in the report, which was also delayed due to the shutdown. That included average wait times of 17 minutes rather than four minutes in 2018 for account management lines.

"Make no mistake about it, these numbers translate into real harm to real taxpayers," Olson wrote.

Read More