(CNN) Deputy Attorney General Rod Rosenstein discussed with special counsel Robert Mueller what prosecutors should pursue in the Russia probe as early as last May, when Mueller was first appointed, the special counsel's office said Thursday.

At a court hearing, Justice Department attorney Michael Dreeben said Rosenstein discussed specific legs of the investigation with Mueller months before he wrote a classified memo in August that outlined Mueller's directive.

Dreeben argued that the Justice Department shouldn't have to produce documentation of those early talks about the investigation's scope that occurred before Rosenstein's August 2 memo. Rosenstein has been overseeing Mueller's investigation following the recusal of Attorney General Jeff Sessions.

While Rosenstein and Mueller's communication about the Russia probe may help the special counsel's office defend its criminal case against former Trump campaign chairman Paul Manafort, Mueller's team is also indirectly pushing back against the President and Republicans' assertions that the investigation is rogue and corrupt.

"We have an alive and awake acting attorney general who's looking at this investigation, [and] knows what's going on," Dreeben said.

Read More