The FBI declined a Congressional panel’s request to turn over memos penned by former Director James Comey on Thursday, but committee isn’t giving up that easily.

House Oversight and Government Reform Committee Chairman Jason Chaffetz, R-Utah, first requested the Comey documents on May 16. Chaffetz told Acting FBI Director Andrew McCabe his committee may “at any time” investigate “any matter,” citing House rules.

The memos were written by Comey to detail his meetings and conversations with President Trump regarding the FBI’s investigation into former National Security Adviser Michael Flynn

But the FBI responded on Thursday, a day after Chaffetz’s 24-hour deadline, and said the bureau cannot provide the memos until it consults with Robert Mueller, the special counsel now overseeing the investigation into alleged collusion between the Trump campaign and Russia.

“In light of this development and other considerations, we are undertaking appropriate consultation to ensure all relevant interests implicated by your request are properly evaluated,” FBI’s assistant director for the Office of Congressional Affairs Gregory Brower wrote to Chaffetz. “We will update this response as soon as possible.”

But Chaffetz, who intends to resign from his seat in the House of Representatives on June 30, is not taking no for an answer.

“The Committee has its own, Constitutionally-based prerogative to conduct investigations. But the Committee in no way wants to impede or interfere with the Special Counsel’s ability to conduct his investigation,” Chaffetz wrote in a response letter to Brower. “In fact, the Committee’s investigation will complement the work of the Special Counsel.”

Despite Brower’s initial decision, Chaffetz is continuing the push to obtain memos, notes, summaries, and recordings to assist in the Committee’s investigation. Chaffetz gave the FBI a second deadline to comply with the request by June 8, 2017.

Fox News' Mike Emanuel contributed to this report.