Former FBI Director James Comey on Friday defended his conduct and the bureau's handling of the months-long investigation into Hillary Clinton's private email server, even as he acknowledged in hindsight that the actions of key officials involved in the case may have created an appearance of bias.

In transcripts of a closed-door hearing released on Saturday, Comey reiterated that there was "no substantive case" against the former Secretary of State for mishandling classified information. As part of an agreement with the Republican-led House Judiciary Committee, lawmakers agreed to release a full transcript of Comey's testimony within 24 hours.

However, Comey told the U.S. House Judiciary Committee and House Oversight Committee that he was unaware of a series of text messages between Peter Strzok and Lisa Page, two agents formerly assigned to Special Counsel Robert Mueller's Russia probe.

Strzok and Page were romantically linked, and exchanged thousands of messages disparaging to President Donald Trump during and after the election, while appearing to promote Clinton's candidacy. Replying to a Congressman's question, Comey suggested he would have blocked the agents from continuing in their roles in the probe.

"To the extent you're asking about communications of Page and Strzok, if I had known about those things that they were communicating that I've seen in [the media], I would not have had them...playing any role in connection with that investigation," Comey said, in response to a lawmaker's question.

"The challenge for me is I haven't read all the texts, but based on what I saw -- have seen in the media since I left the FBI, that unless there was some explanation for that that I was missing, in my judgment, they wouldn't have remained part of the investigation," Comey added.

He also characterized the infamous Phoenix tarmac meeting between former Attorney General Loretta Lynch and former President Bill Clinton during the probe of his wife as inappropriate given the circumstances. "I was very concerned by the appearance of that interaction," Comey said, according to the transcript.

"I remember being concerned about whether she should remain involved, especially after the tarmac visit," the transcript of Comey's remarks read. "But before I had an opportunity to discuss that with anyone at DOJ, the Attorney General announced that she would not recuse but would accept my recommendation and that of the career prosecutors."

Comey sidestepped many of the questions related to the ongoing Russia investigation, but repeated that Trump was not the subject of the 2016 counterintelligence investigation involving four unnamed people linked to his campaign.

The transcripts cover the first of what is expected to be two days of hearings with Comey in the House of Representatives. The former FBI director is due to appear before the committees again on Dec 17.

Download a copy of the transcript here.

--Reuters and CNBC's Kevin Breuninger contributed to this article.

Read the transcript of Comey's entire testimony below.