Washington (CNN) Rep. Elijah Cummings, a longtime Maryland Democrat and key figure leading investigations into President Donald Trump, has died at age 68, his office announced early Thursday morning.

He died of "complications concerning longstanding health challenges," his office said in a statement.

The congressman, who had represented Maryland's 7th Congressional District since 1996, served as the chairman of the House Oversight and Reform Committee, one of the panels involved in the impeachment inquiry of Trump

And it was his committee that grilled Trump's former personal attorney, Michael Cohen, in a blockbuster hearing this past February

It was not immediately clear who will succeed Cummings as chairman of the Oversight Committee or how his passing will affect the swirling impeachment investigation into Trump. The House Democratic Steering and Policy Committee will nominate a new chairman to fill the vacancy before the full House votes to confirm the pick. House Speaker Nancy Pelosi will have a major say on who the next chair will be.

A senior Democratic leadership aide said Thursday that Democratic Rep. Carolyn Maloney of New York "becomes Acting Chair as number two in seniority on the committee. The caucus process to elect a permanent Chair will be announced at a later time."

Cummings had been in failing health in recent weeks. He had been in and out of the hospital, missing votes and business in his committee. He was spotted several times with a breathing tube in his nose connected to oxygen while sitting on the House floor. When speaking to reporters, he would have to wait for 15 to 20 seconds or so to catch his breath before speaking. He would drive around on a motorized wheelchair through the Capitol and then walk in using a walker.

Although he was chairman of the Oversight Committee, he had not been in command of the investigations on his panel. His staff did a lion's share of the work and his staff has been helping lead the charge in the impeachment inquiry.

Still, Cummings' death left a noticeable impact on lawmakers from both parties on Capitol Hill Thursday morning. Pelosi appeared visibly emotional as she started her weekly press conference.

"In the Congress, Elijah was considered a north star," Pelosi said, referring to Cummings as "my brother in Baltimore." "He was a leader of towering character and integrity. He lived the American dream. "

Pelosi said that Cummings "spoke with unsurpassed clarity and moral integrity," adding "everyone wanted to hear what Elijah had to say. That was the case in Congress, in his committee and in the country."

Prominent Trump critic

JUST WATCHED Rep. Cummings makes fiery speech in defense of democracy Replay More Videos ... MUST WATCH Rep. Cummings makes fiery speech in defense of democracy 07:58

As he has led the investigative efforts, Cummings also clashed publicly with the President. Over the summer, Trump tweeted disparaging remarks toward Cummings and his Maryland district, which includes much of Baltimore, calling the majority black district a "disgusting, rat and rodent infested mess."

Responding to some of the President's tweets -- in which Trump suggested the congressman needed to spend more time fixing his district -- Cummings said on Twitter: "Mr. President, I go home to my district daily. Each morning, I wake up, and I go and fight for my neighbors. It is my constitutional duty to conduct oversight of the Executive Branch. But, it is my moral duty to fight for my constituents."

Trump's first reaction to Cummings' death came in a tweet Thursday morning in which he wished his "warmest condolences."

"I got to see first hand the strength, passion and wisdom of this highly respected political leader. His work and voice on so many fronts will be very hard, if not impossible, to replace!" Trump tweeted

Trump and Cummings did not always disagree. More than two years ago, Cummings emerged from a White House meeting with Trump and told reporters that the two men had found common ground on their shared interest in lowering drug prices.

At the time, Cummings also said he urged the President to rethink his language on African American communities after Trump repeatedly painted a grim picture of inner-city life on the campaign trail.

"I want you to realize that all African American communities are not places of depression and where people are being harmed," Cummings told reporters, recalling his conversation with Trump. "When we hear those words about carnage and we are living in depressed situations, I told him it was very hurtful."

In her news conference Thursday, Pelosi mentioned that the House Democrats were considering naming legislation aimed at lowering the cost of prescription drugs, a proposal currently being discussed in the chamber, for Cummings, given his advocacy on the issue.

Friendship with other Republicans

Despite being an outspoken and passionate defender of Democratic causes in Congress, Cummings fostered strong relationship with Republicans across Capitol Hill.

In one high-profile moment earlier this year, Cummings stood up for Republican Rep. Mark Meadows, one of the President's closest allies and staunchest defenders in Congress, in the face of accusations of racism. The chairman referred to Meadows as one of his "best friends." When Meadows learned of Cummings' passing Thursday, he said he was "truly heartbroken."

"I have no other words to express the loss," he told CNN's Dana Bash.

JUST WATCHED Heated debate over racism engulfs Cohen hearing Replay More Videos ... MUST WATCH Heated debate over racism engulfs Cohen hearing 06:38

Rep. Jim Jordan of Ohio, the top ranking Republican on Cummings' committee, said the Maryland Democrat "injected an unyielding passion and purpose into his work on the Committee."

"Our thoughts are also with his staff, who are among the hardest working people on Capitol Hill," Jordan said in a statement. "Their loyalty and affinity for him speaks volumes about his character."

Former Rep. Trey Gowdy of South Carolina, who regularly sparred with Cummings on the polarizing House's committee investigating Benghazi several years ago, said Cummings was "one of the most powerful, beautiful & compelling voices in American politics."

"We never had a cross word outside of a committee room," Gowdy wrote in a series of tweets on Cummings. "He had a unique ability to separate the personal from the work. The story of Elijah's life would benefit everyone, regardless of political ideation."

Leading African American voice on Capitol Hill

Cummings, who grew up during the Civil Rights Movement, had become a leading voice among African American lawmakers on Capitol Hill at the time of his passing, and his death triggered an outpouring of grief from his colleagues.

"He spoke truth to power, defended the disenfranchised and represented West Baltimore with strength and dignity," Rep. Hakeem Jeffries, House Democratic caucus chairman and a fellow member of the Congressional Black Caucus, tweeted Thursday morning. "Congress has lost a Champion. Heaven has gained an Angel of Justice. May he forever #RestInPower."

House Judiciary Chairman Jerry Nadler called Cummings a "giant of public service," and Sen. Ben Cardin said his fellow Marylander "guaranteed a voice to so many who would otherwise not have one."

Rep. Debbie Wasserman Schultz told CNN's John Berman on "New Day" that Cummings was "a mentor and someone who in every situation would do the right thing, would put his community and the cause above everything else. Including himself."

The Florida Democrat said watching Cummings continue with his duties despite his health struggles was inspirational.

"We'll walk in his shadow, in his shoes that will never be filled," she said.

Veteran of Civil Rights Movement

JUST WATCHED CNN analyst: Cummings wanted to work for the people Replay More Videos ... MUST WATCH CNN analyst: Cummings wanted to work for the people 01:17

Earlier this year, Cummings discussed how, even at a young age, he faced racial violence in trying to integrate parts of his neighborhood.

"We were trying to integrate an Olympic-size pool near my house, and we had been constrained to a wading pool in the black community," Cummings told ABC's "This Week" in July . "As we tried to march to that pool over six days, I was beaten, all kinds of rocks and bottles thrown at me."

Cummings said Trump's racist remarks against four minority members of Congress echoed the same insults he heard as a 12-year-old boy in 1962, which he said were "very painful."

"The interesting thing is that I heard the same chants. 'Go home. You don't belong here,' " he told ABC News' George Stephanopoulos. "And they called us the N-word over and over again."

Cummings was born and raised in Baltimore -- the city that is home to his district. The son of former sharecroppers, Cummings was born in 1951 and graduated from Baltimore City College High School in 1969.

He practiced law and served for 14 years in the Maryland House of Delegates, where, according to his congressional website , he became the first African American in Maryland history to be named Speaker Pro Tem.

In 1996, he was first elected to the US Congress. Cummings was reelected last year in the 7th Congressional District with 76% of the vote.

This story has been updated with additional developments Thursday.