Story highlights "We must come together to find solutions to move this country forward," Poe said

"Leaving this caucus will allow me to be a more effective member of Congress," he said

(CNN) Rep. Ted Poe announced Sunday he has resigned from the conservative House Freedom Caucus over its opposition to the Republican health care plan, becoming the group's first public casualty in the fall-out over its role in defeating the bill.

"In order to deliver on the conservative agenda we have promised the American people for eight years, we must come together to find solutions to move this country forward," the Texas Republican said in a statement. "Saying no is easy, leading is hard, but that is what we were elected to do. Leaving this caucus will allow me to be a more effective member of Congress and advocate for the people of Texas. It is time to lead."

Speaking on CNN's "New Day" Monday, Poe said the group needs to "change their concept."

"The issue was always the conservatives were not brought to the table in the Republican Party," he said. "Now we've been brought to the table."

Read More