(CNN) -- Texas Gov. Rick Perry has removed a fourth member of a state commission charged with investigating claims that an innocent man may have been executed, his office said.

A family photo shows Cameron Todd Willingham with his wife, Stacy, and daughters Kameron, Amber and Karmon.

The Texas governor has now replaced all of the four members that, under law, he is allowed to appoint to the commission. The remaining five members are appointed by the state's lieutenant governor and attorney general.

Perry's critics say his actions are politically motivated, a charge he denies.

The investigation into claims that faulty evidence led Texas to execute an innocent man in 2004 was at a "crucial point" when the shakeup occurred, one of the replaced members said.

The commission was to hear from the author of a scathing report in the case of the executed man, Cameron Todd Willingham, when Perry announced on September 30 that he would replace three members.

The session was postponed indefinitely because of the new appointments, and Perry's critics accused him of trying to quash the Willingham probe.

"I think people are making a lot of this issue," Perry said earlier.

Commissioner Alan Levy was replaced by John Bradley, a district attorney who was also named commission chairman. Aliece Watts was replaced by Norma Farley, chief forensic pathologist for Hidalgo and Cameron counties. Perry said at the time the replacements were "pretty normal protocol."

In a statement Thursday, Perry's office said he had appointed attorney Lance Evans of Fort Worth to replace former commission chairman Samuel Bassett of Austin, and that Randall Frost of Boerne, chief medical examiner for Bexar County, would replace commissioner Sridhar Natarajan.

"If you've got a whole new investigation going forward, it makes a lot more sense to put the new people in now and let them start the full process, rather than bring people in there for a short period of time and then replace them," Perry said two weeks ago. "I think it makes a whole lot more sense to make a change now than to make a change later."

But the Forensic Science Commission began investigating the Willingham case in 2008, hiring Maryland fire investigation expert Craig Beyler to examine the evidence used to convince a jury the fire that killed Willingham's three daughters was deliberately set.

Levy said at the time of his replacement he had told Perry's office "that it would be disruptive to make the new appointments right now."

"The commission was at a crucial point in the investigation," he told CNN. Asked about the future of the Willingham investigation, he said, "I don't know if it will ever be heard."

Both Levy and Bassett said they had asked to remain on the board.

Evans declined to give an opinion on the Willingham controversy when contacted by the Fort Worth Star-Telegram for a story Friday, saying all he knows on the matter is what he's read in the newspaper.

"Because I haven't had involvement with the commission, I can't really comment on what's been going on in the commission in the past," he said. "I will work very hard to make sure the duties of the commission ... as given by the Texas Legislature are carried out."

Beyler's report -- the first commissioned by a state agency -- is the latest of three to conclude that arson was not the likely cause of the 1991 fire.

Death-penalty opponents say an impartial review of Willingham case could lead to an unprecedented admission -- that the state executed an innocent man. The Beyler report concluded that the ruling of arson at the heart of Willingham's conviction "could not be sustained" by modern science or the standards of the time.

Perry, who faces a Republican primary challenge in his bid for a third term next year, refused to issue a last-minute stay of execution for Willingham in 2004 and has said he remains confident that Willingham was guilty. So have authorities in Corsicana, south of Dallas, who prosecuted Willingham in his daughters' deaths.

Asked earlier whether the governor wants to see the Willingham investigation go forward, Perry spokeswoman Katherine Cesinger told CNN, "That's a decision of the commission."

However, the first replacements raised concern among Willingham's relatives, who worked to avert his execution and to clear his name after his death.

"It sounds like someone made Governor Perry mad," his stepmother, Eugena Willingham, said after hearing the news during an interview with CNN at her home in Ardmore, Oklahoma.

"I think it's going to delay things," she added. "It makes me wonder why."

Neither Bassett nor Levy would say whether they believed political considerations were behind their replacement, though Bassett said in a written statement that the investigation should not be dropped "because there might be political ramifications."

Bradley told CNN at the time of his appointment he did not believe it was meant to stall the Willingham probe.

"People tend to read into these things the kind of preconceived notions they come to these issues with," Bradley said. "They're not going to change those positions overnight. They're going to watch how the commission does its work." He said if he does his job right, the commission will "probably satisfy most of those people."

CNN's Matt Smith contributed to this report.

All About Capital Punishment • Rick Perry • Texas