INDIANAPOLIS -- Curtis Painter and Larry Coyer are out in Indianapolis, and it could be just the start of the changes for the winless Colts.

Coach Jim Caldwell broke up a normally quiet Tuesday by announcing that Coyer had been fired as defensive coordinator and that Dan Orlovsky would replace Painter at quarterback when the Colts play at New England on Sunday. Longtime linebackers coach Mike Murphy will replace Coyer.

The moves come in the midst of a shocking 0-11 season, Indianapolis' worst start since 1986, and two days after Indy may have lost its last chance at a victory.

"It's not an easy day," Caldwell said. "Anybody that would tell you any differently is cold-hearted and callous, and that's not me."

The relationship between the two coaches dates to the 1970s when Coyer was an assistant coach at Iowa and Caldwell was a player there. And over the past three seasons, Caldwell has gone out of his way to back Painter even after Painter was serenaded by boos when he failed to save Indy's perfect season in his NFL debut two years ago. Caldwell yanked Peyton Manning in the third quarter against the New York Jets in 2009, and Indy lost 29-15, dropping its record to 14-1.

As recently as Monday, Caldwell still seemed to back Painter, saying the Indiana native and Purdue alumni had improved in Sunday's 27-19 loss to Carolina. Painter rallied the Colts after another slow start and got them in position, twice, to force overtime late. Both times Painter threw interceptions in the end zone.

Some thought these changes were overdue and could have been made during the Colts' recent bye week.

But this could be just the start for Indianapolis.

All this losing has prompted fans to call for the firing of Caldwell, vice chairman Bill Polian and general manager Chris Polian, both of whom stood behind Caldwell during Tuesday's news conference. The decision to fire Coyer, Caldwell's first hire and close friend, will undoubtedly create more speculation about the fate of Indy's head coach.

In an interview earlier this month, team owner Jim Irsay said he strongly backed the Polians. He showed more tepid support for keeping Caldwell.

The front office also faces a busy offseason in which it will have to make big decisions about Manning, who still hasn't healed from neck surgery in September, and at least a half-dozen other key veterans. As the clear front-runner for the No. 1 pick, Indy will probably need to decide whether to use a first-round pick on Manning's successor, presumably Andrew Luck.

Quarterback Curtis Painter, center, lost his job Tuesday and coach Jim Caldwell, right, could be next if the Colts continue to lose. Zuma Press/Icon SMI

On Monday night, Bill Polian told listeners on his weekly radio show that changes were coming.

"The message isn't getting across as clearly as it should be," Polian said as he talked about the defense. "We probably have to make some changes there and when you're in the position we are, you should probably be making changes."

Caldwell acknowledged there was a communication gap between the pipe-smoking, 68-year-old Coyer and his players. The defense is ranked 29th overall and 31st against the pass.

But it's not just the defense struggling.

Painter's quarterback rating of 66.6 is the second-lowest among all qualified NFL starters. Only Jacksonville rookie Blaine Gabbert (62.2) has a lower mark.

Still, it's unclear if Painter anticipated that a change was coming after throwing eight interceptions and one TD in his last five games.

"I think coming out of this past game we were able to move the ball and score some points, so I think we have a little bit of confidence," he said Monday. "We just have to trust that we'll get it done on all sides of the ball, and it won't be a big scoring shootout."

Now, it's Orlovksy's job.

The seven-year NFL veteran has appeared in three games this season. He's started seven career games, all with the 0-16 Detroit Lions in 2008. Orlovsky is 14 of 21 for 122 yards with no touchdowns or interceptions this season.

What can be accomplished by making these moves so late in the season?