(CNN) J Crew's CEO is stepping down less than two years into the job, a sign of turmoil as the retailer heads into the holiday shopping stretch.

In a surprise announcement Saturday, the company said that Jim Brett was leaving because of a disagreement with the board of directors over how to "evolve" J Crew. Four senior executives will share CEO responsibilities.

Brett took over last July from veteran chief executive Mickey Drexler, who led the company for 14 years after a stint at Gap. Drexler helped position J Crew as a top clothing brand known for its quality and preppy aesthetic before a years-long sales slump.

"The departure of Jim Brett is worrying for J Crew," said Neil Saunders, managing director at GlobalData Retail. "It leaves the brand leaderless at a time when it needs to focus its efforts on rebuilding sales and reconnecting with customers."

Brett came to J Crew from West Elm and faced the tough job of revitalizing the brand. J Crew's sales have been on the decline and it carries close to $2 billion in debt from a 2011 leveraged buyout.

Read More