Former Olympic gymnasts, including Aly Raisman and McKayla Maroney, won’t participate in the U.S. Olympic Committee’s investigation into the sexual-abuse scandal surrounding former national team doctor Larry Nassar, saying they don’t trust the body to conduct a thorough and independent inquiry.

In a letter Friday to the USOC general counsel’s office, a copy of which was reviewed by The Wall Street Journal, a lawyer representing the gymnasts called the investigation “belated” and faulted the USOC for not taking action against Dr. Nassar when top officials learned of the allegations in 2015.

The gymnasts are among more than 150 women who alleged Dr. Nassar sexually abused them, and their refusal to cooperate is a blow to the USOC’s efforts to resolve the crisis. The USOC said last month it had opened an independent investigation, led by law firm Ropes & Gray LLP, into the sexual abuse of national gymnastics team members by Dr. Nassar, who has been convicted of state sexual-abuse and federal child-pornography charges.

The letter from the gymnasts’ lawyer was a response, in part, to calls that USOC Chief Executive Scott Blackmun made late last week to several former national-team and Olympic gymnasts, days after the gymnasts’ lawyer told Ropes & Gray his clients didn’t want to take part in the investigation. Mr. Blackmun called Ms. Raisman late last week on her cellphone, leaving a message, according to her lawyer, John Manly.

Mr. Blackmun also called national-team members Maggie Nichols and Mattie Larson, Mr. Manly said. Ms. Maroney didn’t receive a call, he added. She filed a lawsuit in December against the USOC and others, alleging that the organization failed to protect athletes and had lied about when it learned about the Dr. Nassar allegations.