CBS News has learned that during the investigation into Dennis Hastert, the FBI became aware of as many as two, maybe three, potential victims alleging sexual misconduct by the House speaker.

Hastert last month was indicted and charged with violating federal banking laws and lying to FBI investigators. According to the indictment, Hastert agreed to pay $3.5 million in 2010 to a person identified only as "Individual A," in an effort to "compensate and conceal" Hastert's "prior misconduct."

"Individual A" is one of the alleged victims the FBI may have interviewed, according to law enforcement sources.

Word of the more expansive FBI investigation came on the same day of more explosive charges from the family of an alleged abuse victim.

Get Breaking News Delivered to Your Inbox

Jolene Burdge told ABC that Hastert molested her brother, Stephen Reinboldt, all through high school. At the time, Hastert was the wrestling coach and Reinboldt was the student equipment manager at Yorkville High School in Illinois.

"He damaged Steve I think more than any of us will ever know," Burdge said.

Reinboldt is not "Individual A" mentioned in Hastert's indictment. According to Burdge, Reinboldt died in 1995 at the age of 42 from AIDS. When her brother came out as gay, Burdge said he told her a secret.

"I asked him, 'Steve, what was your first same-sex experience?' And he looked at me and said, 'It was with Dennis Hastert,'" Burdge said. "And, you know, I was stunned."

While Reinboldt was a student, he went to the Bahamas along with several others from their local Explorer troop. Hastert was on the trip as well.

"I mean, here was the mentor, the man who was, you know, basically his friend, who was the one abusing him," Burdge said.

Jolene Burdge told The Associated Press on Thursday that the FBI interviewed her in mid-May about Hastert.

Gary Matlock was a Yorkville wrestler from 1969 to 1973, and has called Hastert a mentor. He also went on that Bahamas trip but recalls nothing untoward.

When asked if the allegations have shaken his respect for Hastert, Matlock told CBS News correspondent Dean Reynolds, "I don't know if the word is 'shaken.' It's surprised me. It's disappointed me."

If given an opportunity to speak with Hastert, Matlock says he would ask him, "Denny, what the hell happened?"

Hastert served as speaker from 1999 to 2007 and stood second in line to the U.S. presidency. He has not been charged with sexual abuse.

Instead, Hastert is charged with violating federal banking laws when he made big cash withdrawals and with lying to the FBI about why. His arraignment is set for next Tuesday in federal court.