At Monday's Women Tell All special, the Armed Forces member revealed details about her childhood she didn't share with Bachelor Ben Higgins

She went home on week five, but Jubilee Sharpe was one of the most talked about women on Bachelor Ben Higgins‘ season, in part because of her heartbreaking past. And for the first time, Sharpe, who told Higgins that her entire family passed away, is opening up about what really happened.

“I told Ben the story of my family, but I left out the biggest part,” Haitian-born Sharpe tells PEOPLE. “My three brothers and my parents died, but I don’t know the details. I was 6, but I don’t remember my parents. My 4-year-old little sister and I went to live with my grandmother but she had leprosy and was dying and too sick to take care of us. So she put me and my little sister in an orphanage.”

Get push notifications with news, features and more.

Continues Sharpe, “My dad came to Haiti to do relief work and he showed up to the orphanage and [wanted to adopt me]. He tried to adopt my sister, but after they went through the medical process, they found out she had an incurable disease and wasn’t able to be adopted.”

A couple of years ago, Sharpe, 25, tried in vain to find her sister.

“The person who did my adoption told me she was probably dead,” says Sharpe. “That’s where my guilt had come from.”

• PEOPLE’s collector’s edition The Bachelor, with new interviews and updates on your favorite contestants, is on sale now in stores and online!

Confronting her emotional past wasn’t the only difficult experience Sharpe had on the show. During Monday night’s Women Tell All, Jami Leitan and Amber James accused Sharpe of making offensive remarks. Sharpe insists she would never do such a thing.

“I said I am the one full black girl in the house,” says Sharpe. “I think of that as a fact. The way they tried to portray me really got to me, like I was the racist black girl. Race has never been a thing for me. My whole [adopted] family is white. I wasn’t raised seeing color.”