In an exclusive statement to PEOPLE, Josh Duggar, his wife Anna, and his parents Jim Bob and Michelle are speaking out in response to reports that Josh was once accused of molesting five underage girls as a teen.

“Twelve years ago, as a young teenager, I acted inexcusably for which I am extremely sorry and deeply regret. I hurt others, including my family and close friends,” Josh, 27, tells PEOPLE in a statement. “I confessed this to my parents who took several steps to help me address the situation. We spoke with the authorities where I confessed my wrongdoing, and my parents arranged for me and those affected by my actions to receive counseling. I understood that if I continued down this wrong road that I would end up ruining my life.”

Get push notifications with news, features and more.

Josh has resigned from his position at the Family Research Council as a result of the accusations becoming public, but his family has stood behind him.

“Back 12 years ago our family went through one of the most difficult times of our lives,” Jim Bob, 49, and Michelle, 48, said in a joint statement. “When Josh was a young teenager, he made some very bad mistakes, and we were shocked. We had tried to teach him right from wrong. That dark and difficult time caused us to seek God like never before.”

Their statement continues, “Even though we would never choose to go through something so terrible, each one of our family members drew closer to God. We pray that as people watch our lives they see that we are not a perfect family. We have challenges and struggles every day. It is one of the reasons we treasure our faith so much because God’s kindness and goodness and forgiveness are extended to us – even though we are so undeserving. We hope somehow the story of our journey – the good times and the difficult times – cause you to see the kindness of God and learn that He can bring you through anything.”

And Josh’s wife Anna, 26, says she has remained supportive of her husband – even after he confessed his “past teenage mistakes” two years before he proposed marriage.

“I can imagine the shock many of you are going through reading this. I remember feeling that same shock,” she says. “When my family and I first visited the Duggar home, Josh shared his past teenage mistakes. I was surprised at his openness and humility and at the same time didn’t know why he was sharing it. For Josh, he wanted not just me but my parents to know who he really was – even very difficult past mistakes.”

But Anna said she had confidence that the counseling her husband received “changed his life.”

“He continued to do what he was taught. [I know] who Josh really is – someone who had gone down a wrong path and had humbled himself before God and those whom he had offended,” she says. “Someone who had received the help needed to change the direction of his life and do what is right.”

And for his part, Josh says he continues living with regret.