Dylan Farrow’s Feb. 1 open letter to The New York Times detailing sexual molestation she says she suffered at the hands of her father Woody Allen reignited a controversy that has divided their family bitterly for more than 20 years.

Now her brother Moses Farrow is speaking out to defend Allen – and accuse their mother, Mia Farrow, of poisoning the children against their father.

Get push notifications with news, features and more.

“My mother drummed it into me to hate my father for tearing apart the family and sexually molesting my sister,” Moses, 36, tells PEOPLE in the magazine’s new issue. “And I hated him for her for years. I see now that this was a vengeful way to pay him back for falling in love with Soon-Yi.”

A Family Divided

Moses and Dylan, 28, both adopted by Allen and Farrow, and their brother Ronan, 26, were in the center of a 1993 custody battle in which both sides testified about Allen’s affair with Farrow’s adopted daughter Soon-Yi Previn, whom Allen went on to marry in 1997. Farrow was awarded custody of the couple’s three children. (In total, she has 14 kids from her marriages and solo adoptions.) Allen, 78, who was investigated but not charged with molestation, has for decades denied abusing Dylan, maintaining that Farrow, 69, coached Dylan, an accusation Farrow has always denied.

“Of course Woody did not molest my sister,” says Moses, who is estranged from Farrow and many of his siblings and is close to Allen and Soon-Yi. “She loved him and looked forward to seeing him when he would visit. She never hid from him until our mother succeeded in creating the atmosphere of fear and hate towards him. The day in question, there were six or seven of us in the house. We were all in public rooms and no one, not my father or sister, was off in any private spaces. My mother was conveniently out shopping. I don t know if my sister really believes she was molested or is trying to please her mother. Pleasing my mother was very powerful motivation because to be on her wrong side was horrible.”

Dylan’s Response

Dylan insists that she is telling the truth.

“This is such a betrayal to me and my whole family,” she tells PEOPLE in response to her brother’s comments. “My memories are the truth and they are mine and I will live with that for the rest of my life.”

“My mother never coached me,” Dylan says. “She never planted false memories in my brain. My memories are mine. I remember them. She was distraught when I told her. When I came forward with my story she was hoping against hope that I had made it up. In one of the most heartbreaking conversations I have ever had, she sat me down and asked me if I was telling the truth. She said that Dad said he didn t do anything. and I said, ‘He’s lying.’ ”

Painful Memories

Moses accuses Farrow of bullying him as well. “Our mother has misled the public into believing it was a happy household of both biological and adopted children,” he says. “From an early age, my mother demanded obedience and I was often hit as a child. She went into unbridled rages if we angered her, which was intimidating at the very least and often horrifying, leaving us not knowing what she would do.”

“I don’t know where he gets this about getting beaten,” counters Dylan. “We were sent to our rooms sometimes.”

“I will not see my family dragged down like this,” she adds. “I can’t stay silent when my family needs me and I will not abandon them like Soon-Yi and Moses. My brother is dead to me. My mother is so brave and so courageous and taught me what it means to be strong and brave and tell the truth even in the face of these monstrous lies.”

Farrow, who declined to respond to Moses’s accusations, Tweeted, “I love my daughter. I will always protect her. A lot of ugliness is going to be aimed at me. But this is not about me, it’s about her truth.”

Moses, a family therapist, says that his own life has been made better by spending time with Allen.

“I think my sister is missing a great deal in life in not reconnecting with her father, who had always adored her,” he says. “It s important that she assert her independence from our mother and not go through life with the false impression that she has been molested by my father. I am very happy I have come into my own power, separating from my mother, which has led to a positive reunion with my father.”

‘Horrible Tragedy’

Allen’s family says that the director is devastated by Dylan’s letter.

“This is a horrible, horrible tragedy,” Allen’s sister Letty Aronson tells PEOPLE. “He feels very badly for Dylan, that she has been so poisoned by her mother.”

Dylan, of course, feels very differently.

“I have a wonderful family,” she says. “We are brave and we are truthful and anyone who says anything otherwise does not know us.”

• With reporting by K.C. BAKER