In the Parramatta Girls Home, a state-controlled institution for neglected girls in New South Wales, children were stripped of their identity and silenced. Their heads were shorn, they were assigned derogatory names, and personal items including sanitary napkins were confiscated. Little girls were deprived of food, forced into harsh labour, and were physically abused, sexually assaulted and raped.

The most “difficult” girls were sent to the Hay Institution for Girls, a maximum-security facility attached to the home where a military-style of discipline was enforced. There the children were made to march, their eyes on the ground. . They lived an alternative moral universe where a twisted regime reigned supreme. The girls were barred from talking to each other or from standing within 10 metres of one another. Girls were drugged. They were forced to undergo invasive and unnecessary medical checks. They were deprived of the toilet and assaulted for wetting the bed.

The home, opened in 1887, was forced to close in 1974.

The culture that exists within organisations is one of the most significant contributors to abuse occurring Institute of Child Protection Studies' Daryl Higgins

It is just one of more than 4,000 establishments, including schools, out-of-home care facilities and churches, in which the royal commission into institutional responses to child sexual abuse has heard catastrophic crimes against children occurred.

In 2014 the attorney general, George Brandis, announced the commission’s work, which began in 2012, would be extended by two years. The commission’s final report was initially due in December 2015. But the victims, survivors and witnesses to historical abuse did not stop coming forward. More staff were needed to handle them. Commission staffing levels peaked at 325 people in October 2016. To date, the six royal commissioners are still holding private sessions for those wanting share their stories of being abused within institutions.

The royal commission report synthesises testimonies from more than 1,200 witnesses. Photograph: Jeremy Piper/PR IMAGE

After more than five years of work, the commission will table its final report to parliament next Friday. A printed version of the 21-volume reportwould require a wheelbarrow to transport. It will synthesise testimonies from more than 1,200 witnesses, transcripts spanning more than 45,400 pages from 444 days of public hearings, and insights gleaned from scrutinising more than 1.2m documents.

It will also contain the central findings gathered from almost 100 pieces of original research the commission ordered be conducted to fill gaps in understanding about how and why abuse of children was so pervasive. What the report will boil down to is a benchmark for society to use to keep children safe in institutions tasked with their care.

Along with the reckoning and the serious attempt to explain this dark and grim history, what survivors, their advocates and those with children of their own are eager to know from the report is: will children in such settings be safe from such atrocities in future? And are they safe right now? Although the commission’s work focussed largely on historical abuse, it commissioned research that has uncovered weaknesses and vulnerabilities in institutions such as residential care that still exist today. And until society addresses these vulnerabilities from the top down and comprehensively changes its attitudes towards children, including “good” and “bad” children, experts say Australia risks failing to learn from the historical travesties the commission and survivors have worked so hard to uncover.

According to research by the Australian Institute of Family Studies, based on reports of sexual assault to police, the vast majority of sexual abuse occurs in homes, but the royal commission was tasked at looking at institutional abuse only. Institutional abuse represents just the tip of the iceberg. Many of the institutions examined by the royal commission, including orphanages, no longer exist. Much has been learned about the care of children and child psychology since those days, and widespread reform has already taken place, including the introduction of mandatory reporting laws.

But Prof Daryl Higgins, the director of the Institute of Child Protection Studies within the Australian Catholic University, says children in the care of institutions will always be at risk as long as leaders within those institutions hold unchallenged, unaccountable power. Higgins is seen by peers as a leader in his field, and received funding from the royal commission to research children’s’ views of safety within institutions.

“I very much hold to the view that it’s leaders who are responsible because leaders are the ones who set the culture within organisations,” he says. “And if you dig through all of the evidence that’s come before the royal commission and the reports that they have that have come out with already, there’s a very clear message that the culture that exists within organisations is one of the most significant contributors to abuse occurring. Not only was abuse being tolerated, but in fact the conditions in which abuse occurs were allowed to flourish by leaders in so many organisations in the past.”

A report published by the commission on Tuesday found paedophiles were allowed to thrive across parishes operated by the Catholic archdiocese of Melbourne for years due to the failure of leaders to do anything to address allegations raised by children, parents and parish staff. Failures of leadership allowed paedophiles to congregate and collude within the safety of church parishes and schools.

The case of paedophile priest Father Peter Searson was remarkable, the commission found, “in terms of the volume of complaints against him and the number of church personnel to whom they were made”. The archbishop in charge between 1974 and 1996, Thomas Francis Little, “dismissed or ignored serious allegations of child sexual abuse against a number of priests,” the commission said. He never reported the allegations to police. Searson was allowed to continue rape children across parishes, from Doveton to Sunbury. He died before facing charges. But seminal research by lawyer Dr Judy Courtin, who has represented dozens of abuse survivors before the commission, found that justice for the abused was not just about perpetrators being charged. Justice also meant the leaders who concealed and protected the crimes of paedophiles also needed to be punished.

To date, not one person has been convicted in Australia for the crime of concealment of child sexual abuse.

Research funded by the royal commission and led by Donald Palmer from the University of California’s sociology department uncovered the organisational culture that exists when children are abused. Palmer found six features conducive to the perpetration of child abuse, one of them being macho cultures where leadership positions are filled by men. To trigger an institutional response to abuse, credible reports about the abuse must be made to leadership, the reportsays.

“However, in private sector organisations, men tend to fill upper-level management positions,” the report found. “Further, in the not-for-profit sector, men tend to fill upper-level management positions, while women fill lower-level staff positions There is no evidence to suggest that the situation is any different in organisations that deliver services to children and young people. As a result, it may be that many detected instances of child sexual abuse fail to trigger a robust institutional response simply because they are observed by women and communicated to men. In patriarchal societies, men are assumed to possess strong sexual impulses and their pursuit of sexual gratification is viewed positively and considered normative.”

Purple ribbons are seen tied to the gates of Knox Grammar School as a tribute to victims of abuse. Photograph: Dan Himbrechts/AAP

A report from the royal commission published in 2014 found that in Parramatta Girls Home, numerous male staff, and occasionally other girls, physically and sexually abused girls. The public hearing heard evidence about 11 men, most of whom were superintendents or deputies at the home. “These men were entrusted with the girls’ care but witnesses spoke of regular bashings, rapes and assaults,” the commission’s report found. “Sometimes, a pair of men would reportedly beat or rape a girl together.” These powerful men who led institutions were often charismatic, adored and respected by parents and other staff. They were subject to no checks or balances.

Manny Waks, the whistleblower who first uncovered abuse within Yeshivah, which practises the Orthodox sect of Judaism known as Chabad, told Guardian Australia from where he now resides in Israel that the Rabbinic leadership “seemed like such good men”. When allegations of abuse within Yeshivah emerged, and accusations were made about leadership covering it up, Waks described how people would say; “they are such good people, they are such moral people, look at their track record, they sit on this board on that board”. “That’s part of the problem, these guys are so charismatic,” he says.

“We need to differentiate between perpetrators of child sexual abuse and perpetrators of other forms of abuse. In particular what we’re talking about here is the abuse of power and authority. That is still an abuse, and it’s a form of abuse that impacts me personally, and many of the other victims and survivors who I know in some cases say it is at least on par with the abuse we experienced by the perpetrators.”

That’s part of the problem, these guys are so charismatic Manny Waks, whistleblower

Waks is frustrated at the reluctance of senior figures who were influential at the time abuse was occurring to step down from positions of power within Yeshivah. Many only did so after public pressure. Some resigned from senior positions only to be reappointed in leadership roles elsewhere. Some, including those who should have known about the abuse, and those who did know but did nothing, continue to hold powerful positions today.

When this unchallenged and unaccountable leadership is combined with a culture that diminishes the voices of children, especially children who are already vulnerable due to their life circumstances or socioeconomic background, conditions become ripe for abuse. The Palmer report describes how organisational cultures that inhibit and undermine children’s voices, and which prioritise the reputation of the organisation over the welfare of victims and the prosecution of perpetrators, allow abuse to thrive.

“The more status and power the perpetrators and their allies possess in an organisation, the more difficult it will be for victims and third-party observers to have their disclosures heard and believed,” the report found. “Victims and third-party observers may pay the price for disclosing abuse with retaliation by the perpetrator and institution.”

Prof Leah Bromfield, the co-director of the Australian Centre for Child Protection at the University of South Australia, for a time served as professorial fellow to the commission, helping to determine its research priorities. The commission had a program of research where it spoke directly to children currently in institutions such as residential care facilities. Researchers spoke to children from preschool through to high-school age, across different ethnic backgrounds and abilities.

“As adults we we cannot glean what children think without asking them,” Bromfield says. The work of Bromfield and others found involving children in decisions that will impact their lives is essential to protecting them from being abused within institutions such as sporting clubs, churches and schools. Historically, however, children were silenced, seen but not heard. While Bromfield says the historical abuse uncovered by the commission is horrific, “I continue to be appalled at the way in which people can treat children.”

“I found the research into children who are living today in residential care and children living today with disability to be devastating,” she says.

“With this royal commission there’s been a real focus on the horrific harm that occurred in orphanages, in children’s homes and institutions of the past. But some of the work of the research agenda really showed that for a particular high-risk group of children, this is not a story of the past. These institutions continue the exploitation of our most vulnerable children and they continue to feel unsafe. So this royal commission has shined a light on this problem. It has exposed a national shame in terms of the way our children were treated in the past. But it’s not over. We can never rest on our laurels. We have to continue to improve the lives of children. Personally, I’ve never lost my ability to be astounded at the way that some adults can treat children.”

Just last month, a scathing report described the appalling treatment of youth within the Don Dale detention facility in the Northern Territory. That report found children were deprived of basic needs, including water, and subject to “regular, repeated and distressing mistreatment”. In Victoria, the state government has been consistently criticised for its treatment of youth in the justice system, with a court finding children were robbed of their human rights and their dignity. A report published in 2015 by the Victorian Department of Human Services found that children living in residential care facilities, who are often there because of abusive family homes and because the foster care system is too overwhelmed to take them in, “are particularly vulnerable to sexual exploitation due to their histories of trauma and abuse and because they are specifically targeted by some sexual predators in the community”.

In a speech given in November, the chair of the royal commission, Justice Peter McClellan, said the sexual abuse of children was not “just a problem from the past”. “Child sexual abuse in institutions continues today,” he said. The commission had been told of cases of abuse, he said, that occurred in the last 10 to 15 years in institutions including schools, religious institutions, foster and kinship care, respite care, health and allied services, performing arts institutions, childcare centres and youth groups. In private sessions commissioners heard about children as young as seven years of age who had been recently abused, he said.

Bromfield says: “There is still a cohort of children who we continue not to value.” She adds that the child protection system is still weighted towards responding to the most high-risk children after abuse has already occurred. “If you think about the many different ways in which services interact with children and families, we identify very early in life those children and families who are vulnerable,” she says.

Manny Waks, who first uncovered abuse within Yeshivah: ‘they seemed like such good men’. Photograph: Mal Fairclough/AAP

“We often identify it in pregnancy, and [yet] we don’t prioritise early intervention. We have a system that’s all about waiting until it’s bad enough to respond. It would be like a health system that was primarily predicated on emergency departments. And with a system like that we miss opportunities to intervene. And then we’re surprised when children end up early school leavers involved in the youth justice system with substance addiction.”

Higgins agrees that the royal commission’s recommendations must focus on prevention. But more rigorous screening of staff who work with children, a popular suggestion, is not the answer to preventing abuse, he says. Screening only picks up on people who have already offended, but will never detect those at risk of becoming first-time offenders. And given most sexual assault offences are never reported to police, police checks and working-with-children screening processes alone can’t be relied on, he says.

“So even if someone has offended, the chances of them being detected and the chances of them having that concerning behaviour recorded somewhere so that [they] could then get picked up in a subsequent screening is still relatively small,” Higgins says. “Another important reason why screening of adults with concerning behaviour or past criminal actions isn’t adequate is that one of the things that young people tell us is that they are at risk of harm not just from adults, be they staff or volunteers, but also from other young people. And so those screening processes are not going to pick up on the risks that young people face from their fellow young people.”

These children are not asking for holidays to Disneyland … they’re asking for respect Leah Bromfield, Australian Centre for Child Protection

Another barrier to keeping children safe in future is that potential perpetrators with problematic thoughts towards children have nowhere to go to seek help or get treatment. Higgins says there are no evidence-based treatment programs for them, and most people harbouring concerning thoughts towards children would never share those thoughts with anyone else, including psychologists. Even if they did, he says, there is a lack of services for psychologists or general practitioners to refer them to.

Bromfield agrees. “We had evidence within the research that showed people right now are calling up helplines like 1800-RESPECT and Lifeline and saying ‘I’m really worried about these thoughts, where can I go for help’,” she says. “There’s nowhere for those people to go. We have this travesty really where there’s only services available after they’ve committed a crime.”

Higgins hopes the commission will make key recommendations around prevention and targeting people before they offend. He also expects some damning findings about the repeated failures of leadership within institutions to believe children, report abuse and act to stop it. Much of the commission’s report is likely to focus on the failures of the Catholic church in particular, and for good reason. As of July, 37%of people who came forward to the commission in a private session reported abuse occurring in Catholic church institutions.

But Higgins hopes that, when the royal commission officially draws to a close next week, the reaction from Australians will not be just to condemn perpetrators and all of those who allowed their abuse to occur by concealing it. He hopes the commission’s work will encourage all adults to examine their own thoughts, attitudes and behaviours towards children and abuse.

“As the population in general we have all contributed to the problem because we as a society haven’t wanted to have a national conversation about sexual abuse prevention,” Higgins says. “Having that conversation means talking about and having the skills to have good conversations with children and young people about healthy sexuality and sexual development, about healthy sexual relationships, consent, and appropriate boundaries between adults and children and young people.

“Adults generally feel uncomfortable about those conversations. Many parents feel ill-equipped and unsupported in having those conversations with their own children. I think collectively as a society we have all let down a generation, or in fact more than one generation of children, when we should have been enlightened enough to know that change requires deep and sophisticated conversations about appropriate sexual development and appropriate boundaries. Despite what we know and despite the way that some of these institutions have changed, we are still letting them down.”

For her part, Bromfield hopes that society learns to listen to children and value their views on decisions that affect them, regardless of their ability or background. One of the six royal commissioners, Justice Jennifer Coate, told a child safety symposium held in Melbourne in May that survivors of child sexual abuse often shared their stories in the hope they could help stop the scourge of child sexual abuse into the future. She said when they were children they were often unheard, or heard but ignored and punished.

Bromfield says institutions tasked with the care of children should look in particular at research conducted for the commission by the Institute of Child Protection Studies at the Australian Catholic University that involved interviews with children and young people living in residential care in Australia. The children were asked about what they thought might prevent sexual abuse, what helped them to feel safe, how well their concerns were responded to, and what could be done to increase their safety.

“When you look at it, these children are not asking for holidays to Disneyland or, you know, pools of money,” Bromfield says. “They’re asking for respect. They’re asking for people to think about what’s best for them when they put them in institutions. They’re asking to be treated with dignity and humanity. And I don’t think that we can say as a society that that’s too hard to give.”