"I was meant to pick her up that day for court," says Jeannette Chambers, with a painful smile. She tucks her chin down, gulps for breath, then looks up with fierce intent. She's trying to form the words that will explain how she discovered that her sister and two-year-old niece had been killed.

Recalling that terrible morning in June 2011, at times Jeannette shakes so hard, she can barely speak. Christine, she explains, had recently separated from David Oakes, the father of her youngest daughter Shania, after enduring an abusive relationship for six years. He reacted by abducting the little girl and threatening to kill her unless Christine went back to him. For more than two weeks, he refused to return Shania, giving her up only when a court ordered him to.

"When Chrissie finally got Nia back, at the end of April, he didn't want to just have visits and have Nia for the weekend," Jeanette says. "He wanted to get back with Chrissie. It was all, 'Let's go out for dinner as a family.' And she was like, 'I don't want to be with you. You can see Shania, that's fine, but we're not together.' And he couldn't handle that."

Christine was terrified Oakes would abduct Shania again. On 6 June, she had been due back in court to ask for a permanent residency order, which would ensure Nia would continue to live with her; Oakes was enraged. In the early hours of that Monday morning, he let himself into Christine's house with a set of keys he had kept. Over four hours he tortured Christine with a number of implements he had brought with him. Some way through the attack, Christine managed to persuade her older daughter, aged 10, to escape from a first-floor window and flee to her father's house nearby.

"In the trial," Jeannette recalls, "it was all computerised pictures of where Chrissie was lying. I knew exactly where she was, at the foot of the bed. Nia was in the doorway, screaming. I think by then he'd already shot Chrissie in both legs. Then I think Nia must have run in and he's turned, shot her, then shot Chrissie in the stomach."

Oakes died of cancer last year while serving a life sentence for their murders.

Two and a half years on, we are sitting at the kitchen table in Jeannette's neat house on the new estate she moved to soon after. Her three-year-old son's toys are tidily stacked in and around a small fire engine in the front room, below a picture of Christine in her early 20s. She's beautiful, with long dark hair and a lovely open smile. Next to it is the order of service for the funeral held for her and Shania. Jeannette rolls a cigarette in trembling fingers and takes a drag. "I've not talked about it for a while. I can't believe how hard this is."

Suppressed rage simmers and occasionally explodes in bitter recollections of how her sister was let down by the agencies that should have protected her: Jeannette is furious that her sister was never assessed as being high risk, even though police had been called out many times because of Oakes' increasingly violent behaviour. On around two dozen occasions, Oakes had been reported to police: concerned neighbours had told them that he had threatened Christine and her daughters, and had a weapon; they had been called after Shania's abduction; and he had been arrested for being in breach of a non-molestation order.

Neither of Christine's children was asked about the violence in the period before the murders. This incenses Jeannette. "If the woman is too petrified of the outcome to open up, the next thing is to ask the kids," she says forcefully. "They are there all the time. They know everything that's going on. Chrissie's daughter who escaped, she didn't hold back when they finally asked, but by then it was too late. If they're not asking the kids any questions, they're not going to get any answers, are they?"

Christine Chambers with her daughter Shania. Photograph: SWNS.com

When women and children are killed by an abusive partner or ex, two common themes emerge. One is that the systems for sharing information about women and children at risk repeatedly fail, if they exist at all. Police had recorded 10 incidents involving Christine and Oakes since 2009 – seven of them in the three months before the killings. But none of the other services knew of the dangerous escalation in violence, because of a massive backlog in inputting data. The last seven incidents were entered on the police computer system only after Christine and Shania were killed.

The second theme is that the factors increasing risk are poorly understood by the very people who are meant to assess them (usually police attending emergency call-outs). Assessing a victim's degree of risk is hard to do accurately, and in the past often relied on subjective judgments made by people who were rarely specialists in the dynamics of domestic abuse. To correct this knowledge gap, in 2009 experts introduced a checklist for frontline professionals, known as Dash (domestic abuse, stalking and honour-based violence). Its aim, explains Diana Barran, chief executive of the charity Co-ordinated Action Against Domestic Abuse (Caada), "was to introduce a degree of consistency into the way risk is identified".

The 29 questions on the form are mapped to specific risk triggers: for instance, it's known that the danger faced by victims escalates dramatically at distinct moments in a relationship. Pregnancy, a victim's decision to end a relationship, and conflict over contact with children are flash points at which, to reassert his absolute control over a woman, a perpetrator will often ramp up his abuse.

With the completed questionnaire, police are better placed to make a judgment as to whether a woman is at high risk. If she is, she will be referred for a monthly meeting where all key agencies who have had contact with that family share information and put a protection plan in place. Unfortunately, the form is only as reliable as the answers given, and many women are scared their children will be taken away if they reveal the true extent of violence in the home.

It seems extraordinary that Christine and Shania Chambers were never identified as high risk, even after Oakes kidnapped the girl for two weeks. He had bombarded Christine with hundreds of text messages and breached a non-molestation order, both of which she reported to police. Though he was arrested, no further action was taken. If she had been referred to agencies for a protection plan, things might have turned out differently.

I ask several forces if they will let me spend a day with their public protection team. Only detective superintendent Tim Smith of Kent police agrees. This is brave of him, because Kent, like many forces, has faced criticism for how it responds to domestic abuse. There have been 11 domestic homicides in Kent since 2011 and the scale of the problem remains immense: domestic abuse makes up a third of Kent's violent crime. In 2012, over 1,300 cases were assessed as high risk and referred to a monthly multi-agency risk assessment conference. Nearly 2,000 children in the county were living in those abusive households.

Smith has worked to transform the way risk is assessed. On a trading estate on the outskirts of Ashford, nine specially trained sergeants now work with a team of dedicated administrators; this team checks every single Dash form submitted by frontline officers over the past 24 hours against that couple's history on the police database.

The unit is a big, buzzing hub where police officers sit beside social workers, health professionals and child protection and probation officers. Embedded with the police team since last October is an independent domestic violence adviser who can be consulted at any time. "It's designed for information sharing and speed; hours can be critical in certain cases," Smith says.

If a victim calls 999, he explains, a police officer is dispatched to the scene and fills out the Dash form. Even if no arrestable offence has been committed, but the victim is identified as being at high risk, local officers are swiftly charged with putting in place protective measures, such as a panic alarm that rings through to the station. High-risk victims are also immediately referred to the multi-agency conference, so that everyone, from school to health visitors, is alerted.

"By no means is it foolproof," Smith admits, "but it's a huge leap from the old setup. Consistency of decision-making is vital. If you can get the risk assessment right, and refer high-risk people to the conference, you have a chance of reducing abuse."

What does John Cooper, the detective inspector in charge, think needs to be done better? "We need to ensure our frontline officers are refreshed with domestic abuse training," he says diplomatically.

How often do they get trained at the moment?

"Not often enough. They're a jack-of-all-trades. I'm not defending it, but under austerity, it's not easy to find the time and resources. A woman was killed here a couple of weeks ago. She was on our radar, but not [assessed as] high risk. When you're identified as high risk, you get a platinum service."

In another part of the country, Fiona, 37, a mother of two young children, escaped from her abusive husband last year. Assessed as high risk, Fiona was given an injunction to say her ex couldn't attack, stalk or harass her, as well as an occupation order which meant he could come to the family home to pick up the children only with her agreement. Recently, he phoned repeatedly to tell her he was on his way over. Panicked, she rang the police to ask what she should do if he suddenly turned up.

"The woman on the line said, 'We don't normally get involved in court orders, and he's their dad, so it's reasonable he should be able to come and pick them up.' I couldn't believe I was hearing it. She even checked with her superiors, then came back and said, 'If there's a breach of the peace, ring 999.' But if you have a situation on your hands, you need an answer – the right answer – there and then."

Another time, Fiona gave police a transcript of "a whole lot of texts he'd sent over just a few days, that kept on, on, on haranguing me, with no letup. The officer who came acknowledged that he didn't really know whether this was harassment, even though I had a court order specifically forbidding my ex from bombarding me in this way. There was no cross-referencing with my computer records, so they couldn't see the pattern and history of how he acted towards me."

Convincing a sceptical police officer that a spate of text messages, or a shove, may forewarn of a violent attack may be impossible for a woman in a heightened state of agitation and fright.

Does Fiona feel she is still in danger? "The risk doesn't go away," she says. "So yes, I am."

Louisa Rolfe, assistant chief constable for Avon and Somerset, is leading improvement in UK forces' response to domestic abuse. She has just been given the results of a nationwide review into police performance on domestic abuse by Her Majesty's Inspectorate of Constabulary (HMIC) that shows more than 50,000 women and children are at risk of serious harm and murder, and is scathing about a widespread inadequacy in police response. Results were dismal nationwide, and some forces were singled out for particularly harsh criticism.

Daniel Pelka, who was killed by his mother and her partner. Photograph: West Midlands Police/PA

"I think we have some work to do with culture and understanding," Rolfe says carefully, "to ensure that frontline officers understand the implications of what they do. At the moment that training is only sporadic." An officer attending the umpteenth seemingly trivial domestic abuse call-out to a couple's home may have painstakingly worked through the risk assessment form the first, second and third times, but might skip a few questions on subsequent visits, particularly if they have tried hard to help and can see no change.

To challenge prevailing attitudes in her own force area, Rolfe recently commissioned a survey of how victims had been treated by frontline officers. "Some of the first things we got back were insensitive comments such as, 'Why don't you just leave him, love?' " She smiles through a mix of horror and embarrassment that experienced police officers can still spout the simplistic line that leaving someone you love, with whom you may have children and on whom you may be financially dependent, is an easy thing to do.

"We've played that back to our staff. When we played it to commanders, they were dismayed."

Rolfe is backing the new protection orders that were recently piloted. These can be authorised by a superintendent on the spot and can, for example, force the offender to leave the family home immediately. "They give victims time: they can run from 48 hours up to a month," Rolfe says. If an order is breached, a perpetrator can be arrested without a warrant. No assault is needed and no further proof of an offence required. But none of this will be triggered if agencies do not perceive that a woman or children are at high risk.

Four-year-old Daniel Pelka was killed by his mother and her partner in 2012. Throughout his short life, shocking levels of abuse and alcoholism devastated his home. At one point, his mother was assessed as at high risk. Poor-quality social care assessments did not identify Daniel and his siblings as being at risk of, or suffering, serious harm. Healthcare services knew he was emaciated, and facial injuries and strangulation marks had been observed on his neck by teachers. But still the risk to him was not considered.

Similarly, though Christine Chambers had told police that Oakes had made threats to kill her and Shania, and they knew she had recently left him, each incident was treated in isolation; the danger to the girls was never considered independently of the risk to their mother. According to research just released by Caada, nearly half a sample of 877 children receiving support had been exposed to high-risk abuse that put them at imminent risk of serious injury or death; 62% had not only seen or heard abuse, but had been directly harmed as well.

"There is an extremely fine balance to strike," Diana Barran observes, "because if we ask too much about risk to children, women will shut down. They fear that if you tell a police officer about assaults and bruises on children, social services will be coming round. But we need to understand the risks to both the mother and her children in order to protect them both."

One woman told a focus group for the HMIC report: "Immediately after the police have been around, the beatings are much worse – double as bad. It puts you and the children at greater risk."

"Abuse affects everything in a child's life," says Sharon Bryan, head of the family recovery programme at Westminster council, and herself an abuse survivor. "If a child up to a year old is exposed to domestic violence, it's known to affect them later in life. There can be bed-wetting, aggressive behaviour, problems at school or being violent themselves. The worst is post-traumatic stress disorder. Lots of children suffer from that. They get flashbacks, eating and sleep disorders."

Children rarely see their mother being beaten, but this doesn't protect them. "They'll often be in the next room and that's worse, because they don't know what's happening. When it goes quiet, is Mum OK or is Mum dead? They'll play truant from school because they're afraid of leaving her."

Professionals need better training in the complexities of how women and children survive years of abuse, Bryan says. "If a social worker says a child seems really resilient, unaffected by what's going on, to me that's a measure of how bad it is – it's absolutely normal and all they've ever known."

The man may hit the child to make the mother do something, and mothers are not always able to protect their children; sometimes they even participate in the abuse themselves. "Women get to a stage where they lose their identity," Bryan says. "To be able to survive that sort of abuse, you have to shut down. You're trying to save your own life, concentrating solely on staying alive."

In Blackpool, a charity called Empowerment employs a team of independent advisers, dedicated to supporting children living in violent households. Four make time to meet me on a blustery morning before heading off to help their young clients draw up personal "safety plans". The youngest child on their books is aged just three.

These women are in no doubt that children's accounts of the abuse they hear, see or experience directly is different from the version given by their carer – usually the victim, their mother – and they need to be listened to. "The struggle we often see is their feelings of loyalty towards the person doing it. And their 'safe carer' may be minimising what's happened," Sarah Midgley says. "But children see a lot: the honeymoon when things are made better, the part where things deteriorate again, and being in the room when the violence is happening."

"There are children who think they can help," says her colleague, Katie Kershaw, "because they think it won't be done if they're there."

Blackpool social services was recently criticised by its regulator, Ofsted, for closing domestic abuse cases involving children too early. More than half the children living in an abusive household – and a staggering two-thirds of those whose parent had been assessed as high risk – had not been known to local child protection social workers. There is frustration in the Empowerment team that, as Kerrie Fawcett puts it: "You may know a child is at risk and the situation escalating, but they don't yet meet the threshold for statutory intervention. It sounds awful, but you're almost waiting for an incident to happen before social services kick in."

Getting risk assessment right is not only about preventing women and children being killed. An acknowledgment that children face significant risk means that organisations such as Empowerment can make a compelling case for funding at a time when domestic abuse services have been decimated. The work they do not only saves and improves lives now, but helps to buffer the psychological and neurological harm that can lead to a child becoming an abuser, or having so little experience of healthy relationships that as an adult they accept abusive treatment themselves.

Do these women worry that the young people they're working with now are at serious risk? "Yes," they all say. Have they known cases where risk has been poorly assessed and a child has been hurt? They look at each other. "Yes," Kershaw says. "But that's just normal to us."

At the charity Caada, Diana Barran believes the support of independent advisers such as the Empowerment team is vital: "We repeatedly hear that women will tell them more about what is going on at home than they'll say to the police." The Dash form, she points out, was used by police who were called to the numerous incidents at Daniel Pelka's home. "It has a question that asks, 'Do you think there are risks to children in the household, and if so, what have you done to make them safe?' But if somebody doesn't disclose, you will not get a full picture of the risk."

She is pleased the Dash form has been adopted nationally, but knows it is only a first step. "I was at a high-risk case conference recently where we were told a woman had been held underwater in the bath and that had not initially been picked up as a major incident," she says. Nevertheless, she believes professionals do now have a common understanding of how to evaluate risk and so stand a better chance of getting their response right.

Barran regularly speaks to chief constables who tell her they're dealing with up to 100 risk assessments a day, the vast majority of which involve children. "Realistically, children's services cannot deal with that," she says bluntly. "What's happened is that we've identified so many more cases of high risk: the good news is we now know there are all these children in danger; the bad news is that we have no resources to deal with it."

Christine Chambers' sister Jeannette is adamant, too, that children need more help than they get. "Not knowing anything different from when they were kids: that's why women put up with it. That's why boys think it's OK to hit women, too," she says. "But people need to understand that this isn't normal. None of us should have to put up with any of that. Not children, not women, not men. They do it because they like to be in control, don't they? Because they're the man, they've got the balls, they're in charge. But" – she laughs, a tough, defiant laugh from a woman who has been through too much and is still fighting – "do you know what? It's 2014 now and it's not the way it works."

Some names and details have been changed.