Inside a chaotic government welfare office in Victoria, crying children were tired of waiting in line, desperate adults couldn’t get their cheques, and overwhelmed ministry staff faced relentless criticism over a provincewide computer failure.

When the province’s new social-welfare computer system crashed in May 2014, it left B.C.’s most vulnerable citizens struggling to get money for food and rent.

“It was just chaos,” said Katie McCready, a single mother of two who was among more than 60 people crammed into the office in Victoria that day during the peak of the failure.

“(Kids were) screaming, crying. Other parents were freaking out. The front line people, everybody was yelling at them ... ‘What the hell am I supposed to do?’”

McCready, who had used the last of her money for bus fare to the office, broke into tears when told staff couldn’t issue her cheque. Beside her, a female ministry staffer with red puffy eyes issued apologies.

“They couldn’t help and didn’t seem to know why,” said McCready. “It was just this new computer system and they weren’t getting any information at all from who was running the show.”

Today, the Integrated Case Management system, designed to link the children’s ministry and the social development ministry into one database, has been fully online since November 2014. Despite overshooting its original $108 million budget by $74 million, it is performing only a third of the tasks the government had initially hoped it would do.

It is one of eight high-profile IT projects recently launched by the province — with a total price tag of $2.5 billion — that The Sun has investigated for this series of stories. Several have taken much longer than originally planned; many have defects or are underperforming; and five of them have shot over budget, so far by a combined $350 million, including $72 million spent on an electronic health records system that had to be reset after the contractor was fired.

To put that into context, $350 million would pay for seismic upgrades to more than half of Vancouver’s aging schools.

The findings, support the call for a new approach to tackling these large information technology projects, which will become more common as technology continues to advance.

The two ministries using Integrated Case Management say the system has been available 99 per cent of the time over the past year, and don’t anticipate another system-wide crash like the devastating one in 2014. While most social workers acknowledge it is an improvement over the child protection software it replaced, they insist problems continue and its complicated design occasionally causes information to get lost.

“This system is slow, it’s clunky, it goes down, it isn’t stable yet. It is hard to find information,” said Doug Kinna, the social worker spokesman for the B.C. Government Employees Union.

B.C.’s information technology woes are not unique, said Ellen Balka, a Simon Fraser University professor.

She cited two academic studies that reveal universal failure rates of 65 per cent — or two out of every three projects — for complex systems.

“Large scale computing infrastructures frequently fall far short of articulated goals and expectations, and a new approach to understanding IT projects is warranted,” said Balka, an expert in information technology systems.

“It is astonishing to me that as a culture we think (this failure rate) is OK. We really haven’t, as a culture, said, ‘We have to do this better. We can’t accept this.’” Over the last decade, success rates for large-scale IT programs have been only 35 per cent, according to The Standish Group, which researches software projects around the world. A mere two per cent are outright successes.

The more expensive the plan, the higher the chance of failure

It is not uncommon, Balka said, for managers in charge of these projects to focus on the hardware and software, without spending enough time considering the complex work the technology is supposed to support or the needs of the staff who do that work.

“We live in a culture that believes in the infallibility of technology,” she said.

B.C. spends approximately $500 million a year on IT infrastructure and systems, according to the auditor general. The auditor’s office has issued a series of scathing reports about some IT projects, and has recently started a governmentwide look at technology problems.

NDP critic Adrian Dix, who has closely monitored many of these projects, argues government needs expertise to work with the companies that build the systems, and to understand what they do.

“In general, the kind of questions that people ask about bridges and roads and hospitals and schools and about things that we might understand more, we don’t ask about computer systems. It’s as if decision makers get completely distracted by bells and whistles,” Dix said.

Bette-Jo Hughes, the government’s chief information officer, said it is frustrating that the public only hears about the IT projects that go poorly.

She said the government has 51 small IT projects on the go right now — collectively worth $200 million — and all are running effectively, on scope and on budget. She pointed to a $150,000 emergency lab database that helped coordinate the 2014 avian flu outbreak, and a $335,000 wildfire tracking system used to help fight forest fires.

“There’s a huge number of projects that occur on an annual basis that do go very well. … There is a lot of good stuff going on, unfortunately nobody ever hears about those,” said Hughes, whose office provides oversight to government on information technology.

“(And) when we do have issues on some of the larger projects, we absolutely do learn from that.”

With the case of the maligned Integrated Case Management system, financial pressures and critical problems processing child welfare cases led the government to quietly abandon two-thirds of what the project was initially promised to do, an auditor general’s report concluded.

The software is used for income assistance, disability payments, and child welfare, but social workers say they still have to rely on an old computer system to do part of their jobs.

A recent BCGEU report found B.C. was the only Canadian province to choose this type of software, while five other provinces rejected it as either too costly or not suitable for the job.

The government of New South Wales in Australia purchased Integrated Case Management, but eventually abandoned plans for costly upgrades after backlash from staff and the public.

The B.C. government announced the Integrated Case Management project nearly a decade ago, and said then it would cost $107.3 million. Victoria “refined” its plans in 2010 and increased the budget to $182 million for the capital costs of designing and building the system.

The ministries did not include implementation costs in this final price tag, which they say follows accounting rules.

Kinna, of the Government Employees’ Union, argues the staggering costs to get the system running should be counted, such as bringing multiple ministry workers to Victoria over a two-year period to work on it. That pushed the budget much higher, perhaps in the $220-million range, he argued.

The two ministries that use the software — children and family development, and social development and social innovation — argue the system now works well, and that continual upgrades are made to reflect government policy changes and feedback from staff.

The goal will surely be to avoid another system-wide shutdown like the one in May 2014, which greatly delayed nearly all welfare services in the province.

For McCready, the interruption meant an almost two-week delay getting her social assistance cheque.

“At the time, we were really just living on nothing,” said the mother of two boys, a teenager and a younger one with autism, who had then lost her job, was separated from her husband and was searching for housing while living with her parents.

“[The cheque] is the only thing you have to get groceries. You are down to rice and soup for a week.”

Integrated Case Management was not the only problematic IT project to dominate headlines in B.C.

Until recently, fare gates had collected cobwebs for years in Metro Vancouver transit stations, a constant reminder of the challenges with TransLink’s Compass Card.

The system is supposed to allow riders of buses, SkyTrains, WestCoast Express and SeaBuses to prepay fares on an electronic card, and was intended to reduce fare evasion.

Compass cards work on trains, but because of problems with tap-out functions, TransLink has given up on buses.

Instead the transit provider was forced to lower bus fares to one zone across the region.

The system is also nearly $25 million over budget and more than two years late.

Crown corporations BC Hydro and the Insurance Corp. of B.C. have both budgeted $400 million to upgrade aging software, and to overhaul their claims, client, financial and human resources computer systems.

ICBC asserts success in getting two-thirds of its project online, with $340 million spent so far and the final component (an insurance policy system for brokers) online next year.

But the independent regulator, the B.C. Utilities Commission, has ordered the corporation to better explain publicly why it keeps jettisoning certain costs from the project budget and into its “corporate expenses.”

In Hydro’s case, the NDP have alleged Hydro lied to the utilities commission about its IT software choice, and has spent $496 million while only half way through.

The corporation fired a top IT official and Energy Minister Bill Bennett admitted mistakes have been made.

“The IT program within BC Hydro did have problems, there’s no question about that,” he said in the legislature in May.

“I have faith in the CEO to resolve issues around IT and make sure ratepayers are being treated respectfully.”

In a statement, Hydro said its IT work is constantly adjusted based on priorities, and insisted it has spent within its budget for the past five years.

Even the large projects the government contends are on time and on budget have other problems.

The B.C. Lottery Corporation’s $124-million Gaming Management System, launched in February, is designed to link financial data and to improve operations in all 37 B.C. gambling facilities.

A 2014 Finance Ministry review, however, found anticipated benefits from the new system is about 32 per cent lower than initially forecast, from $515 million (over the first 10 years) to $352 million, “indicating a weakness in project planning that overstated the revenue projections.”

In response, the lottery corporation said it is not uncommon for financial predictions to change over the years of a large project.

The most ire from IT critics has been directed at two massive health software projects, perhaps with justification since their combined budgets are more than $1 billion.

One took an extra four years before it worked and nearly tripled its budget, while the other is just being built but is already behind schedule.

lculbert@vancouversun.com, rshaw@vancouversun.com

Click here to report a typo or visit vancouversun.com/typo.

Is there more to this story? We'd like to hear from you about this or any other stories you think we should know about. CLICK HERE or go to vancouversun.com/moretothestory