David Cameron has unveiled plans to end the era of "old fashioned" delivery of public services by allowing almost all of them to be opened up to competition from the private and voluntary sector.

In a speech in east London, Cameron said that the provisions in the Open Public Services white paper would mark the first step on the road to a "better, fairer country" in which people enjoy more choice, less bureaucracy, improved services and equal access for rich and poor.

The prime minister promised to loosen "the grip of state control" by bringing in a "big society" approach and putting "power in people's hands". The contents of the white paper would be felt in "every state school, hospital and prison, by every doctor, teacher, parent, patient and citizen", he said.

Cameron made clear he was "absolutely determined" to see through changes which he said were not just about improving schools and hospitals but represented a "vital part of building a bigger, stronger society that is so central to my vision for our country".

"I know there are those who thought we might be pulling back or losing heart for the task ahead," he said. "So let me assure you of this: we are as committed to modernising our public services as we have ever been. I'm not going to make the mistakes of my predecessors … blocking reform, wasting opportunities and wasting time. This is a job that urgently needs to be done, and we are determined to see it through."

Unions responded furiously to the proposals, accusing the coalition government of seeking to break up the public services.

Under the changes, every public service except for national security, frontline policing and the judiciary will be opened up to providers from the private and voluntary sector.

A "general right to choose" in areas such as education, health, social care and housing would be enshrined in law, with new powers for the ombudsman to act as an "enforcer of choice in public services".

The consumer organisation Which? will also extend its role as an advocate for consumers in the private sector to cover the public sector too.

"This white paper says loud and clear that it shouldn't matter if providers are from the state, private or voluntary sector - as long as they offer a great service," the prime minister said. "The old narrow, closed, state monopoly is dead."

Cameron said that while public services were centralised "with all the right intentions", the impact had been "incredibly damaging" to users.

Under his proposals, communities will be allowed to set up neighbourhood councils to commission services on a hyper-local level, individuals will get more personal budgets to buy their own services and the use of payment by results will be expanded to encourage markets to develop across the public sector.

Cameron made the case for the need to shake up provision by drawing on his own past experiences of trying to get the right wheelchair for his late disabled son, Ivan. He was still hearing too many stories from others that the right wheelchair only arrives once the child has almost outgrown it, he said.

As another example, the prime minister seized on children who qualify for free school meals who are "half as likely" to get five good GCSEs as their better off peers.

"The last time they counted, just 40 people who had had free school meals were going to Oxbridge – out of 80,000," said Cameron. "We've got a welfare state that doesn't deliver welfare, that doesn't get people back into work but traps them in poverty instead.

"So let me tell you what our change looks like. It's about ending the old big government, top-down way of running public services … releasing the grip of state control and putting power in people's hands. The old dogma that said Whitehall knows best – it's gone. There will be more freedom, more choice and more local control. Ours is a vision of open public services."

The white paper was published on the same day that private care home provider Southern Cross announced it was closing down.

But Cameron's official spokesman brushed aside any suggestion that Southern Cross's failure cast doubt on the wisdom of allowing private sector firms to deliver public services.

The spokesman told reporters: "Obviously, the Southern Cross deal pre-dated this government and it was possible for private sector providers to be involved in the provision of some public services in the past. Yes, we are talking about increasing diversity of provision, but we also need to make sure that there are appropriate checks and balances in place and that people are looked after."

Cameron first laid out his plans to roll back the boundaries of the state to allow private providers to deliver more public services in February, but it is widely understood that the plans have since been downgraded as result of an internal battle with the Liberal Democrats.

The Lib Dems have sought to ensure that any outsourcing and market-driven reforms maintain a strong degree of accountability, prompting a Downing Street source to describe the resulting document as "more greenish than white".

Dave Prentis, the leader of Unison, said the white paper was "just another stepping stone towards Cameron's same old goal to privatise public services".

"If this is Cameron's big idea, he needs to go back to the drawing board.

"The collapse of Southern Cross today gives a clear warning as to what happens when you let the market run public services. The sale of elderly care homes started out small, but it wasn't long before private equity companies got their teeth into the market. Now that the profits are less, they want out, and thousands of elderly residents and their families are bewildered and angry over their uncertain future.

"Not only does Cameron want to wash his hands of providing public services, he wants cut price privatisation. He is removing protections on workers' pay and conditions, creating a race to the bottom, so that services are more attractive to private buyers."

Confidential documents obtained by the Guardian under the Freedom of Information Act reveal that ministers have been privately advised to allow schools and hospitals to fail if the government is to succeed in its overhaul of public services.

They reveal research by civil servants warning that markets are susceptible to failure, and that costs could in fact rise unless a true market is created by allowing public services to collapse if they are unsuccessful.

It opens up the potential for schools, hospitals, social care systems and nurseries to fold without the government stepping in to prop them up – a revelation described as "appalling" by Labour.

The documents obtained by the Guardian were prepared by civil servants as part of an internal government review into the consequences for democratic accountability of the coalition's localism, big society and outsourcing reforms that are integral to the white paper.