There is a burnt-out metal box at the National Transportation Safety Board's offices in Washington that once housed what may well prove to be the most expensive battery in history.

The charred metal box housed a lithium-ion battery that once powered the auxiliary power unit on a Japan Airlines Boeing 787 Dreamliner. That plane, one of 50 in service of the 850 sold so far, caught fire at Logan International Airport in Boston earlier this month. The same kind of battery is thought to have led to the grounding of a Nippon Airways flight this week. That plane was forced to make an emergency landing after a burning smell was detected in the plane's cabin.

Boeing's battery woes are the latest in a series of problems to have beset the Dreamliner. Such problems have led to a global grounding of the aircraft, including all US-registered 787s, and a wide-ranging Federal Aviation Administration (FAA) investigation, the first time in four decades that it has pursued such drastic action.

Boeing's chief executive, Jim McNerney, expressed "deep regret" for the debacle and said the company was "working round the clock" to restore faith in the aircraft. The company will need all its considerable political clout in Washington to speed through a resolution from regulators who are already facing allegations that they fast-tracked the troubled aircraft in the first place.

But the Dreamliner's problems are not just a Boeing issue. They are a lesson in the limits of outsourcing and the all too cosy relationships between regulator and regulated that have caused problems across industries from automotive to food and financial services in recent years.

Boeing started work on what would become the Dreamliner in the late 1990s. The first planes were delivered to Nippon Airways in 2011, years late and billions over budget. Boeing's local newspaper, The Seattle Times, puts the eventual cost of the plane's development at $32bn.

The 787 was pitched as the airline of the future – a revolutionary plane that that would use new technology to bring aircraft design into the 21st century. The Dreamliner is made of carbon-fiber reinforced plastic composite. More radically still, pneumatic and hydraulic systems have been ditched for electric systems.

The technological leap was always likely to cause teething issues. But these were exacerbated by Boeing's decision to massively increase the percentage of parts it sourced from outside contractors. The wing tips were made in Korea, the cabin lighting in Germany, cargo doors in Sweden, escape slides in New Jersey, landing gear in France.

The plan backfired. Outsourcing parts led to three years of delays. Parts didn't fit together properly. Shims used to bridge small parts weren't attached correctly. Many aircraft had to have their tails extensively reworked. The company ended up buying some suppliers, to take their business back in house. All new projects, especially ones as ambitious as the Dreamliner, face teething issues but the 787's woes continued to mount. Unions blame the company's reliance on outsourcing.

Bill Dugovich, communications director at SPEEA, the professional aerospace union, said his members had first voiced their concerns in 2002. "Outsourcing in general lengthens supply lines, creates problems with language and culture and is extremely hard to coordinate. You have seen a plethora of problems at Boeing. Things get outsourced then they have to come back to Boeing to get fixed," he said.

Dr Amar Gupta, dean of Pace University in New York, has studied the construction of the Dreamliner and is not convinced that outsourcing itself is the issue. "We have been outsourcing since the industrial revolution," he said. The problem is one of communications, he argues, and complexity. A car has roughly 15,000-20,000 parts; a plane has more than 2,000,000 parts.

"The concern is that each organisation did what it was asked but there was a failure to bring the whole thing together, to integrate the systems," he said. Gupta thinks that with better communication and organisation – what he calls "24 hour knowledge factories" – outsourcing could pull off feats as complex as the Dreamliner.

'A powerful force in Washington'

Arguably, it is not just Boeing's fault that the Dreamliner wasn't ready. Boeing is a powerful force in Washington. Barack Obama toured a plant working on the Dreamliner last year and chose Boeing boss McNerney to chair the president's export council in March 2010.

Consultant and former airline executive Robert Mann said Boeing's clout put pressure on the Federal Aviation Authority (FAA) to speedily approve the Dreamliner, despite its radical design and manufacturing process. The Dreamliner is a huge black eye for Boeing, said Mann. Ultimately he believes the company and the plane will pull through but the industry needs to take a good look at what went wrong.

Even after all those delays and teething issues the Dreamliner was passed under a very compressed schedule, said Mann. "And there was an electrical failure and an emergency landing during the test-flight programme," he said. "That was blamed on a 'foreign object'."

Mann said the FAA's mandate changed under administrator Marion Blakey, appointed by president George W Bush in 2008 as Boeing was working on the Dreamliner. "Blakey saw the FAA as a 'customer services organisation,'" said Mann. The FAA was working with the airlines to cut regulation, not to impose it, he said.

A burnt auxiliary power unit battery removed from a Japan Airlines Boeing 787 Dreamliner jet. Photograph: Handout/Reuters

This relaxed attitude to regulation brings us back to that charred battery. Lithium-ion batteries are the preferred power source for a range of modern technology but they have a spotty safety record. Laptops, electric cars, cell phones – all have caught fire thanks to their lithium-ion batteries. Federal air regulation specifically limits the size and number that can be carried by passengers. Boeing was able to obtain a waiver for the size, quantity and manner of use of its batteries in September 2007, after the FAA received assurances and extensive test data, much of which was provided by Boeing.

Mann is worried by the sheer number of innovations that the FAA seems to have nodded through. "There's leading edge and there is bleeding edge," he said. "There were so many innovations on this plane that it is hard to fathom how it got approved so quickly. Thankfully, no one was hurt."