Hundreds of millions of dollars worth of suspect gold has poured into the global markets as major breaches occurred in new international rules designed to tackle an underground trade linked to African warlords and human rights abuses.

Confidential papers shown to the Guardian by a whistleblower detail how, in 2012, one of the world's biggest gold refineries ignored guidelines designed to stop the trade in so-called conflict gold.

It paid more than $5bn (£3bn) in cash for the metal and accepted gold from more than 1,000 customers who walked in off the street with no paperwork.

While there is no evidence that the refinery accepted conflict gold, major breaches in new guidelines were uncovered, raising concern about the history of huge volumes of shipments. Details of the damning findings have not been spelled out in full in public documents, but have been uncovered in a Guardian investigation. The leaked papers are also being reported by BBC2's Newsnight, al-Jazeera and the campaign group Global Witness.

Allegations that the full scandal has not been laid bare are a huge setback to international efforts – championed by Barack Obama, the UN, the EU and campaign groups – to stamp out the illicit trade in conflict gold by requiring the world's largest refineries to be independently audited to check that they are sourcing gold responsibly and publishing the findings.

Full details of the findings are set out in confidential inspection reports from Ernst & Young, which was brought in to review the practices of the Kaloti Group, a $12bn refining and trading business based in Dubai. Kaloti owns the largest refinery in the Middle East and is at the centre of Dubai's booming gold industry, estimated to be worth $70bn in 2012. Documents show the group systematically flouted new rules on conflict gold.

The UN has been pushing for tough international guidelines covering the trade in gold and other "conflict minerals" because of mounting evidence linking extortion at hundreds of mines in the Democratic Republic of Congo to armed groups involved in war crimes.

The UN's largest peacekeeping force is deployed in the country, with more than 20,000 uniformed personnel. More than 5 million people are estimated to have died in the DRC or its neighbours in the last two decades, making it the most deadly conflict since the second world war.

All refiners around the world must be able to show they meet the new international standards, and have been independently inspected, if they want to continue trading with US multinationals from May this year. Guidelines recommend companies "make and receive payments for gold through official banking channels where they are reasonably available, avoid cash purchases where possible, and ensure that all unavoidable cash purchases are supported by verifiable documentation".

The rules of the Dubai Multi-Commodities Centre, set up by Dubai's ruler Sheikh Mohammed in 2002 to promote and regulate the gold industry, require refiners to make extra checks on suspicious, cash-in-hand deals above 40,000 dirhams (£6,540). Kaloti had no system in place to do this in 2012.

In fact Kaloti paid out $5.2bn in cash-for-gold deals – equivalent to almost 45% of its business in 2012. E&Y Dubai's inspectors also found the refinery group had:

• Taken millions of dollars of gold knowing it was plated in another metal and seemed to have been smuggled out of Morocco.

• Accepted 2.4 tonnes of gold in more than 1,000 transactions with customers who provided no paperwork.

• Paid cash to Sudanese suppliers who had hand-carried gold to Dubai – sourced from small-scale, artisan mining operations – but did not check whether they had approved mining licences.

These details from E&Y inspections have never been disclosed before. In Kaloti's much-delayed compliance report, published last December, the failings are mentioned only briefly and in terms that do not leave the reader with a clear picture of the inspectors' findings. Kaloti maintains all failures in its procedures were rectified by November last year, with all but one of its remedial measures having been independently verified by E&Y. In a statement to the Guardian, it said: "Kaloti had shortcomings in the initial stages of the long multi-staged audit process … Our fully compliant final result was confirmed by Ernst & Young."

Both Kaloti and E&Y stressed all shortcomings were properly reported and acknowledged in published reports. Other comparable reports from major refineries in Dubai and elsewhere also offer only very scant descriptions where inspectors have discovered failings, suggesting this may be emerging as common industry practice for these new reports.

The Guardian has seen no evidence that published reporting of Kaloti failings was out of line with regulatory rules or industry practice. But leaked emails show their proper presentation in published filings was hotly contested.

Both Dubai rules and international standards set by the OECD say that the purpose of publishing reports on the outcome of independent inspections is to demonstrate transparency in relation to refineries' performance on responsible sourcing measures in order to "generate public confidence". But before the inspectors' work was finished and their reports filed, the Dubai authorities learned of E&Y's findings and changed their own rule book in a way that meant that the overall verdict of inspectors would stay confidential.

The Dubia Multi-Commodities Centre insists these changes were made simply to keep rules in line with similar international standards. "DMCC strongly refutes any allegation that it sought to influence or interfere with the review process, or that it altered or softened the review process to favour any member refinery," it said, adding: "All findings have been made public and the review and reporting processes are robust."

Emails show how some at E&Y felt pressure had been put on them by a regulator "keen to promote the Dubai gold industry", pressure that led Amjad Rihan, the partner in charge of the inspection division, to break ranks and come forward as a whistleblower. Rihan said he was horrified that the Dubai authorities had altered the rules and that E&Y had signed off as "fair" public reports obscuring full details of Kaloti's failings.

So brief were eventual descriptions of the failings reports published last December that in press releases the DMCC and Kaloti made no mention of them, celebrating the inspection outcome and stressing Dubai's leading refinery had ended the audit fully compliant, certified as offering conflict-free, responsibly sourced gold.

Rihan said: "I am not a lawyer, but in my opinion, what is 'ethical' and what is 'legal' should not conflict. The DMCC, Ernst & Young and Kaloti were all aware that the risk of conflict gold entering Dubai had been very high. In my opinion, the way they acted is appalling, amoral and extremely unethical."

At one stage, so serious had E&Y's concerns about how to treat the Kaloti findings become that the case was referred to some of the firm's top global executives and legal experts in London. At one stage, Mark Otty, E&Y's managing partner for Europe, Middle East, India and Africa (EMEIA) based in London, said in an email: "Rest assured that we are taking this issue very seriously … I have taken the lead in relation to our investigation of it." Otty said later that he was satisfied with the outcome because "both client and regulator will report deficiencies".

A spokesman for both E&Y divisions declined to answer questions from the Guardian. "This matter … concerns work done by EY Dubai, in Dubai, for a Dubai-based client under guidelines promulgated by the DMCC, a Dubai regulatory body."

Joe Murphy, managing partner at E&Y Dubai, said: "EY Dubai took the views of our former partner [Rihan] very seriously and prior to issuing our … report undertook a comprehensive review, including consulting with internal and external experts, who supported the actions we took. We firmly believe that by identifying elements of non-compliance we have played an important role in achieving improvements in the client's supply chain controls."

E&Y's global code of conduct says: We are robust and courageous in our challenge to clients and are not afraid to deliver unwelcome information to them ... We support people and will withdraw from working for any clients that put our people under undue pressure". Rihan does not feel that is what happened in Dubai "I went to senior figures at EY asking them to back me up in my professional opinion but instead I think they went to great lengths to please the client and they pushed me out."

EY deny this. "EY Dubai took the views of our former partner very seriously and prior to issuing our … report undertook a comprehensive review; including consulting with internal and external experts, who supported the actions we took".

New international guidelines have emerged to tackle conflict gold, after similar to curb the trade in blood diamonds.

According to a UN report at the end of 2011: "A significant proportion of gold financing armed groups and criminal networks [within the DRC military] continues to end up in the United Arab Emirates, making the due diligence of refiners, smelters and jewellers based in that country of particular importance."central Africa, unless from long-standing industrial suppliers or well-known global traders. Kaloti, the UN report said, insisted it "had not purchased artisanal gold from the region for three years."

Dubai is one of the world's global hubs and attracts business from all quarters. Last October the Responsible Jewellery Council, whose members include Cartier, Tiffany & Co, H Samuel, Ernest Jones and Argos, signed a deal recognising Dubai's conflict gold refinery inspection programme and permitting RJC jewellers to market gold bought from the emirate as conflict free.

In recent weeks both Apple and Intel have also announced that all gold used in the circuitry of their phones, computers and other devices is now conflict free.