The Office of Fair Trading has referred the payday loans market to the Competition Commission, saying there are deep-rooted problems with the way competition works and that lenders are too focused on offering quick loans.

The regulator said variable levels of compliance with credit laws and guidance meant firms that invest time and effort to comply were at a competitive disadvantage.

The referral follows a year-long review of the sector which exposed widespread evidence of irresponsible lending and breaches of the law, which the OFT said were causing "misery and hardship for many borrowers".

Although lenders say their high-cost loans are designed to be taken out over short periods and that annual interest rates of often more than 4,000% are not a fair indication of the cost, the regulator found companies were making up to 50% of their money from customers who extended or rolled over loans or incurred late payment charges.

Announcing the referral, the OFT highlighted several features of the £2bn market, which it said might be preventing, restricting or distorting competition:

• borrowers using payday loans have "poor credit histories, limited access to other forms of credit and/or a pressing need to borrow", which could be weakening competition on price

• lenders are using practices which make it difficult for consumers to identify and compare costs

• there are barriers to switching when loans are rolled over.

It also expressed concern about the focus on quick loans, which it said compromised affordability checks and meant competition was on the basis of speed rather than price. Too many people were being granted loans they could not afford to repay, and there was no incentive for lenders to cut interest rates.

The OFT's chief executive, Clive Maxwell, said: "Competition appears not to be working properly in the payday lending market, allowing firms to profit from making loans that cannot be paid back on time. We have seen evidence of financial loss and personal distress to many people.

"The Competition Commission can now conduct a detailed investigation to get to the root causes and, if necessary, use its far reaching powers to fix the payday lending market."

In March, lenders were given 12 weeks to reform and notify the OFT of the changes they had made. The final deadline is the end of July, and the regulator said it was still waiting for 30 of the 50 lenders to respond.

In the runup to Thursday's announcement, three lenders had opted to leave the market, but that figure has since increased to five. Two have left the credit market; the other three have left the sector.

The OFT said payday lending would remain a top enforcement priority until it handed regulation of the sector to the Financial Conduct Authority (FCA) in April 2014.

Payday lending will remain a 'top enforcement priority', the financial watchdog has said. Photograph: Murdo Macleod for the Guardian

It said by referring the market now, the commission would be able to provide the FCA with a sound evidential basis on which to develop its rules and apply its new powers after it took over responsibility.

The Consumer Finance Association (CFA), the trade body for some of the largest players in the industry, including The Money Shop, Cash Converters and QuickQuid, said it would have liked the referral to have been deferred.

Its chief executive, Russell Hamblin-Boone, said: "The CFA and its members have always supported well-designed, well-implemented regulation in order to protect consumers and drive up standards. However, no other sector has faced such intense scrutiny in such a short space of time.

"We would have preferred the inquiry to have been deferred to allow the significant improvements that lenders have made to take effect before the industry faced further judgement. We urge the Competition Commission to take this into consideration during its inquiry."

Debt charities and consumer groups welcomed the announcement but urged the OFT to continue to scrutinise the market.

The executive director of Which?, Richard Lloyd, said: "People under financial pressure being given high cost loans in minutes without proper affordability checks is a recipe for disaster. This referral doesn't mean the OFT can now stand down, it needs to stay tough with lenders and continue to take early enforcement action against any company found to be lending irresponsibly."

On Monday the government will hold a summit on the industry between lenders, consumer groups and industry watchdogs to explore whether further regulator of the sector is necessary. A growing number of organisations have taken a stand against the industry, blocking adverts and even access to lenders' websites.

The Competition Commission has previously stepped in to shake up the home credit market. In 2006 it told doorstep lenders, who like payday lenders offer small, short-term loans, that they had to make charges more transparent and share data to make it easier for consumers to shop around and drive down prices.