The Nasdaq market has agreed to pay a $10 million penalty for alleged securities laws violations resulting from its "poor systems and decision-making" during Facebook's initial public offering, the Securities and Exchange Commission said Wednesday.

The SEC called the penalty the largest ever against an exchange. It said in a statement that it charged Nasdaq with securities law violations for how it handed Facebook's public stock debut a year ago, which had been touted as the largest IPO ever, but suffered a slew of problems, including order disruptions, that marred the event.

Nasdaq had already agreed to pay $2 million to investors for their losses.

"According to the SEC’s order instituting settled administrative proceedings, despite widespread anticipation that the Facebook IPO would be among the largest in history with huge numbers of investors participating, a design limitation in NASDAQ’s system to match IPO buy and sell orders caused disruptions to the Facebook IPO. NASDAQ then made a series of ill-fated decisions that led to the rules violations," the SEC's statement said.

Nasdaq's CEO Robert Greifield said in a statement that although the exchange prepared extensively for the IPO, "the challenges we encountered that day were unprecedented." He said the exchange has reviewed what happened and is taking steps to improve its trading technology.

Let our news meet your inbox. The news and stories that matters, delivered weekday mornings. This site is protected by recaptcha

Facebook's IPO on May 18, 2012 was marred by technical glitches that left the market makers — who facilitate trades for brokers and are crucial to the smooth operation of stock trading — in the dark for hours as to which trades had gone through.

Nasdaq said it tested its systems before the IPO but the testing did not reveal the "design flaw" that caused the glitches.

(Read More:Time to 'Defriend' Facebook?)

The SEC said several members of Nasdaq's senior leadership team convened a "Code Blue" conference call, at the SEC's request, and opted to not delay the start of secondary market trading in shares of Facebook, thinking they had fixed the problem by removing a few lines of code.

But they didn't understand the root cause of the problem, the SEC said. The decision to resume trading without fully understanding the problem resulted in violations of several rules, according to the SEC, including Nasdaq's own rule governing the price/time priority for executing trade orders.

Trading was delayed until 11:30 a.m. the day of the IPO. Confirmation of the initial trades didn't post until 1:50 p.m. That left more than 30,000 Facebook orders to remain stuck in the Nasdaq's system for more than two hours when they should have been promptly executed or canceled, the SEC said.

(Read More: The 10 Biggest Internet IPOs)

The SEC also said the Nasdaq broke its own rules when it assumed a short position, betting that a stock will fall, of more than three million Facebook shares in an "unauthorized error account." Nasdaq then covered that short position for about $10.8 million, which also violated Nasdaq's own rules. It also cited three other violations of Nasdaq's own rules during the opening of trading.

The SEC also charged Nasdaq's affiliated third party broker-dealer, Nasdaq Execution Services, with failing to maintain sufficient net capital reserves on the day of the Facebook IPO as a result of that big short position in the unauthorized account.

"This action against Nasdaq tells the tale of how poorly designed systems and hasty decision-making not only disrupted one of the largest IPOs in history, but produced serious and pervasive violations of fundamental rules governing our markets," George S. Canellos, co-director of the SEC's Division of Enforcement, said in a statement.