NEW YORK (Fortune) -- Guaranty Bank is hardly a household name. But the Austin, Texas-based thrift's looming failure is shaping up as a big headache for bank supervisors -- not to mention a black eye for Carl Icahn and others in the smart money set.

Guaranty (GFG) could be soon seized by the government in what would be the biggest bank failure in a year that has already had 64 of them. Last week, the bank warned investors to expect a federal takeover after regulators forced a writedown of its risky mortgage investments and a bid to raise new capital failed.

Guaranty has $13.4 billion in assets and operates 160 branches in Texas and California -- two of the three best banking markets in the nation, thanks to their size and population growth.

But the bank's capital problems and its smallish, scattered network of branches could detract from Guaranty's appeal, making it tough for regulators to find a buyer quickly -- or without substantial federal subsidies.

"This may not be closed as quickly as you think, since it will require bids and rebids," said Miami banking consultant Ken Thomas.

That means resolving Guaranty's failure is likely to be costly to the FDIC's deposit insurance fund, whose balance is at its lowest point in almost two decades.

The Federal Deposit Insurance Corp. isn't the only one taking its lumps. So have some big investors.

Shares of the bank's parent, Guaranty Financial, have dropped 97% since a group led by billionaire Texas hotel mogul Robert Rowling and Icahn, the renowned New York corporate raider, poured $600 million into the company in June 2008.

Other big Guaranty holders whose stakes stand to be wiped out include hedge fund managers David Einhorn, who was among the most persistent skeptics of Lehman Brothers before its collapse, and Dan Loeb.

"Relatively low franchise value and the fact that two big money investors already got burned on this bank may suggest less interest than with BankUnited," said Thomas, referring to the Florida thrift that failed in May and was bought by a group of private equity investors.

BankUnited had half as many branches and operated in only one state, but had a strong competitive position in the most lucrative counties -- something Guaranty lacks.

Despite BankUnited's relative attractiveness, its sale to investors led by vulture investor Wilbur Ross was hardly a walkover for the FDIC. The deal cost the FDIC insurance fund $4.9 billion.

A big tab on Guaranty would be costly to the deposit fund, whose balance was $13 billion at the end of the first quarter. The FDIC has estimated failure costs on cases since then at $11.2 billion.

A spokesman for the FDIC stresses that it has already set aside an additional $22 billion for failure-related costs in 2009, and adds that congressional action this spring gave the agency access to $500 billion in Treasury credit.

Though Guaranty has been around since 1988, it came public less than two years ago. Guaranty was part of the Temple-Inland (TIN) cardboard-box conglomerate until Icahn pressured the company to split up at the end of 2007. Guaranty shares were then distributed to Temple-Inland holders.

Guaranty's chief executive at the time, Ken Dubuque, assured investors that despite the gale force winds sweeping the financial world, the bank would be safe.

"We're keenly aware of the importance of good credit, disciplines and effective risk management, in good times and in difficult times," he said on the bank's first earnings conference call in February 2008.

But Guaranty's risk management soon was found wanting. The bank aimed to expand beyond lending to the builders of office buildings, shopping centers and houses to new areas such as small business and corporate energy lending.

Because its thrift charter obliges Guaranty to keep 70% of its assets in housing-related investments, the bank matched growth in other areas with expanded investments in housing. That, Dubuque said, is how the bank ended up taking on a giant portfolio of mortgage-backed securities, backed largely by option adjustable-rate mortgages in California and Texas.

"We needed to increase the size of the balance sheet, so that was a relatively risk-free way of doing it," Dubuque told investors in 2008. "We also have liked the returns in that business as well."

But securities backed by option ARMs are anything but risk-free, as investors have learned. Among institutions that dealt most heavily in those were Washington Mutual, the Seattle thrift that collapsed in September with $307 billion in assets, and Wachovia, which was sold to Wells Fargo (WFC, Fortune 500) later in 2008. Other big option ARM users included failed California savings banks Downey Financial and PFF.

Losses built at Guaranty over the past year, and Dubuque quit without explanation in November. In April regulators told Guaranty to raise more capital. When that effort failed, they told Guaranty to write down the value of the mortgage-backed securities by more than $1 billion. That move, announced this month, left the bank with negative capital of $748 million, according to filings.

Despite its many problems, Guaranty is -- for now -- operating as usual.

"We are open for business. We continue to work with our regulators," Guaranty said Friday in an emailed statement. "We are focused on providing the best customer service possible and believe we can avoid any disruptions to our customers."

Talkback: Do you think more big banks are likely to fail? Share your comments below.