(Fortune Magazine) -- Just after 9 a.m. on a humid Friday in late September, the trustees of Auburn University gather in a campus hotel conference room for their first meeting of the new school year. Even this early in the day there's a bit of electricity in the air. It is football season, it's a home-game weekend, and expectations are running high.

The Tigers have a new coach and are undefeated through their first three games. When 87,000 orange-and-blue-clad fans gather in Jordan-Hare stadium on Saturday night to watch Auburn vanquish another overmatched out-of-conference visitor, chests will be puffed up.

But first a bit of higher-ed theater must be played out. Auburn is a state-funded school, and the trustees' meetings are open to the public. More than 100 university staffers, faculty, reporters, and interested citizens have gathered this morning to watch the board tend to school business.

After presentations from the audit and compensation committees, the focus turns to the school's 2010 budget and to Robert E. "Bobby" Lowder, the chairman of the finance committee and the longest-serving member of the board. Because of the ailing economy, Auburn is looking at a drop-off of close to $100 million in state funding from 2008 to 2010, and hard choices need to be made. "What we're facing here is a tough situation," says Lowder. "And it could get worse."

The same could be said of Lowder himself. On Aug. 14, just a few months after Lowder, 67, announced his retirement as CEO and chairman of Colonial Bancgroup, federal and state regulators took control of the company he had founded and spent 28 years building into a regional banking powerhouse.

The seizure of Colonial's roughly 350 branches and $26 billion in assets -- the bulk of which were then handed over to BB&T (BBT, Fortune 500) of North Carolina -- made it the sixth-biggest bank failure in U.S. history, the worst this year, and the third largest since the beginning of the credit crisis that plunged the markets into turmoil in 2008. (Only Washington Mutual and IndyMac, which went under last year, were bigger.)

At an estimated cost of $2.8 billion to the Federal Deposit Insurance Corp., the Colonial collapse helped push the already depleted FDIC further into crisis mode. (In September, the FDIC proposed a plan to raise $45 billion to replenish its coffers.)

As Colonial's largest shareholder, with some 8 million shares, Lowder personally lost $164 million on paper during the stock's plunge from its 2006 high to its delisting from the New York Stock Exchange in August. To many of his enemies it's more than a little ironic that Lowder (pronounced "louder") has retained his role overseeing Auburn's finances.

And Lowder has made plenty of enemies over the years. His name might not be familiar outside Alabama, but he is easily one of the most feared, loathed, and some say misunderstood men to wield power in this state since George Wallace -- the governor who first appointed him to the board in 1983.

Lowder has been accused of making backroom deals with governors and treating the Auburn football program like a private fiefdom. (Because of his influence over Auburn's athletic program, three years ago ESPN named him the most powerful booster in college sports.)

At various times Lowder has been at war with Auburn's faculty, its student newspaper, its alumni association, and some of his fellow trustees -- developing a reputation along the way as a tyrant with a vindictive nature. It has been alleged that Lowder made a death threat to one board member he clashed with.

To Lowder's defenders, he is a tough but ethical businessman whose two passions in life have always been Colonial Bank and Auburn, and not necessarily in that order. They describe a shy person whose introverted nature has often been misperceived by critics, and point to his and his family's almost unmatched generosity over the years in giving back to his school.

The Edward L. and Catherine K. Lowder Business Building (named for his parents) and the Charlotte G. Lowder Student Development Center (named for his wife) are among the monuments to his giving. One acquaintance who has seen him recently says he appears "humbled." (Through his lawyer, Lowder declined to answer questions or respond to allegations for this article.)

Lately Lowder has suffered personal as well as business setbacks. In the past two years his 37-year-old son died suddenly from an aneurysm, his beloved 93-year-old mother passed away, and his longtime consigliere -- a brilliant lawyer named Jack Miller who served as both Colonial's primary outside counsel and its vice chairman -- waged a losing battle with cancer that ended with his death in July.

As much as Lowder has already lost, he could still lose much more. A slew of class-action suits have been filed by both Colonial shareholders and former employees that charge Colonial with reckless and dishonest conduct and name Lowder as a primary defendant.

One thing that seems clear is that Colonial Bank, led by Lowder, binged on the real estate bubble probably as much as or more than any sizable bank in the country, making dozens of acquisitions to get into fast-growing markets such as Nevada, Atlanta, and especially Florida, and concentrating more than 85% of its loan portfolio in real estate.

Perhaps most worrying for Lowder is an investigation by the FBI and the Office of the Special Inspector General for the Troubled Asset Relief Program into Colonial's so-called warehouse-lending business.

Colonial applied for $550 million in TARP funds last fall but was never cleared to receive a bailout. On Aug. 3, just 11 days before regulators shut down Colonial, agents raided the bank's offices in downtown Orlando, where the warehouse lending was managed, and spent hours carting away boxes of documents.

As the man who signed Colonial's financial statements, Lowder could face civil or even criminal charges if evidence of fraud is found in the bank's TARP application.

No matter what the outcome of the lawsuits and the investigations, the story of Bobby Lowder is more than another tale of a tyrannical banker or overzealous college booster. It's a classic yarn of a steady rise to power and a dizzying fall -- a narrative of hubris with a Southern twist. "I'm anxious to see what comes out like everybody else, but I can't see Bobby ever knowingly doing anything dishonest," says a former executive at Colonial. "I think he just overreached."

Troubles with TARP

For a man who's often been portrayed as a larger-than-life villain, Lowder hardly looks intimidating in person. About 5 feet 8 inches tall, he is slight and stoop-shouldered. When he smiles, he tends to bust out in a wide, outsize grin. He has a tall forehead, and the back of his skull, rather than curving naturally, angles down sharply. A fellow Auburn trustee once described him as "this li'l ol' harmless guy who looks like Howdy Doody."

Like many a Southern banker, Lowder favors conservative gray suits. But tailoring is no trivial matter to him. One former employee describes him as "completely into clothes. If someone was wearing a cheap suit he would know it, and he would let them know it. If he didn't think a woman was dressed nicely, he might tell someone to give her directions to Neiman Marcus."

At Colonial, Lowder surrounded himself with attractive, well-groomed women in high-ranking positions. "One of the great things about working for Bobby Lowder was that he encouraged the females that worked with him to grow within the company," says one former longtime female employee. When he retired, the CFO, COO, and chief credit officer at Colonial were all women.

"I asked him about that once," says another woman who worked closely with Lowder at Colonial. "He said, 'In my experience it's always been men that betray me. Women are more loyal.'" His current wife was once his trusted secretary.

Since announcing his retirement in May, Lowder and his wife have left his hometown of Montgomery -- the state capital and the city in which Colonial had its headquarters -- and moved full-time to Auburn, a quaint but fast-growing town of 56,000 people. From there he will presumably spend a good amount of time working on his legal defense. There is a lot to sort through.

At the heart of Colonial's legal troubles is the bank's attempt to participate in the TARP program. Colonial announced last October that it had applied for a bailout. And on Dec. 2 the bank said that it had obtained preliminary approval to receive $550 million from the Treasury, subject to certain conditions, which it didn't specify. Its stock, trading at $2 at the time, jumped 54% that day.

When Colonial announced its fourth-quarter earnings on Jan. 27, however, Lowder and his executive team revealed that Colonial would receive the TARP money only if it could raise $300 million in private capital from another source -- roughly double the amount of its equity at the time. Angry about the belated disclosure and sensing the unlikelihood of that scenario, investors dumped the stock in droves. And the class-action lawyers started to circle.

At the end of March, hours before a deadline set by regulators to boost its capital and stay eligible for TARP, Colonial announced that it had reached an agreement under which Taylor Bean & Whitaker, an Ocala, Fla., mortgage lender, would inject the $300 million Colonial needed.

In exchange, TB&W, a major Colonial customer founded by a burly college dropout named Lee Farkas, would get 75% of Colonial's common equity. Regulators never approved the agreement. It was abandoned on July 31. On the same early August day that federal investigators raided Colonial's Orlando office, they also executed a search warrant at TB&W, which shut down and declared bankruptcy later that month.

Federal investigators are tightlipped, but they appear to be focusing on Colonial's warehouse-lending business. Warehouse lending involves extending short-term lines of credit to mortgage brokers so that they in turn can make individual loans, which are then sold, packaged, and securitized.

Over the past decade Colonial had become the biggest warehouse lender in the country. Warehouse lending allowed Colonial to rake in generous fees for keeping loans on its books for a short time. But it required immense amounts of paperwork to track the underlying loans. And that paper trail could be crucial evidence for investigators.

"What they're looking for is whether or not Colonial was way undercapitalized before they submitted the TARP application, and as a consequence was the balance sheet fraudulent," says Tony Plath, a banking professor at the University of North Carolina at Charlotte and an industry consultant. "That's one issue: Did they write down their loan book fast enough? And then No. 2, Did they take proper collateral with the loan positions they had outstanding to brokers? And I'm guessing they didn't."

If investigators conclude that there was willful deception, they could bring criminal charges. "And then they will strip Bobby Lowder of his cars and his houses and his football tickets at Auburn," says Plath. "And it won't matter anymore that he's got political connections in Alabama or that his daddy was a big shot."

In the shadow of Daddy

Most boys grow up admiring their fathers. Bobby Lowder had to follow a living legend. The story of Ed Lowder's rise from southern Alabama country boy to captain of industry reads like a chapter from "The Greatest Generation."

After getting a degree in agriculture, he worked as a county extension agent before serving as an artillery commander in the Battle of the Bulge during World War II. When he returned to Alabama in 1946, he was asked to help start and run an insurance company for the Alabama Farm Bureau. Under Ed Lowder's leadership, the company, known today as Alfa Insurance and based in Montgomery, grew into one of the largest insurers in the South.

Ed Lowder made his name in the insurance business, but he built a family dynasty in real estate. In the mid-1950s he started buying and selling houses on weekends. Then he got into homebuilding and commercial land development, and acquired a mortgage-lending company that he renamed Colonial.

There was never any doubt that Ed's three sons were going to follow their hard-driving father into business. Bobby, the oldest, was a quiet, intense student but not particularly popular.

"Ed had a big presence," remembers Boyd Christenberry, who worked for the senior Lowder for 25 years. "Bobby, he never was too eat up with a warm, attracting personality." He was a scrawny backup center for two years on the football team at Sidney Lanier High School, where Bart Starr had played just a few years earlier, but he stood out more for his grit than his talent.

"Bobby was real competitive," says Billy Livings, one of his coaches. "He got that from his daddy." Also like his father, Bobby went to Auburn and spent a few years in the Army. In 1966 he returned to Montgomery and began running Colonial Mortgage.

In the early 1970s Bobby and his younger twin brothers, Tommy and Jimmy, bought the Colonial company from their father and started investing in banks. Eventually they divided the real estate empire into three parts: owning property, building, and lending.

"To keep us from killing each other, we needed to be in charge of different things -- is that a good way of saying that?" said Jimmy in an interview in 2004. Today Tommy runs Colonial Properties Trust (CLP), a Birmingham real estate investment trust that trades on the New York Stock Exchange with a market value of $700 million. Jimmy runs the Colonial Co., a private builder based in Montgomery. Bobby, of course, became the banker.

In 1981, Lowder created Colonial Bank by acquiring a failed institution called the Southland Bancorp. From the beginning, his strategy was to build his deposit base by snapping up community banks around the state and using the capital to make real estate loans.

After the Riegle-Neal Act of 1994 gave banks greater freedom to operate across state lines, he set his sights on faster-growing markets, such as central Florida. He established Colonial Bancgroup as a holding company to own Colonial Bank's different operations and took the company public with a listing on the NYSE in 1995.

Though its headquarters remained in Montgomery, beginning in the late 1990s Colonial became essentially a Florida lender. In 1996, when Lowder first entered the market with the purchase of the $230 million Southern Banking Corp., Colonial had $4.9 billion in assets. By 2002, after a half-dozen more acquisitions in the Sunshine State, the bank's assets had grown to $15.8 billion, and 44% were in Florida. By 2008 about 53% of its loan portfolio and 62% of its $26 billion in assets were in the state.

Lowder's management style, insiders say, was bound to create problems as the bank grew bigger. He insisted on being in complete control. He stacked his board with high-profile allies such as former Auburn football coach Pat Dye and Montgomery pastor John Ed Mathison, or recruited businessmen he knew through Auburn ties.

"I thought he was a good guy, always looking out for the shareholders," says one former executive who left within the past two years. "He just had a very difficult time giving people authority. When an organization is $3 billion, you can do that; when it's $26 billion, you can't."

The problem was exacerbated by Lowder's fickle nature, some former colleagues say. "He either loves you or hates you -- there is absolutely no in-between," says a onetime member of his inner circle. "And everyone who got into a role where they worked with him closely knew that one day he would probably turn on them."

Consider the Biscuits story. In 2004, Montgomery got a new minor-league baseball team called the Biscuits. A Colonial marketing executive bought a few season tickets to use with clients. A few days later, while on a business trip, he got a call from his boss. She asked whether he'd bought Biscuits tickets, and when he said yes, she told him to return to Montgomery. "Will I have a job when I get there?" he asked. "No," was the response. Mr. Lowder, he was told later, didn't like the Biscuits.

The football fanatic

One thing that no one disputes about Lowder -- friend or foe -- is his passion for Auburn. In 1995, when then-Gov. Fob James tried to replace Lowder on the Auburn board, he refused to leave. Lowder fought James in court and enlisted fellow trustee and political ally Lowell Barron, a state senator, to prevent a vote on a nominee to replace him.



Meanwhile, he threw his support and his money behind Lt. Gov. Don Siegelman, James's opponent in the next gubernatorial election. Siegelman defeated James. In 1999, he reappointed Lowder to another 12-year term and, some prominent Auburn alumni say, agreed to let Lowder vet new appointees. Siegelman says he admired Lowder's service to Auburn and denies that Lowder had input on other picks.

Lowder had no problem mixing Colonial business with Auburn business. When the alumni association launched its first branded credit card in the mid-1980s, Colonial was awarded the contract. Over the years Lowder has installed a few of his fellow trustees on the Colonial board, and others have had business relationships with his bank. "People always talk about the millions he's given to Auburn," says Paul Davis, a local newspaperman who's been writing columns critical of Lowder for two decades. "I always say, What about the millions he's made off of Auburn?"

And, critics say, Lowder used his sway to settle scores. For years, Lowder's most vocal opponent on the board of trustees was a lawyer named John Denson. Denson had a particular interest in Auburn's Ph.D. program in economics, which had a free-market bent and attracted graduate students from around the world. When the board conducted a university-wide review of academic programs in 1999 the Ph.D. program was eliminated despite its robust health.

During the same review, the journalism department, which Lowder had publicly blamed for pushing the school newspaper to write negative stories about him, was folded into the communications school.

"I've never met anybody like that before," says Denson, who is now a judge in the Auburn area. "Lowder plans long range. He plots. It's not a flare of temper. It's a calculated plan of control."

Denson has particular reason to view Lowder in such a negative light: Lowder once allegedly threatened to have him killed. In a 2006 lawsuit involving Auburn trustees that was to have appeared before Denson, Auburn's former athletic director submitted an affidavit saying that Denson had told him that Lowder had once said to him, "John, I can have you killed with no problem at all." Denson recused himself from hearing the case and says now he can't comment because it is a judicial matter.

In 2001 former Sun Microsystems president Owen Brown, like Lowder a 1964 graduate of Auburn, withdrew a $2 million pledge to the school to protest Lowder's meddling. "I don't think Bobby knows how to debate something logically," says Brown. "He's either in control or he's down your throat."

Not surprisingly given that it's Alabama -- where, as they say, "college football isn't life or death; it's more important than that" -- Lowder's most high-profile clashes have involved the football program. (It might be a good juncture for the writer to offer, as full disclosure, that I am a graduate of the University of Alabama. On learning that, certain Auburn readers will give this story no credibility.)

From the time he joined the board of trustees, Lowder has shown a special interest in day-to-day operations, chairing an athletics committee of the board that met in private and, insiders say, calling coaches directly to mandate changes. And he has shown that he runs out of patience with football coaches just as he does with bank executives.

In 1998 Terry Bowden resigned as football coach with five games remaining in the season, despite a 47-17-1 overall record, after he was given the message that Lowder wanted him gone. Lowder denied any involvement. There was an outcry among alumni and students, but soon Bowden's replacement, Tommy Tuberville, was winning and the controversy subsided.

A few years after his resignation, however, Bowden gave an interview to local reporter Davis alleging that Lowder had organized a program in which wealthy alumni contributed cash to a slush fund to pay players to sign with Auburn.

Several people with knowledge of the system and Lowder's role in it told Fortune that Bowden was telling the truth and that they had seen a book with a list of the slush-fund donors. Again, Lowder would not comment for this story, and we could find no record that he had addressed this question in the past.

Lowder's next showdown over football -- what became known as Jetgate -- attracted even more national attention. When Auburn failed to live up to expectations in 2003, Lowder apparently got fed up with Tuberville. That November he dispatched the university's interim president, its athletic director, and two fellow trustees to Louisville to try to hire away its coach, Bobby Petrino, a former Auburn assistant.

The news leaked, and the groundswell of support for Tuberville forced Auburn to not only keep him but also extend his contract. The president and the athletic director left their jobs in the aftermath of the incident. Lowder denied any involvement, despite the fact that the contingent had traveled on his private jet.

Last year, after a losing season, Tuberville was forced out. "It was only a matter of time," says one former athletics department insider. "Lowder was going to get him eventually."

Lowder's high-profile role in athletics and his clashes with faculty eventually attracted attention from the school's accrediting organization. In December 2003, shortly after Jetgate, the Southern Association of Colleges and Schools put the university on probation and specifically named Lowder in a report citing micromanaging by trustees.

The outcry has empowered others to push back against Lowder. Alabama's current governor, Bob Riley, has installed some new trustees -- such as former Vodafone chairman Sam Ginn and Birmingham investor Raymond Harbert -- who are strong, independent voices and not beholden to Lowder.

"I was kind of a supporter of Lowder until I saw him up close," says Andy Hornsby, 65, an Auburn alum, former state official, and onetime director of the U.S. Food Stamp program, who has clashed with Lowder. "I think I'm like a lot of Auburn folks who've been waiting until the day that scoundrel stops holding our university hostage."

Into the storm

Even after the real estate bubble in Florida began to burst, Lowder expressed confidence that Colonial wasn't going to have major problems. "I think our credit quality remains excellent," he said during a conference call to discuss Colonial's second-quarter earnings in July 2007. He added, "Real estate goes through cycles, and that's what we are experiencing now. What you should be comfortable with about Colonial Bank is our ability to manage ... through these cycles as we have done successfully in the past."

By July 2008, however, as the storm clouds gathered in the credit markets, Lowder apparently realized that some sort of dramatic action was needed. So he created a war room: On the 25th floor of Colonial's Tampa office, a team of bankers commandeered a conference room and four adjacent offices to store documents relating to what Lowder said were 42 bad credits on Colonial's books. A parade of potential buyers for the underlying properties came in to review them.

Lowder started repeating a speech to investors, employees, and regional bank directors alike. "He would pull out this piece of paper," says a banker who worked closely with Lowder during this time, "and say, 'Here are our bad loans. They're all right here. It's $325 million of notes, and we're working through it right now. Our family has been in homebuilding for 50 years, and we know how to do this. We just have to get the stuff on this piece of paper fixed and it's over, and we'll move on.'"

As more loans started to go bad, people inside Colonial would joke, "Does Lowder know his piece of paper is getting bigger?"

Lowder's credibility was not helped by the fact that he was viewed by many in the industry as shopping regulators. In 2003 Colonial switched its charter from the Alabama Banking Department to the federal Office of the Comptroller of the Currency.

According to sources inside and outside Colonial, Lowder didn't like the fact that the examiners were pushing him to decrease his exposure to real estate and increase his capital reserves. In June 2008 Colonial switched back to state supervision, saying that it wanted to save money on fees.

"He always had a belligerent attitude toward regulators," says a former Colonial executive in Florida. "And when he switched the charter again in 2008, that sent a bomb up their butts."

In his final months as CEO, Lowder scrambled to pull Colonial out of its death spiral. In meetings he projected confidence that a solution would be worked out. And he held on to all of his Colonial stock, something his lawyers are eager to point out.

The day after the trustees meeting, Auburn hosted the not-so-mighty Ball State Cardinals. A few minutes before kickoff, Lowder showed up at the stadium with his grandchildren and chatted politely with the staff before taking an elevator to the luxury suite he still owns near the president's box.

At kickoff, a steady rain was falling. Near the south end-zone scoreboard in the stadium, the Colonial ad that had once stood side by side with an Alfa billboard had been replaced by one for BB&T. After a slow start, the Tigers began to rack up points on their way to an easy 54-30 victory. Cheers of "War Eagle!" rang through the night. And Auburn's most notorious supporter watched from high above, still secure in his role at Auburn. Section 60 of the Alabama constitution provides that "no person convicted of ... infamous crime, shall be ... capable of holding any office of trust ..." For now, that's not him.

Reporter associates Doris Burke and Beth Kowitt contributed to this article.