Dawn Gilbertson, Samantha Incorvaia, and Graig Graziosi

The Republic | azcentral.com

Airline blames a failed router for the problems

Southwest's computer woes, reminiscent of a similar meltdown a year ago, come during the busy summer travel season

Nearly 1,000 flights canceled over two-day period

Southwest Airlines CEO Gary Kelly apologized for his company's nationwide system meltdown Thursday, even as travel nightmares lingered from the computer outage.

The airline, which has a major hub in Phoenix, said most of its computer issues had been fixed, but announced it had canceled another 450 flights at 12:30 p.m. Phoenix time Thursday. That number was up from 300 earlier in the day, and due largely because flight crews and aircraft were not in the right cities and airports. Passengers remain stranded across the country, and Southwest is offering "flexible rebooking" to those who hold tickets through Sunday.

But customers reported and complained of hold times 90 minutes or greater as they attempted to rebook flights.

Ironically, the problems began the day before the airline announced record profits of $820 million in the second quarter.

More than 700 flights were canceled Wednesday, and an estimated 2,000 were delayed..

“I want to apologize to all of our customers who were affected yesterday and today,” Kelly said in the earnings call Thursday morning. “It is absolutely not the kind of service our customers expect or what we expect of ourselves.”

RELATED:Southwest extends fare sale in light of computer failure

Kelly said problems occurred when computer equipment and backup systems failed at about 1 p.m. Wednesday. It took roughly 12 hours to bring systems back online. A failed router started a cascade of problems, airline officials said Thursday. Backup systems also failed, with catastrophic results.

"We've not had anything like this in our history," Kelly said. "There were multiple attempts to recover quickly but it all went to bedrock and we had to reboot."

Kelly also blamed aging technology, which is scheduled to be upgraded over the next three to five years. Wednesday's failures will not hasten those improvements, though adjustments will be made to lessen the chances of a repeat.

Cancellations at Phoenix Sky Harbor International Airport early Thursday morning mostly appeared to be arriving flights, but departing flights were scheduled on time. Check-in and security lines were long but normal during the peak summer vacation season.

In Las Vegas, a flight from Phoenix that landed at McCarran International Airport around 8 a.m. sat on the tarmac for 40 minutes because there were no open gates. The pilot blamed the technical "glitch" and it appeared many passenger would miss connecting flights.

Southwest's website was up and running Thursday morning, but a "travel advisory" warned passengers that lingering effects would continue well into the day.

"Thank you for your patience today as we recover from yesterday’s technology outage," the advisory said. "Southwest.com has been restored with full functionality and is available for your travel needs."

In Phoenix, many customers were busy adjusting their travel plans.

"I was supposed to fly out at 7:45 this morning, but my flight was cancelled," Josh Baker said. "I'm not rescheduling my flight, I'm just canceling the trip."

Tricia Godlewski was waiting for her daughter's arrival from the Midwest, she said.

"My 15-year-old daughter was stranded in Chicago. She was flying back from Connecticut and she made it on the three-hour flight to Chicago, but the one from there was cancelled. She'd flown direct flights alone before, but this was her first time dealing with connecting flights. She called me crying and freaking out, but Southwest took care of her," Godlewski said.

"They came and got her and two other girls from other flights and had them all sleep on cots in a little room and they got them food and everything. They said she was going to fly in on the 5:30 flight this morning but that was cancelled too, so now I'm hoping she'll be flying in on the 8:00. But Southwest was wonderful in helping her."

They system failure froze Southwest's entire operation. Travelers were unable to check in online, book tickets or check flight status. The error message apologized for the airline's "major outage.''

An employee at the airline's Phoenix telephone reservations center said Wednesday that employees were answering a flood of calls but unable to help passengers on any front because their computer systems were frozen. One said they were staring at a never-ending hourglass on the screen.

CEO Kelly, in the conference call, praised the "heroic" efforts of his employees over the past 24 hours.

Travelers going to the airline's website, Southwest.com, Wednesday afternoon were greeted with this message: "We're working hard to get you where you want to be. Please do not refresh your browser. We will automatically transfer you to Southwest.com as soon as possible. Thank you for your patience.''

The problems on Southwest come during the hectic summer travel season. Last summer, Southwest also had major computer woes during its twice-a-year fare sale. Those problems lingered a couple of days.

The 700 canceled flights nationwide Wednesday represents 17 percent of Southwest's 3,900 daily departures. Out of Phoenix late Wednesday night, the airline canceled flights to destinations including Denver and San Diego, San Francisco, Los Angeles, Sacramento and Santa Ana, Calif., according to FlightStats.com. A Southwest spokesman said 30 flights nationwide have been proactively canceled for Thursday nationwide.



At Phoenix Sky Harbor International Airport on Wednesday, a line of customers snaked in front of the Southwest check-in desk in Terminal 4 for the majority of the afternoon. Airline officials darted back and forth along the line, hectically answering questions and directing customers to waiting areas or information kiosks.

At one point in the afternoon, Southwest grounded all of its flights, which resulted in delays for some customers and cancellations for others.

Southwest has more than 170 daily non-stop flights from Sky Harbor.

Late Wednesday night, a Sky Harbor official said restaurants and retail shops in Terminal 4 were keeping extended hours.

Jill Donovan was planning on flying out of Sky Harbor with seven of her co-workers to attend an out-of-state corporate event. She was told her flight was canceled once she reached the check-in desk. She said she and her co-workers were unsure whether they would be able to join the rest of their company for the event.

Ray Mercado had to put his job on hold due to the delays. Mercado, who delivers vehicles professionally, was supposed to fly out of Phoenix to Tulsa, Oklahoma, to pick up a school bus and drive it back to Phoenix. As of Wednesday afternoon, Mercado was unsure whether he would spend his night driving a bus or sleeping on the floor of Sky Harbor.

While many of the customers were understandably annoyed by the delays, Rodney Benton was content knowing the problem posed no danger to a passenger for whom he was waiting.

"My dad is 93; he's flying in from Austin. Honestly, in this day and age, as long as he's safe, that's all I care about, so a few-hour delay isn't a big deal to me," Benton said.

RELATED:Looking back on Southwest computer meltdown | Fare war: $39 flights from Phoenix to LA | What to expect for entertainment on your summer flights

RELATED: Looking back on Southwest's computer woes

Includes information from Republic reporters Alexis Egeland and Scott Craven.