An attack late last week that compromised the personal and business Gmail accounts of Matthew Prince, chief executive of Web content delivery system CloudFlare, revealed a subtle but dangerous security flaw in the 2-factor authentication process used in Google Apps for business customers. Google has since fixed the glitch, but the incident offers a timely reminder that two-factor authentication schemes are only as secure as their weakest component.

In a blog post on Friday, Prince wrote about a complicated attack in which miscreants were able to access a customer’s account on CloudFlare and change the customer’s DNS records. The attack succeeded, Prince said, in part because the perpetrators exploited a weakness in Google’s account recovery process to hijack his CloudFlare.com email address, which runs on Google Apps.

A Google spokesperson confirmed that the company “fixed a flaw that, under very specific conditions, existed in the account recovery process for Google Apps for Business customers.”

“If an administrator account that was configured to send password reset instructions to a registered secondary email address was successfully recovered, 2-step verification would have been disabled in the process,” the company said. “This could have led to abuse if their secondary email account was compromised through some other means. We resolved the issue last week to prevent further abuse.”

Prince acknowledged that the attackers also leveraged the fact that his recovery email address — his personal Gmail account — was not taking advantage of Google’s free 2-factor authentication offering. Prince claims that the final stage of the attack succeeded because the miscreants were able to trick his mobile phone provider — AT&T — into forwarding his voicemail to another account.

In a phone interview Monday, Prince said he received a phone call at 11:39 a.m. on Friday from a phone number in Chico, Calif. Not knowing anyone from that area, he let the call go to voicemail. Two minutes later, he received a voicemail that was a recorded message from Google saying that his personal Gmail account password had been changed. Prince said he then initiated the account recovery process himself and changed his password back, and that the hacker(s) and he continued to ping pong for control over the Gmail account, exchanging control 10 times in 15 minutes.

“The calls were being forwarded, because phone calls still came to me,” Prince said. “I didn’t realize my voicemail had been compromised until that evening when someone called me and soon after got a text message saying, ‘Hey, something is weird with your voicemail.'”

Gmail constantly nags users to tie a mobile phone number to their account, ostensibly so that those who forget their passwords or get locked out can have an automated, out-of-band way to receive a password reset code (Google also gets another way to link real-life identities connected to cell phone records with Gmail accounts that may not be so obviously tied to a specific identity). The default method of sending a reset code is via text message, but users can also select to receive the prompt via a phone call from Google.

The trouble is, Gmail users who haven’t availed themselves of Google’s 2-factor authentication offering (Google calls it “2-step verification”) are most likely at the mercy of the security of their mobile provider. For example, AT&T users who have not assigned a PIN to their voicemail accounts are vulnerable to outsiders listening to their voice messages, simply by spoofing the caller ID so that it matches the target’s own phone number. Prince said his AT&T PIN was a completely random 24-digit combination (and here I thought I was paranoid with a 12-digit PIN).

“Working with Google we believe we have discovered the vulnerability that allowed the hacker to access my personal Gmail account, which was what began the chain of events,” Prince wrote in an update to the blog post about the attack. “It appears to have involved a breach of AT&T’s systems that compromised the out-of-band verification. The upshot is that if an attacker knows your phone number and your phone number is listed as a possible recovery method for your Google account then, at best, your Google account may only be as secure as your voicemail PIN.”

AT&T officials did not respond to requests for comment.

Beyond such vulnerabilities, customer support personnel at mobile phone providers can be tricked into forwarding voicemails to other numbers. And this is exactly what Prince claims happened in his case.

“In this case, we believe AT&T was compromised, potentially through social engineering of their support staff, allowing the hacker to bypass even the security of the PIN,” Prince wrote. “We have removed all phone numbers as authorized Google account recovery methods. We are following up with AT&T to get more details.”

If you use Gmail and haven’t yet taken advantage of their 2-step verification offering, I’d strongly encourage you to take a moment today and do that. But even if you already use 2-step verification, you may want to review other security options. For example, you probably have supplied Google with a backup email address. If that backup email is also a Gmail account, is that account protected with 2-step verification? If not, why not?

Also, consider whether you want your 2-step verification codes sent to a mobile device that resides on a network whose customer service personnel could be vulnerable to social engineering attack. Alternatives include sending text messages or automated voice calls to a Skype line, or removing the mobile network entirely. Google has a mobile app for Android, Blackberry or iPhone called Google authenticator; it works much like a token or key fob, in that it can be used to generate a new code whether or not your phone has access to the mobile network or Internet.

Tags: 2-factor authentication, 2-step verification, Android, AT&T, Blackberry, CloudFlare, gmail, Google Apps, Google Authenticator, iPhone, Matthew Prince