The Russell family of Kirkland, Wash., makes about $230,000 with Charles Russell, 43 years old, working as a systems analyst for Microsoft Corp. According to the U.S. Census Bureau, that puts them among the top 5% of American households.

But their affluence might not be apparent based on the way Mr. Russell conducts his personal finances. He has no bank account, having dumped it due to irritation over fees and overdraft penalties. Instead, for more everyday transactions he uses a debit-card offered by NetSpend Holdings Inc.

"I have no need, desire or want to go to a regular bank," says Mr. Russell, who adds that a savings feature on the card offers a competitive interest rate.

Middle-class Americans are spending less time in the bank lobbies their parents would recognize. Today, 8.2% of the nation's households—nearly 12 million—are managing their finances without a bank, according to Census-based data the Federal Deposit Insurance Corp. will make public Wednesday. That is up from the 7.7% the bank regulator found in its 2009 report.

Another 24 million households, defined by the government as "underbanked," have a bank account but still dabble outside of the formal banking system by using payday loans, prepaid cards and other alternative means, the FDIC found. This population that has left banks, or uses their services infrequently, makes up 28.3% of America's households, the agency says. That is a slight increase from the 25.6% who fit into those categories in 2009.