You might think that just about everybody is buying groceries online today, as retailers like Walmart, Kroger and Amazon race to perfect their delivery services and tout their abilities to get food to shoppers' homes in under an hour. But that's not exactly the case.

Grocery shoppers are still concerned they're being charged higher prices online and complain about delivery drivers being late, among other disappointments.

In the U.S., a mere 3 percent of grocery spending takes place online today. Americans haven't been as quick to jump on board with placing their grocery orders from their computers or smartphones, especially when compared with markets like the U.K. and South Korea, where online grocery penetration can be as high as 15 percent.

Only a quarter of consumers have tried an online grocery service in the past year, according to a new survey of more than 8,000 U.S. grocery shoppers completed by consulting group Bain & Co. in collaboration with Google. And only 26 percent of those shoppers, or 6 percent of all U.S. consumers, went on to say they order groceries online more than once a month. Instead, most Americans are taking multiple trips to the grocery store each week.

"We've been early adopters in this country in almost every other retail category," Bain & Co. partner Stephen Caine said. "We know online grocery will explode at some point."

For now, though, grocery chains like Albertsons and Ahold Delhaize and delivery providers like Instacart and FreshDirect alike are grappling with how to get more shoppers to take advantage of their services. Fear of Amazon's dominance has pushed many companies to make these investments, even if they eat into profits.

"If one retailer is doing it, the others need to offer it," Stewart Samuel, program director at food and consumer goods research organization IGD, said. "It does have the potential to bring in new customers to your business. ... Retailers wouldn't be expanding [grocery delivery] at this pace if customers weren't responding."

Still, there's some convincing to be done.

Many shoppers want to be able to see and even touch certain foods like meat or produce before they buy it. That's also why packaged goods like chips and granola bars tend to be the most popular items placed in online shopping carts. And then there are always flaws with delivery services, like items being out of stock or a driver being late.

Only 42 percent of people using a grocery delivery service for the first time say it actually saves them time, according to Bain & Co.'s survey. One bad experience can potentially ruin a shopper's perception of the concept and make them never want to try it again. It's important for a company to get it right the first time because 75 percent of online grocery shoppers say they continue to use the first retailer they shopped from, the survey found.

And then there are other trust issues over pricing.

Typically, there are two pricing models that retailers follow when deciding how to mark up groceries and delivery fees online, Caine explained.

One is: a retailer will price items on the internet exactly like they would be in stores and then be transparent about how much the extra service charges are on top of that. Second is: a retailer will hike prices on items online to cover for the extra fees. Because some prices are noticeably inflated, "people tend to minimize what [groceries] they buy online because they think they are getting fleeced," Caine said. "You don't know whether you are getting a fair price."