A $4 supermarket sandwich has to be pretty damn good for two adults to start fighting over it. Looking at it, there was nothing special about this one – a plain roll in a plastic package, no salad or garnish, a little too much mayo. It was only the gasp of pleasure that accompanied my girlfriend's first bite that told me it was worth trying. In it was the freshest, softest lobster I'd ever eaten, not minced-up leftovers (as in too many lobster rolls) but large meaty chunks of claw and tail that had us squabbling over the last mouthful.

We were picnicking on lovely Good Harbor Beach in the town of Gloucester on our first morning in New England, and that sandwich was a great taste of things to come. It was the sheer abundance, quality and low price of the lobster and other seafood here that drew me to New England.

I wasn't the first Brit to be struck by the profusion of Homarus americanus, the American lobster. When the colonialists arrived they saw Native Americans simply picking them off the seashore, and until the mid-19th century, the "poor man's supper", as it was then known, was so readily available that servants demanded not to be made to eat lobster more than twice a week.

Unlike in most of the world, lobster still has blue-collar connotations in New England, particularly in the hundreds of shacks that serve "lobster in the rough" along the coast from Rhode Island to northern Maine. This no-frills atmosphere was in evidence at our first shack, Roy Moore Lobster Co in Rockport, Massachusetts, a classically pretty New England village – all clapboard houses and small craggy bays. Roy's simple store, with its formica tables and plastic wineglasses, makes few concessions to wealthy weekenders from nearby Boston. As we cracked open our first succulent lobster ($12.50 each) with a pair of nutcrackers, our inexperience showed: a jet of water squirted out, almost hitting the diners at the next table. (So that's what the plastic aprons are for!) They didn't seem to mind, but a group of young, well-dressed girls turned their noses up ... whether at us or at the no-frills service I wasn't sure.

The girls might have been happier up the road at the Lobster Pool, with its millionaire views of Folly Cove. This red wooden bungalow has one of the best settings Massachusetts can offer – though the service is still no-nonsense. We ordered lobster and scallops at the self-service hatch, and sat at a picnic table, uncorking a bottle of Californian wine (Rockport is dry, so bring your own) and enjoying a fabulous sunset as our kids played on the rocks.

Folly Cove is at the mouth of the Essex river, home of New England's other speciality, the clam. Woodman's restaurant is the spiritual home of the clam – Chubby Woodman claimed to have invented the fried clam in 1916. Owner Steve Woodman, grandson of Chubby, took me down the river on his boat to see where they come from.

In the hazy morning light, the clam diggers were bent double in the thick mud of the estuary, searching for the large white shells with nothing but a pitchfork – as the law decrees.

"It hurts my back just looking at them," I told Steve.

"Sure does," he replied. "It's the only job in the world where you spend all day with your ass below your head."

Out on the mudflats Sergio, a professional clammer from Albania, was going at it like a JCB. Beside him was Bob Burns, a 70-year-old local who has been clamming for fun for 40 years.

"I don't do it for the money," said Bob. "I just like to take some for the family. I love it out here. It's a peaceful place with its own silent beauty."

In this part of the world, clams are as important as lobsters, and back at Woodman's a queue was forming at the self-service counter at 11.30am. People were piling their trays with fried and steamed clams, lobster, onion rings and fries. Steve fixed me a bowl of clams, coated in a thin batter and fried, as they always have been, in lard. Eaten with homemade tartare sauce, they were deliciously sweet – but half a bowl was enough. I enjoyed the "steamers" more – thick juicy clams dipped in clam broth to clean off any mud, and then in melted butter.

That evening at my hotel, the 19th-century Emerson overlooking the ocean, I was singing the praises of the local seafood shacks when a salty old dog at the next table leaned over. "They're not bad down here," he said, "but for the real deal you need to head up my way to Maine. The water's colder, so the lobster gets nice and fat and juicy."

I'll be heading up there next time.

• Five nights' B&B at the Emerson Inn in Rockport, with Virgin Holidays (0844 557 3859) starts from £626, including car hire and direct Heathrow to Boston flights. Further information: Discover New England (020 8237 7977). For river excursions see Essex River Cruises.