The last time I wrote a blog that included oysters, the story ended with me running out of John Besh's August restaurant in New Orleans and beelining it to a toilet in the Windsor Court Hotel across the street. Not that I got sick off of Besh's food. The oysters actually came from another place a few hours prior. It was the summer of the infamous BP oil spill in the gulf. No matter.

This time, I was at the iconic American Hotel in Sag Harbor, New York, a historical village incorporated in the 1700s and a town of beautiful homes. Somehow, Jonathan Drew was able to secure a table for us at the hotel's coveted front porch and I recently had the opportunity to join him for lunch, which began with local Peconic oysters that did not make me sick.

Each year, Jonathan takes the time to visit Sag Harbor so he can spend a few days with family and decompress. He's been coming here since childhood, so, needless to say, there's a special part in his heart for this harbor. Because Sag Harbor belongs to both East Hampton and South Hampton, it also gets the typical influx of summer tourists and moneyed Manhattanites looking for a waterfront reprieve from the city. The locals, of course, have mixed feelings about the seasonal crowd, but it's hard to not be charmed by Sag Harbor Village. Even the smallest of homes are very rich in architectural details, and The American Hotel has done a great job preserving its interior and exterior character over the decades. It's also somewhat of a tony town with some pricey restaurants and great little shops.

When I stepped up to The American Hotel's porch, I admit that I fully expected Jon to be puffing a cigar, brazenly flouting the town's laws and festooning the hotel's exterior with plumes of smoke. Instead, I came upon a rather sedate guy sipping a Talisker, watching the scenery and chewing on an unlit pipe. It was one of the new pipes from his Drew Estate Tsuge pipe line introduced at last month's IPCPR trade show. They're made in Japan and are pretty hip, designed with a very nice combination of classic burl and newer materials. Very Drew Estate. Jonathan developed an eye for classic design at a young age. I never knew this until our quiet lunch, but he comes from a family background of art and antiques. His parents owned antique shops and he used to accompany his father to the auction houses. This type of early exposure to Victorian era art, Art Deco and Art Nouveau is sometimes exactly what's required to really turn on someone's artistic genes.

"We had all kinds of antiques, we had bronzes," said Jon. "A few things from Tiffany, but that stuff is so expensive. My father was cleaning off two beautiful Tiffany lamps once, but the shades were very close together and he broke them accidentally." Jon winces when he tells this story. I don't blame him. Those lamps went for thousands of dollars even 30 years ago. Jon is looking down at the table when he tells the story, as though all that broken opalescent glass is suddenly in front of him. Then he grabs an oyster. We then moved on to a series of different caviars—colorful caviars from bright orange to brilliant green to leaden gray to charcoal black—all of which were American. Has anyone ever proposed a cigar and caviar pairing?

I flirted with the idea of lighting something up. Jon gave me a few Norteño by Willy Herrera belicosos. These were also introduced at the trade show. They're dark and box-pressed and come from Willy Herrera, Drew Estate's newly minted master blender and the creator of the Herrera Esteli line.

Jon let me in on plenty of things he has planned for the future. Some of the things I can talk about, like, for instance, a new Liga Privada that Herrera is working on. You won't see it until 2015. Also, Drew Estate will be distributing premium cigars from Balmoral, which is owned by Royal Agio from Holland. A few other things are in the works, too, but I can say no more. Only that Jonathan is a creative mind on fire and comes up with ideas faster than he can put them into motion.

After a main dish of delicious Chatham cod, we went over to his car. He has a gleaming black, fully restored, 1964 Lincoln Continental. It's really an attention grabber.

"This is the same model car that Kennedy was killed in," Jonathan told me. "Every time I put the key into the ignition, I have no idea if she's going to start up or not. I say there's a 20 percent chance that it won't even turn over."

There is a ticket on the windshield of his car, but Jonathan seems unbothered. He had parked his vehicle in a spot with a two-hour time limit, and in this town, they check. I guess that kind of strict enforcement comes with the territory when visiting such a boutiquey town.

I still find it strange that in the entire village of Sag Harbor—a village that's been around for hundreds of years and a village that channels a very cosmopolitan crowd—you can find a waffle bar (yes, a waffle bar) but not a single tobacconist. Not one.

We cruise around town a bit, Jon still chewing on the Tsuge pipe and me with the Norteño. He doesn't like a certain sound his engine is making, so I tell him to drop me off near my car, and to take his car home before it stalls out. I'm parked near the docks where whaling boats and commerce ships have been replaced by behemoth luxury yachts. The harbor glitters beautifully in the evening sunlight and there are sailboats racing in the distance. How picturesque. I return to my car, on which there is also a parking ticket. Did I ever tell you how much I hate that town?