SOME COME TO Block Island to behold the quintessentially New England beaches and rolling hills. Others come to bike, hike and maybe take a dip in the always-colder-than-you-imagine Atlantic Ocean waters.

But I came for a different reason: to sit atop a statue of a deer.

The statue in question occupied a prominent spot on the lawn of the Spring House, arguably the grandest of Victorian-styled hotels on an island with a number of such properties. For much of my childhood in the ’60s and ’70s, a summer week’s stay at the hotel was built into the family calendar, a guaranteed escape from the New York City heat. My father insisted on taking a picture of me “riding” the deer to mark our annual pilgrimage to this pear-shaped dot on the map just 13 miles off the coast of Rhode Island.

Jump ahead three decades to early July and I’m back on the Block, as they say, for the first time since then. And now I’ve got my wife and my son, 23, and daughter, 16, in tow. On our first full day on the island, I lead them to the Spring House, camera in hand.

Except the deer is no longer there.

A lifeguard at Fred Benson Beach Photo: Katherine Wolkoff for The Wall Street Journal

“Nothing stays the same,” I mutter to myself, imagining all the disappointments that await in our weeklong vacation. But it turns out that apart from the disappearance of the statue, the island has retained pretty much everything else.

A good number of hotels, restaurants and landmarks remain, as does the island’s character, shaped by its natural beauty and isolation. If you travel here looking for the conveniences of home—a chain drugstore, a carwash (some roads are very dusty), Starbucks—you might as well stay home. In fact, the island’s laws heavily restrict the kinds of commercial establishments that can open.

You start to come to terms with the island’s remoteness when you map your trip (figure four to six hours from New York, depending on mode of transportation). No direct commercial flights run from the New York area, and only two regional airlines provide scheduled service, from Westerly and Providence, R.I. While there’s boat service from a few places on the East Coast, only ferries from Rhode Island’s Point Judith allow cars, and they hold only 32 to 38 of them. Spots are reserved months ahead.

You don’t have to have a car to traverse the island, which measures 7 miles at its longest, 3 miles at its widest. You can easily make your way to most of the key spots by foot, or you can bring or rent a bike or moped. Arguably, Block Island is meant to be experienced by walking if for no other reason than because that’s the surest way to pick up the island’s signature scent, the same mix of salt air and bayberry I remember from my childhood.

Still, I’ll make a case for renting a car because there are some incredibly tranquil parts of the island, particularly the beaches on the west side, that are slightly beyond walking range. Besides, I like to fish—especially on a moment’s notice. With my trusty minivan, I can make it to my preferred surf-casting spot (which shall go nameless) in less than 10 minutes.

The Southeast Lighthouse, an 1870s Gothic Revival structure Photo: Katherine Wolkoff for The Wall Street Journal

You can cover most of the island’s major “sights” in a day or two. Begin with the Old Harbor area, where the ferries land and many of the hotels are clustered. The harbor itself was essentially established in the 1870s, when federal funds paid for two rugged breakwaters still standing today.

As commercial seaside districts go, this one’s about as quaint as they get. The buildings—mostly one- to three-story wooden structures dating from the 19th century—largely appear unaltered since my last visit in the ’80s. I lost count of how many times I told my family, “This is where I used to go to” buy candy, popcorn—you name it. In several instances, it seemed only the person manning the counter had changed.

Old Harbor is also where you’re likely to eat. Block Island isn’t a foodie’s paradise, just a place where you can enjoy a good meal in an unpretentious setting. For breakfast, try Ernie’s, a fixture since the ’60s. For lunch or dinner, check out the century-old Ballard’s Inn, which offers lobster 13 ways, from steamed to stuffed to fire-grilled. In between, stop at one of the snack shops for an order of clam cakes or ice cream.

There’s a New Harbor, too, on the body of water known as the Great Salt Pond, which was a freshwater pond until a channel was cut in the 1890s to allow for more boat traffic to and from the island. You’ll find three marinas here, and though New Harbor attracts the bigger boats (and the bigger celebrities—I remember seeing country star-turned-sausage king Jimmy Dean arrive with his yacht when I was a kid), it doesn’t have Old Harbor’s charm or as many commercial establishments.

Beyond the harbors are the Mohegan Bluffs, situated on the southern end and named after a Native American tribe that tried to seize the island from a long-established tribe. This is arguably the best view on the Block: clay cliffs rising 185 feet above a beach that looks as if surfers might head here if they were surfing, say, on Mars. Rocks of all sizes (small, big, ginormous) occupy a good chunk of the real estate, and the brownish sand seems added as an afterthought. The Bluffs are right next to the Southeast Lighthouse, an architecturally significant fixture since its 19th-century raising, with its brick facade and 67-foot tower that seems taller because of the low height of most of the island’s structures.

Illustration: Michael Hoeweler

And then there’s the island’s northern end—specifically, the mile-long, lonely stretch of beach that begins at Settler’s Rock, where 16 families of European colonists who originally settled Block Island established themselves in 1661, seeking religious freedom and the sense of independence that defined island living then as now. The northern stretch continues to the very tip, known as Sandy Point, where stands the aptly named North Light, an impressive granite lighthouse that dates to 1867.

Beyond that, the island is pretty much what you make of it. One day, we went clamming (a license from the harbor master runs $20 a week per person). A fairly secluded cove in the Great Salt Pond yielded more than enough tasty quahogs to make a big pot of chowder. (The rest of the provisions came from the island’s only year-round grocery store, though some produce can be found at the twice-a-week farmers’ market.)

Another day, we took a short hike on the north side, ending at the freshwater Middle Pond, which was mercifully nowhere near as cold as the ocean, which stayed in the mid 60s during our visit. The walk also offered views of greener-than-green fields of windblown grasses, coastal shrubs and the occasional butterfly. More than anything, the walk reminded me it’s still not hard to find a quiet place on the island. After one other swimmer left, we had the half-mile-long pond to ourselves.

Yet another day, we visited the Block Island Historical Society. In the late 19th and early 20th centuries, the island was a fashionable New England retreat. Famous visitors included President Ulysses S. Grant and Amelia Earhart. But by the early ’60s, it had fallen into a state of near-abandonment; storms and economic factors (particularly the Great Depression) are said to have been a factor. My father recalls that on his first visit to the Block, just he, my mother and one other guest occupied the vast Oceanview Hotel, an island landmark that burned down in 1966.

If the Oceanview were still around, it probably wouldn’t have a room to rent. The island’s summer population can swell to 20,000 on a busy day, about twice what it was a decade ago, according to the Block Island Tourism Council. (The winter population has held at around 1,000 for several years.) Locals ascribe some of this growth to the fact that the Nature Conservancy, a global conservation group, identified the island as one of the “last great places” on the planet. Tourists come to see a place that’s untouched, but in so doing, arguably make it a little less so.

“ I lost count how often I told my family, ‘This is where I used to go to...’ ”

Then again, through a group called the Block Island Conservancy, islanders have seen to it that nearly 45% of the land here is set aside for preservation. Locals are also very mindful of maintaining the island’s quaint character, so it’s hard to say if a certain coffeehouse chain, for example, would even be welcome.

I’m no fan of Starbucks, so I didn’t miss it. Besides, the place where we stayed, the comfy Island Manor Resort—one of two timeshares on the island—provided free joe. But there were times, I admit, the island was a little too crowded for my taste. I don’t recall quite the same party scene at Old Harbor when I was a kid. And on the ferry ride home, always the most bittersweet part of a trip to the Block, the boat was so jammed I had to stand for the 55-minute trip.

As for the Spring House deer, its disappearance bothered me for a minute, then I realized that the hotel, situated high on a hill, has otherwise stayed much the same, even if it’s pricier. So I stopped my muttering, took in the remarkable island vista from the sloping lawn and enjoyed a fine cocktail at the bar on the porch—not exactly something I could do on those childhood visits decades ago.

The Lowdown // Knocking Around Block Island, R.I.

Getting There: Ferries depart from New London, Conn. (goblockisland.com); Montauk, N.Y. (vikingfleet.com); Fall River, Mass., Newport, R.I., and Narragansett (Point Judith, the only car ferry), R.I. (all three through blockislandferry.com). New England Airlines flies from Westerly, R.I. (block-island.com) and Cape Air (capeair.com) from Providence, R.I.

Staying There: The grande dame is Spring House (springhousehotel.com); $125 to $750 per night, depending on time of year. The Gables (gables-innbi.com) has simple, clean lodgings from $165 in summer.

Eating There: Ernie’s (212 Water Street) serves a fast, unpretentious breakfast. The morning buffet at the 1661 Inn (blockislandresorts.com) is a feast, with bluefish and baked beans. For dinner in a beach setting, Ballard’s Inn (ballardsinn.com) prides itself on its “Lobster 13 Ways.”

Being There: Begin with the Visitor Center, where ferries dock. Some outfits rent cars, but for two-wheeled travel, try Island Moped and Bikes ($30 a day for bikes, $115 for mopeds, bimopeds.com). A key hiking companion is “On This Island: The Block Island Trail and Nature Guide,” available at Island Bound Bookstore (islandboundbookstore.com) or Amazon.com.