A wealthy 19th century sea captain built this sturdy, symmetrical house in the town of Eastham, just above the elbow of Cape Cod. The Cape isn’t very wide at Eastham, which meant the Captain only had to walk a mile or so in either direction to reach the water. It was a pretty short journey—a stark contrast to those he was best known for, setting out for years at a time to chase whales.

By 1868 Captain Edward Penniman had become a successful enough whaler to build himself this fancy French Empire-style house on land he bought from his father. The house is kind of stocky and square, ornate and charming in equal measure. It’s not too tall—two and half stories with a cupola (sort of a widow’s walk with windows)—set starkly on a small hill. At the drive entrance is the first tip-off to Penniman’s profession: a gate, made from a whale’s jaw, leading to the back door and barn.

A native of Eastham, Penniman grew up with the sea. In 1852, at the age of 21, he caught the whaling bug and traveled the 60-odd miles to New Bedford (the whaling capital of the world at the time) to seek his fortune.

It was his first long voyage, but was only the beginning for Penniman. He helmed at least seven major whaling voyages as captain, each lasting up to three or four years. The trips were so long, whenever possible he brought along his wife (a navigator and deck hand in her own right) and any combination of his three kids he could coax into joining them.

The house stayed in the Penniman family until 1963. For the relatively small price of $28,000 (about $220,000 today), the captain’s youngest granddaughter sold it to the Cape Cod National Seashore to look after, and to maintain the family archives and whaling artifacts. Since 1976 it’s been on the National Register of Historic Places, and is run by the National Park Service as a museum, whale bones included.

Update as of August 2020: The whale arch has been removed for preservation.