The rest of the buildings came naturally, if gradually. The idea of having a slew of small houses for different activities, moods and seasons, complemented by decorative “follies,” was Johnson’s conception for the site from early on. He called it a “diary of an eccentric architect,” but it was also a sketchbook, an homage to architects past and present, and to friends like the dance impresario Lincoln Kirstein, after whom Johnson named one of the follies he built on the property, a 30­-foot-high tower made of painted concrete blocks.

In contrast to their whirlwind weekday world in Manhattan, Johnson and Whitney saw life in New Canaan as perpetual camping, albeit of a luxurious, minimalist sort. Neither Grainger nor the 380-square-foot Library has a bathroom, though both are air­-conditioned, unlike the Glass House, which relies on cross ventilation. It originally had heating pipes in the ceiling and the floor, but the ceiling pipes reportedly froze early on and were never adequately repaired. To compensate, on particularly cold winter days the temperature of the water flowing through the radiant heated floors was turned up to nearly 200 degrees. “You couldn’t go in there with bare feet,” Port Draper, the contractor who maintained the house for many years, recalled in The Times in 2007. Johnson was unbothered by the house’s leaks, a problem endemic to a flat roof. Frank Lloyd Wright once referred to one of his houses as a “two-bucket house,” according to Robert A. M. Stern, to which Johnson gaily replied, “Oh, that’s nothing, Frank. Mine’s a four-bucket house. One in each corner.”

While the Glass House was designed with areas for dining, living and sleeping, loosely divided by low cabinetry and a brick cylinder holding the chimney and bathroom, it functioned more as a living space, an occasional office for Johnson and a place to throw parties (lots of them, attended in the early years by a coterie of young Yale architecture students, and later by the likes of Richard Meier, Frank Gehry, Fran Lebowitz and Agnes Gund). The house was astonishingly tchotchke­-free. “I don’t think clutter was allowed,” the painter Jasper Johns, a friend of both men, once said. “One was always aware of their ruthless elegance.”

The Brick House was originally divided into three rooms ­­— with real walls ­­— each with a porthole window. It was intended for guests, but Johnson soon realized the problem with a guest house is you wind up with overnight guests; besides, the sun blasted into the Glass House at sunrise. So in 1953, he remodeled the Brick House, making his first break with modernism by creating a luxurious, cocoon­like master bedroom, with a vaulted ceiling and Fortuny-­covered walls and plush carpet, as well as a reading room with floor-to-­ceiling bookshelves. The bedroom’s slender columns, and the Pavilion he designed for the pond in 1962, both read as trial runs for the thin, white, shapely architecture of the New York State Theater at Lincoln Center. Johnson’s homosexuality was an open secret for decades, and some historians have interpreted the Brick House interior as camp, and its exterior as a closet. Once asked whether the Glass House was exhibitionist, Johnson observed, tartly, that as it had no basement, he didn’t have to confine his sex life to below grade. Given the privacy the property affords, and all those non­glass houses, he hardly needed to.

For Johnson, the landscape ­­— both viewed from inside the Glass House and as he walked briskly from one building to another — was as important as the structures. There are only two pieces of art from Johnson’s collection in the Glass House: an Elie Nadelman sculpture and a landscape from the school of the 17th­-century artist Nicolas Poussin, on a stand in the middle of the space. The painting was a great inspiration to Johnson — he removed and trimmed trees on the area around the house to create more orderly views, allowed some grasses to grow tall while others were close­-cropped. Richard Kelly, who did the moody lighting for Johnson and Mies van der Rohe’s Four Seasons Restaurant in the Seagram Building, collaborated with Johnson on lighting the area around the house, ensuring that at night, the outside would come alive.

THE SPARENESS OF the Glass House makes it hard to think about messy things like eating, but even the wiry Johnson needed sustenance: Whitney, an accomplished amateur chef 33 years younger than Johnson, cooked for them at times in the tiny Glass House kitchen. After Johnson bought Calluna Farms for Whitney in 1981, the couple’s domestic life expanded to the southern edge of the property. Whitney got rid of the suburban pool and tennis court, reduced five bedrooms to two and kept only one bathroom.

Calluna’s kitchen had no dishwasher, and only a modest refrigerator, alongside Whitney’s impressive collection of copper cookware and dozens of well-worn cookbooks. Whitney made a famous fish chowder luncheon every year when his peony and iris garden burst into bloom, and he was as exacting in his housewares as he was in his art and design collections (his estate was sold at Sotheby’s in 2006 to add to the endowment for the property).