After decades of big yards and boxy homes, more homeowners are opting for a throwback to ancient days: the courtyard. Often enclosed on several or all sides, courtyard homes bring the yard inside, offering an extra layer of privacy and allowing natural light to flood into the house.

Jimmy and Margaret Lowder's 4,600-square-foot home in Montgomery,...