The Scarr-Rowe hypothesis predicts that the heritability of cognitive abilities is higher in more privileged socioeconomic conditions, meaning that genetic potential can be more fully expressed in environments characterized by high socioeconomic status (SES) compared to low SES. This gene × SES interaction, however, has been replicated mostly in the United States, but not in other Western nations like the United Kingdom. In the current study, we tested the interaction between childhood SES and the heritability of cognitive ability in 3074 German twin pairs comprising three age cohorts at different developmental stages (mean ages of 11, 17, and 23 years). Higher SES was associated with significantly higher mean cognitive ability scores in the two younger cohorts, with reduced variances at higher SES levels. Results further support the Scarr-Rowe hypothesis in middle childhood, and to some degree in adolescence, but not in adulthood. This indicates that the role of family SES as a moderator of the heritability of cognitive ability changes as children grow older. Moreover, children's shared experiences appear to be explain more variance in cognitive ability at the lower end of the SES distribution in middle childhood and adolescence.