There exists a moderate correlation between MRI-measured brain size and the general factor of IQ performance (g), but the question of whether the association reflects a theoretically important causal relationship or spurious confounding remains somewhat open. Previous small studies (n 〈100) looking for the persistence of this correlation within families failed to find a tendency for the sibling with the larger brain to obtain a higher test score. We studied the within-family relationship between brain volume and intelligence in the much larger sample provided by the Human Connectome Project (n = 1022) and found a highly significant correlation (disattenuated ρ = 0.18, p < .001). We replicated this result in the Minnesota Center for Twin and Family Research (n = 2698), finding a highly significant within-family correlation between head circumference and intelligence (disattenuated ρ = 0.19, p < .001). We also employed novel methods of causal inference relying on summary statistics from genome-wide association studies (GWAS) of head size (n ≈ 10,000) and measures of cognition (257,000 < n < 767,000). Using bivariate LD Score regression, we found a genetic correlation between intracranial volume (ICV) and years of education (EduYears) of 0.41 (p < .001). Using the Latent Causal Variable method, we found a genetic causality proportion of 0.72 (p < .001); thus the genetic correlation arises from an asymmetric pattern, extending to sub-significant loci, of genetic variants associated with ICV also being associated with EduYears but many genetic variants associated with EduYears not being associated with ICV. This is the pattern of genetic results expected from a causal effect of brain size on intelligence. These findings give reason to take up the hypothesis that the dramatic increase in brain volume over the course of human evolution has been the result of natural selection favoring general intelligence.