Locus coeruleus (LC) tau accumulation begins early. Targeting LC (dys)function might improve early identification for Alzheimer's disease (AD) risk. Pupillary responses during cognitive tasks are driven by the LC and index cognitive effort. Despite equivalent task performance, adults with mild cognitive impairment have greater pupil dilation/effort during digit span than cognitively normal (CN) individuals. We hypothesized that AD polygenic risk scores (AD-PRSs) would be associated with pupillary responses in middle-aged CN adults. Pupillary responses during digit span tasks were heritable (h2 = 0.30–0.36) in 1119 men aged 56–66 years. In a CN subset—all with comparable span capacities (n = 539)—higher AD-PRSs were associated with greater pupil dilation/effort in a high (9-digit) cognitive load condition (Cohen's d = 0.36 for upper vs. lower quartile of AD-PRS distribution). Results held up after controlling for APOE genotype. Results support pupillary response—and by inference, LC dysfunction—as a genetically mediated biomarker of early mild cognitive impairment/AD risk. In combination with other biomarkers, task-evoked pupillary responses may provide additional information for early screening of genetically at-risk individuals even before cognitive declines.