The causes and timing of tropical glacier fluctuations during the Holocene epoch (10,000 years ago to present) are poorly understood. Yet constraining their sensitivity to changes in climate1 is important, as these glaciers are both sensitive indicators of climate change and serve as water reservoirs for highland regions2. Studies have so far documented extra-tropical glacier fluctuations3,4, but in the tropics, glacier–climate relationships are insufficiently understood. Here we present a 10Be chronology for the past 11,000 years (11 kyr), using 57 moraines from the Bolivian Telata glacier (in the Cordillera Real mountain range). This chronology indicates that Telata glacier retreated irregularly. A rapid and strong melting from the maximum extent occurred from 10.8 ± 0.9 to 8.5 ± 0.4 kyr ago, followed by a slower retreat until the Little Ice Age, about 200 years ago. A dramatic increase in the rate of retreat occurred over the twentieth century. A glacier–climate model indicates that, relative to modern climate, annual mean temperature for the Telata glacier region was −3.3 ± 0.8 °C cooler at 11 kyr ago and remained −2.1 ± 0.8 °C cooler until the end of the Little Ice Age. We suggest that long-term warming of the eastern tropical Pacific and increased atmospheric temperature in response to enhanced austral summer insolation were the main drivers for the long-term Holocene retreat of glaciers in the southern tropics.