Death tolls and economic losses from natural hazards continue to rise in many parts of the world. With the aim to reduce future impacts from natural disasters it is crucial to understand the variability in space and time of the vulnerability of people and economic assets. In this paper we quantified the temporal dynamics of socio-economic vulnerability, expressed as fatalities over exposed population and losses over exposed GDP, to climate-related hazards between 1980 and 2016. Using a global, spatially explicit framework that integrates population and economic dynamics with one of the most complete natural disaster loss databases we quantified mortality and loss rates across income levels and analyzed their relationship with wealth. Results show a clear decreasing trend in both human and economic vulnerability, with global average mortality and economic loss rates that have dropped by 6.5 and nearly 5 times, respectively, from 1980–1989 to 2007–2016. We further show a clear negative relation between vulnerability and wealth, which is strongest at the lowest income levels. This has led to a convergence in vulnerability between higher and lower income countries. Yet, there is still a considerable climate hazard vulnerability gap between poorer and richer countries.