The vanishingly small response of matter to light above ultraviolet frequencies makes the manipulation of light emission at such frequencies challenging. As a result, state-of-the-art sources of high-frequency light are typically active, relying on strong external electromagnetic fields. Here, we present a fundamental mechanism of light emission that is fully passive, relying instead on vacuum fluctuations near nanophotonic structures. This mechanism can be used to generate light at any frequency, including high-frequency radiation such as X-rays. The proposed mechanism is equivalent to a quantum optical two-photon process, in which a free electron spontaneously emits a low-energy polariton and a high-energy photon simultaneously. Although two-photon processes are nominally weak, we find that the resulting X-ray radiation can be substantial. The strength of this process is related to the strong Casimir–Polder forces that atoms experience in the nanometre vicinity of materials, with the essential difference being that the fluctuating force here acts on a free electron, rather than a neutral, polarizable atom. The light emission can be shaped by controlling the nanophotonic geometry or the underlying material electromagnetic response at optical or infrared frequencies. Our results reveal ways of applying the tools of nanophotonics even at frequencies where materials have an insubstantial electromagnetic response. The process we study, when scaled up, may also enable new concepts for compact and tunable X-ray radiation.