By analogy to transistors in classical electronic circuits, quantum optical switches are important elements of quantum circuits and quantum networks1,2,3. Operated at the fundamental limit where a single quantum of light or matter controls another field or material system4, such a switch may enable applications such as long-distance quantum communication5, distributed quantum information processing2 and metrology6, and the exploration of novel quantum states of matter7. Here, by strongly coupling a photon to a single atom trapped in the near field of a nanoscale photonic crystal cavity, we realize a system in which a single atom switches the phase of a photon and a single photon modifies the atom’s phase. We experimentally demonstrate an atom-induced optical phase shift8 that is nonlinear at the two-photon level9, a photon number router that separates individual photons and photon pairs into different output modes10, and a single-photon switch in which a single ‘gate’ photon controls the propagation of a subsequent probe field11,12. These techniques pave the way to integrated quantum nanophotonic networks involving multiple atomic nodes connected by guided light.