Neuromorphic computing was originally referred to as the hardware that mimics neuro-biological architectures to implement models of neural systems. The concept was then extended to the computing systems that can run bio-inspired computing models, e.g., neural networks and deep learning networks. In recent years, the rapid growth of cognitive applications and the limited processing capability of conventional von Neumann architecture on these applications motivated worldwide research on neuromorphic computing systems. In this paper, we review the evolution of neuromorphic computing technique in both computing model and hardware implementation from a historical perspective. Various implementation methods and practices are also discussed. Finally, we present some emerging technologies that may potentially change the landscape of neuromorphic computing in the future, e.g., new devices and interdisciplinary computing architectures.