Dissecting the brain was a messy affair in the early 16th century when cadavers might spend a while decomposing before finally going under the physician's knife. It was a predicament that Leonardo da Vinci circumvented with characteristic finesse: when he wanted to study deep cavities in the brain, he took the organ from a freshly killed ox and injected it with melted wax, taking care to make a hole at one end for fluid to escape. When the wax had cooled and hardened, he carved away at the brain tissue to reveal an exquisite, life-sized cast of the organ's inner structure.

Da Vinci's casts became the basis for the Italian polymath's anatomical sketches in which he set out to document the appearance and even the workings of the brain. There was a lot to unravel and little to build on. One popular theory doing the rounds at the time held that animal spirits coursed through the human brain and crossed internal cavities by way of tubular and presumably hollow nerves.

The long history of the brain's depiction, from the first raw sketches of antiquity, through early electroencephalograph (EEG) recordings, to the abstract art of modern-day scanners is charted in the newly published Portraits of the Mind – from which a selection of images are shown here – by Carl Schoonover, a doctoral student in neurobiology at Columbia University in New York. This is a journey that has no end, but one that reveals deeper intricacies with every step. Despite the great discoveries that underpin modern neuroscience, the brain remains the most complex and mysterious object known.

The oldest drawing of the nervous system is traced to Cairo circa 1027, when Ibn al-Haytham sketched a nose and two eyes and ran hollow nerves from the latter to the brain. Simplistic it may be, but al-Haytham's depiction captured an essential element of neuroscience: that we observe the world through sensory organs that feed information to the brain. Al-Haytham drew on anatomy learned from studying animals, as both Christian and Islamic worlds placed severe restrictions on human dissection. Physicians only got to grips with the human brain when these laws fell by the wayside. By the mid-17th century, English physician Thomas Willis and his accomplice, Christopher Wren, were drawing the brain in unprecedented detail, as a three-dimensional whole.

But it was Italian physician Camillo Golgi who surely deserves credit for the first major leap in teasing apart the stuff of the brain. Golgi was born in 1843 and developed the reazione nera, the "black reaction", which set in train research that continues today. With a simple mixture of potassium dichromate and silver nitrate, Golgi gave scientists the ability to stain – and so highlight – individual brain cells. Under a microscope, these darkened neurons became bold against featureless grey matter. The fine structure of the brain was emerging.

Golgi did not benefit most from his discovery. It took a contemporary in Spain, Santiago Ramon y Cajal, to grasp the real potential of Golgi's method. Through years of painstaking and skilled work, Cajal stained, isolated and characterised neurons by appearance and location. The work was divisive, not least for Cajal's relationship with the man whose technique he mastered. Golgi asserted that the brain was a continuous lump of matter, but Cajal proved otherwise. The brain, he said, was a collection of individual but interconnected neurons. Each had a thick soma at heart, from which grew long, thin protrusions called dendrites and axons. The dendrites behaved like receivers and listened for signals from neighbouring neurons, while the axons were transmitters that broadcast the neurons' own messages. Cajal put the neuron centre stage and so marked the birth of modern neuroscience.

By the early 20th century, drawings of specific brain regions, such as the neocortex and hippocampus, included fiendishly complicated neural circuits. They were awe-inspiring, but drawn from dead brain matter. What scientists lacked was a way to watch living neurons in action. That problem was overcome in the 1990s, when researchers transferred genes from a bioluminescent jellyfish into growing neurons. At a stroke, the world of neuroscience moved from monochrome to colour. Today, living brain cells can be made to fluoresce in a dazzle of colour, producing images called "brainbows". With them, scientists have mapped out the neural connections that govern our movement, sight and hearing.

Parallel developments in microscopy and electrophysiology have unveiled more of the structure of brain cells and allowed scientists to record the activity of single neurons. Meanwhile, whole brain scanning techniques, such as positron emission tomography (PET) and functional magnetic resonance imaging (MRI) are beginning to reveal, with some caution, how regions of the brain help us plan, remember and respond to the world around us. One thousand years after al-Haytham sketched the neural wiring for eyesight, the brain is rendered in rich colour as 3D computer images that can be rotated, flipped and peered inside.

It is hard to leaf through Schoonover's book without marvelling at how our view of the brain has been transformed. Still the organ remains a mystery. How do neurons give rise to conscious experience? What form does a memory take? We may not know for another thousand years. Neuroscience is more than the study of the brain. It is an unprecedented opportunity to understand ourselves.