Color perception is an ancient and active philosophical problem. It’s an instance of the wider category of sensory perception, but since the color spectrum fits on a single line (unlike, say, touch and taste), it has always been of particular interest. In her new book Outside Color, University of Pittsburgh professor M. Chirimuuta gives a serendipitously timed history of the puzzle of color in philosophy. To read the book as a layman feels like being let in on a shocking secret: Neither scientists nor philosophers know for sure what color is.

“Of all the properties that objects appear to have,” Chirimuuta writes, “color hovers uneasily between the subjective world of sensation and the objective world of fact.” The early history on the color perception debate alternated between partisans of these two camps. The Scholastic or Aristotelian model is a simple realism: Objects have colors that observers perceive in them. Like a seal that leaves a stamp in hot wax, an object’s color leaves its imprint temporarily on our eye. Since Scholastic realism presents no conflict between what we see and what there is, it was a convincing and long-lasting explanation. Color is what it looks like.

In the seventeenth century, the scientific revolution turned this common-sense explanation on its head. Galileo, that philosopher of inversions, suspected the realists had it exactly wrong. Colors, he wrote in Il Saggiatore (The Assayer), “hold their residence solely in the sensitive body; so that if the animal were removed, every such quality would be abolished and annihilated.” In other words, an unseen tree falls in black and white. If color exists in objects, then the scientific-minded must ask where. Unable to define and measure color the same way they located primary or physical qualities (shape, size, and speed), these early philosopher-scientists figured it had to be all in our heads.

Whether color is in our brains or in the world, realists and anti-realists agreed that there are right and wrong answers. Both models are frictionless, with perceiving subjects interacting with a real and objective reality. Chirimuuta calls them “detection models,” but they assume too much. If we imagine that every existing physical referent for something like The Dress—the Internet-famous garment that in February bewildered online America with a debate about whether a Scottish woman’s outfit was white-and-gold or blue-and-black—had been burned in a tragic warehouse fire, neither perspective could give a consistent answer as to its color without also imagining a normative viewer.

Outside Color is most engaging when it’s interrogating fundamental questions like these. “If we step back a moment,” Chirimuuta writes, “we can appreciate how very weird it is to even expect there to be a connection between the manifest visual world, brought to us by our senses, and the rarefied scientific image of a world made up of physical particles, etc.” To believe in the atom—never mind quantum theory or more extravagant scientific models—is to accept that our unaided perceptive faculties have very limited access to physical reality. Our perception gives us useful information about the world that allows us to make decisions as actors within it, but that doesn’t necessarily mean it’s true or consistent.