Although many neuroscientists are trying to figure out how the brain works, Mark Changizi is bent on determining why it works that way. In the past, the assistant professor of cognitive science at Rensselaer Polytechnic Institute has demonstrated that the shapes of letters in 100 writing systems reflect common ones seen in nature: Take the letter "A"—it looks like a mountain, he says. And "Y" might remind one of a tree with branches. He also showed that across different languages most characters take three strokes to write out. That's because, he says, three is the highest quantity a person's brain can perceive without resorting to counting. But Changizi's theories aren't limited to writing. He also believes that primates developed the ability to see in color so that they could figure out if peers were sending emotional cues. He hatched that theory by comparing the light wavelengths given off by the facial skin of someone blushing to that of a person not flushed. The prolific Changizi recently published two papers: one that sets out to explain how our lexical systems evolved and another that suggests how the brain's visual system is adapted to anticipate the future a fraction of a second before we actually see it. (See related slideshow here.) Changizi spoke to ScientificAmerican.com about his newest research; what his forthcoming book, The Vision R(evolution): How the Latest Research Overturns Everything We Thought We Knew About Human Vision, has to do with superheroes; and what kind of scientist he is.

What's the goal of your research?

My goal is to understand the principles underlying the design of the brain or visual system or cultural artifact, like language or writing systems. I'm not as interested in the mechanisms per se. People like me make the point that you can't even study those mechanisms without having an idea what those mechanisms are trying to compute. So you have to have some opinion about what the design or function of those mechanisms are for to even do that. So, I am focusing on the function from a teleological [purposive] point of view. Of course it's unpacked with natural selection or cultural evolution.

Are you characterizing the functions of certain systems, so that other researchers can work on how a system performs its tasks?

It's certainly a consequence of my work that someone else will be in a better position to pose mechanisms when they know the big constraint of: "What is it that my mechanisms need to be computing?" But, that's not why I do it. I'm excited about the selection pressures undergoing why we see in color: What is color for? What is it optimized for? Only 1 percent of me is interested in the fact that it's implemented in the particular way it's implemented in some part of the brain. ... [My work] often makes some predictions about specific aspects of the mechanisms, but once that information is there, there could be infinitely many mechanisms that could carry out that function.

One of your more recent papers deals with the Oxford English Dictionary as an economically organized collection of the words in the English language? What is it about its organization that makes it so optimal?

If you gave definitions of all the words on the basis of some small set of atomic words, then you would have two levels of words: the bottom level, [a] small set of atomic words (between 10 and 50) and the other, roughly 100,000. That would be a very costly dictionary in terms of the size that's required. The signature of an optimally organized lexicon is: you instead take that small set of words and you use them to find a slightly larger set of slightly more complicated words, which are in turn used to build a slightly larger set of still more complicated words and so on. When you do that seven times, or so, that will then allow you to utilize the minimum amount of definition space to find the target words that you are really interested in defining in the first place.

What type of words serve as the bottom rung, or atomic words, in the lexicon?

The ones that come out from WordNet [a lexical database of the English language developed at Princeton that ranks words from the most basic to the most complex] are words like: abstraction, act, entity, event, group, phenomenon, possession, etcetera.

How does this manner of organization reflect a mechanism in our brain?

My interpretation of this result is that culture has over time evolved the meanings of the words in our lexicon so as to minimize the total size of definitions. And the reason that was selected for was because that way we could all fit more words in the head and have a richer vocabulary.

Does that imply an underlying drive toward efficiency or conciseness?

Sometimes when you speak about evolution, you mistakenly say that evolution is striving for developing a wing. But, it's blind cultural evolution. Over time, meanings of words are going to change. The structure of the lexicon is passed on, generation to generation, there'll be selection pressure changing it in certain ways. Sometimes it will change in ways that are hurtful, making it harder for people to remember. Those will tend to change back over time. So, it's blind cultural selection with no directionality per se.

If the dictionary study involves cultural evolution as its driver, then the new work on the visual system involves natural selection–based evolution. Why is it that we need to "perceive the present," as you put it, or see into the future?

Animals who move or are in a world that moves around them—as long as there are things moving somehow relative to you—will be selected to have perceptions that are true. We have about a tenth of a second delay between the time light hits the retina and the time of resultant perception, which is considerable given that you move 10 centimeters [four inches] in that amount of time even if you're only walking one meter [3.3 feet] per second. That means that if you didn't compensate for this neural delay, anything you perceive to be within 10 centimeters of passing…. [It] would have just passed you by the time you perceive it. You'd always be seeing the world as it was a tenth of a second earlier and seeing what the world looks like 10 centimeters behind where you in fact are--if you hadn't run into whatever it is you're looking for.

So, in the new work, you detail various optical illusions. (See related slideshow here.) Do these illusions result from all the errors in our visual system that need to be compensated for?

In this work, it's ones dealing only with forward motion, which is, I think, one of the main kinds of motion that we're good at dealing with. Even when you're standing still or rotating, for example, that's going to be a different kind of optic flow. Potentially, we're able to correct for those, too. But, all of these illusions turn out to be explainable from forward motion correlates. We're doing compensation all over the place. We play video games where there are made-up rules of optical flow that our visual systems can figure out on the fly.

In the new work, you were able to sort optical illusions into categories based on four visual features that were being misperceived. What particular features are those?

There are four different domains of misperception: The first is illusions of size. The second is illusions of speed. The third is luminance, or contrast. The last is illusions of perceived distance. Now, there are different ways of affecting those kinds of misperceptions—the key features that are causing those illusions. For example, size differences within your visual field could cause misperceptions or illusions of speed.

So, does this work fit in with your forthcoming book, The Vision R(evolution)?

The book is about four stories about four of the big areas of vision: The first is motion, which is this perceiving the present stuff; binocular vision, which concerns the evolution of forward-facing eyes; color and luminance, which is like the skin work; and object recognition—this is a little bit more of a stretch—but it connects to the evolution of writing and reading. The four areas all have an evolutionary side to them. Furthermore, they all have a superhero angle to them. You can describe the perceiving the present stuff as future-seeing. People have proposed superheroes that see the future and, in a weak sense, we do, too. For the evolution of forward-facing eyes, I am arguing that it is for a kind of x-ray vision. It actually allows us to see through stuff—like when you hold up a finger vertically and you see through it instead of beyond it. For animals that are large and living in forested environments, there should be selection pressure for forward-facing eyes, because you can actually see more of your environment. For color vision, the cones that we have in our eyes—that [other] mammals don't—are evolved to see the oxygenation modulations in the blood, because we want to sense the emotions in others. We really have external-sensing equipment that…[is]…empathic in nature—mind reading and emotion-reading, like the annoying character in Star Trek, the empath. A bit of a stretch of the theme is spirit-reading, our ability to read the thoughts of the dead. Object recognition (reading and writing) has allowed us to read the thoughts of the dead. So, it's four different stories connected by these kinds of themes.



Interesting. So, putting all this together, what do you consider your field? Is it cognitive science?

I would call it theoretical neurobiology in vision. But, it doesn't get at the fact that I am more evolutionary-directed, rather than computational modeling-directed—so I think evolutionary, theoretical neurobiologist would be slightly more representative.