Ask the Scientists

Join The Discussion

What is the context of this research?

This study aims to create a literal picture from your visual imagination. Previous studies have successfully recovered mental representations for relatively simple categories such as letters or smiley faces. These reconstruction methods have considered images to be two-dimensional arrays of pixels. If one randomly chooses a darkness value for each pixel, then there are numLevels^nPixels possible images that can be created. For example, for even a 64x64 thumbnail-sized image with 255 possible gray levels, there are more possible images that can be created than atoms in the Universe! By contrast, we consider a scene to be represented by extended surfaces rather than pixels. This both reduces the dimensionality of of problem while creating more realistic looking images.

What is the significance of this project?

A fundamental goal for understanding human vision is to identify the mapping between image features and subsequent categorization. Our study will help us understand how personal experiences influence the mental images that we create when we think about a category such as “street”. Do different people extract the same features (common prototype), or do our visual experiences create a unique “templates”? This study will have implications for basic science, as well as potential for applications. In addition to providing critical insight into how we form complex visual categories, the technology that we are developing in this project can be used for enhancing the abilities of police sketch artists, and in architecture and design choices to make for a more legible and memorable world.

What are the goals of the project?

Use features of Convolutional Neural Networks to increase detail in our images.

Test the hypothesis that scene category representations are unique to observers by checking the extent to which reconstructions resemble scenes from three kinds of individual experience: Early visual experience: the city where the individual grew up. Current experience: city where the individual lives. Added experience: Average of the scenes from all locations lived



Altogether, these experiments will provide critical insight into how we conceptualize complex scenes, laying the foundations for understanding how visual information is flexibly represented for recognition.