http://graphics.cs.cmu.edu/projects/whatMakesParis/

What Makes Paris Look like Paris?

People

Abstract

Given a large repository of geotagged imagery, we seek to automatically find visual elements, e.g. windows, balconies, and street signs, that are most distinctive for a certain geo-spatial area, for example the city of Paris. This is a tremendously difficult task as the visual features distinguishing architectural elements of different places can be very subtle. In addition, we face a hard search problem: given all possible patches in all images, which of them are both frequently occurring and geographically informative? To address these issues, we propose to use a discriminative clustering approach able to take into account the weak geographic supervision. We show that geographically representative image elements can be discovered automatically from Google Street View imagery in a discriminative manner. We demonstrate that these elements are visually interpretable and perceptually geo-informative. The discovered visual elements can also support a variety of computational geography tasks, such as mapping architectural correspondences and influences within and across cities, finding representative elements at different geo-spatial scales, and geographically-informed image retrieval.

Paper & Presentation

SIGGRAPH Paper (preprint, pdf, 71MB)

Reduced-size SIGGRAPH Paper (preprint, pdf, 9MB)

SIGGRAPH talk slides (pptx, 238MB)



Citation



Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, and Alexei A. Efros. What Makes Paris Look like Paris? ACM Transactions on Graphics (SIGGRAPH 2012), August 2012, vol. 31, No. 3. [Show BibTex]



@article{doersch2012what, title = {What Makes Paris Look like Paris?}, author = {Carl Doersch and Saurabh Singh and Abhinav Gupta and Josef Sivic and Alexei A. Efros}, journal = {ACM Transactions on Graphics (SIGGRAPH)}, volume = {31}, number = {4}, pages = {101:1--101:9} year = {2012}, }

CACM Citation



Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, and Alexei A. Efros. What Makes Paris Look like Paris? ACM Transactions on Graphics (Communications of the ACM 2012), December 2015, vol. 59, No. 12. [Show BibTex]



article{doersch2015makes, title={What makes Paris look like Paris?}, author={Doersch, Carl and Singh, Saurabh and Gupta, Abhinav and Sivic, Josef and Efros, Alexei A}, journal={Communications of the ACM}, volume={58}, number={12}, pages={103--110}, year={2015}, publisher={ACM} }

Video

Related Papers

S. Singh, A. Gupta and A. A. Efros. 2012. Unsupervised discovery of mid-level discriminative patches. In ECCV 2012.

C. Doersch, A. Gupta and A. A. Efros. 2013. Mid-Level Visual Element Discovery as Discriminative Mode Seeking. In NIPS 2013.

Code

Additional Materials

Full Results: Paris Elements

An example of the results obtained entirely automatically, using our algorithm to find the unique visual elements of Paris. Each row shows the most confident detections in Paris for one of these elements. WARNING: This page contains about 5000 patches; don't click the link unless your browser can handle it!

Take the Paris-NonParis test!

This test presents 100 random street-level images: 50 from Paris, 50 from 11 other cities. Guess whether each one is from Paris (try not to look at the text...) and we'll score you at the end!

Artist Materials

For one experiment, we had an artist sketch an image, and then later re-sketch it with help from our discovered elements. See the resulting sketches, plus the materials we showed our artist.

Popular Press

Funding

This research is supported by: