What does breast cancer screening have to do with fashion? More than rubber bracelets or ribbon broaches.

Adrian Rosebrock, from Catonsville, Maryland, has put into action insights from his day job as a developer at the National Cancer Institute unit to make a visual search engine.

Working in the breast cancer screening unit, Rosebrock has been developing metrics to detect cancer in images and taking those learnings about computer ‘vision’, namely histology, and applied it to the problem of shape and color in visual search in the fashion vertical.

His project, Chic Engine, matches the shape and color of any image led query you input, either via a image file upload or a hosted image URL – provided you are looking for clothing matches. Currently the index of returned products comes mainly from ShopStyle but what is available so far is an impressive demonstration of how visual search could be something to look out for.

Barriers to Entry

Personally speaking, I have been skeptical towards the possibilities of visual search for awhile now because, like we once said of mobile, it’s one of those technologies that seems to stall just before really “taking off” with users.

The main barriers to image search is that in general, it’s just not that intuitive to users. Uploading an image to search for similar matches is not only more effort than typing keywords in a query box, it is also something that the majority of web users are not even able to do.

Although, dear reader, it may be a simple enough thing for you to do, the average web user is limited by the ability to understand how their own computer works, let alone the web. Furthermore, the ostensibly easier feature for power searchers –namely to search against the URL of a hosted image – is near impossible for most people to get their head around without a rather technical demonstration.

With that in mind, when Rosebrock pitched SEW the story, I asked him what his fashion search engine could do with data from the latest dazzling debutant of social media, Pinterest. The next day I got a reply. Chic Engine could now match products pinned on Pinterest.

The Challenges of Computer Vision

In a phone conversation, Rosebrock told Search Engine Watch that there was an interesting technology challenge at the heart of image search – namely what it is possible for a computer to see and what is scaleable for a computer to return. The more accuracy required to find matches the lesser the ability of the search engine to find them.

As Superfish also found, the sweet spot seems to be somewhere between discovering what is just identical and what is actually similar and, just like Superfish, and any “computer vision” technology, Chic Engine has struggled with ‘background noise’ and has to focus on the foreground of the image.

For Rosebrock, the solution comes down to three tenets for designing a good visual search engine:

Accuracy

Purpose

Scalability

For visual search to be accurate, the user expects to find the image they are looking for in the database. Nothing new here.

However, for visual search to be useful, it has to go beyond finding identical matches and expand the users visual vocabulary – and the kicker here is that every new result is possibly a new query. Now, that is pretty interesting.

Imagine a search engine that enables you to delve into an infinite chain of queries to target exactly what you want and then think about how different that is to the incumbent offerings.

Unlike Superfish, Chic Engine runs a much more limited results set, which solves the scaleability problem such that it does not need any “special sauce” to identify the difference between a flat washer and an engagement ring. However, that doesn’t make it any less impressive as what Rosebrock seems to have solved is the problem of even wanting to do a visual search query in the first place.

Snap & Grab

Just how easy Chic Engine is to use should be a win to users with fashion sense. What makes it stand out is the ability to select a certain part of the image and search only on that portion of the image.

Particularly impressive is Chic Engine’s ability to match subtle color tones and hues. The image cropping mechanism also helps to reduce the problem of background noise as the user selection serves as a “hint” as to where the clothing is – Chic Engine then automatically detects the clothing match.

Just like the majority of web users cannot find an URL, let alone copy/paste it, so the majority of people are not blessed with the ability to truly articulate the style, cut, texture and color of a piece of clothing they like.

For most people, the terms “opal”, “chiffon” and “pencil” are not instinctive descriptions of color, texture and shape. Instinct just takes over and one hits the like, +1 or re-pin button in the hope that one’s inspiration is at the very least ‘registered’ and bookmarked.

Now with Chic Engine one could actually take an action after that flash of inspiration, bringing that perfect outfit a little closer to you than it was before.

Making things even easier, Chic Engine has an iPhone app which has notched up over 400 downloads. This means that theoretically I could now take a photo of a fashion shoot in a magazine and zero in on the jacket to find similar items which are probably at a lower price. Even better, the ability for me to take a photo of what some crazy-cool japanese kid is actually wearing (they always have the coolest clothes IMHO) on the streets of Harajuku, Tokyo, and the possibilities to create new and more interesting ways to sell clothes online seems endless.

Comparison shopping mobile apps like Goodzer demonstrate the value of in-store comparison shopping based on the incentive to get a better price for a product you know is in stock somewhere else.

But in-street comparison shopping that can use a passer-by as ‘the query‘? That’s a new one on me.

Style Over Substance

As a man whose fashion sense owes everything to the pity (and subsequent kindness) of his sister, I found myself wanting to use Chic Engine as not just my default search engine for clothes but also my go to destination for fashion inspiration – like suddenly “a new look” seemed possible.

What I realized was that when I am in a brick-and-mortar store buying clothes, I think in categories, with robotic thoughts such as “I need some pants. I need some socks. I need a shirt,” but when I look at an image in a cool magazine or whatever, I don’t really think much beyond “I can’t afford that” and “I doubt I could find anything similar.” Namely, my thoughts tend towards identical matches or just impressions of an overall style – to which the latter seemed ultimately unsearchable and the former is so brain dead obvious that I probably bought the product without even considering it.

To me, that is exactly why I think Chic Engine has real assets to strut. As with Superfish, similar matches are more interesting than identical matches.

Chic Engine is currently in beta with zero investment except for Rosebrock’s own sweat equity. However, with Pinterest and Shopcade proving that images alone can sell products which makes real money via affiliate schemes, he seems to be onto a good thing.

Remarkably, Rosebrock has had difficulty breaking into the world of fashion because it is occupied by trendsetters and a clique of tastemakers who eschew anything mainstream. He’s having difficulty making waves in the fashion community because they are not interested in technology filling the gap.

This is a perspective shared by Gabriel Aldamiz-echevarría, from recently launched community for fashionistas and fashion brands (backed to the tune of $800k in angel investment led by VitaminaK), Chicisimo, who in a Skype conversation told Search Engine Watch that “whilst data is great for matching, the definition of personal style is in the mixing rather than the matching”.

Put another way, how we wear something is just as important to our sense of style as what we wear.

While a recommendation engine can surface all the similar items, a tastemakers attention is focussed on the combination and uniqueness rather than “the mode” surfaced by data. The stat “43 percent of people combine these 10 items from these 4 outlets” is like the touch of death to a trendsetter!

The dissonance between data and ‘flair’ is where Rosebrock faces a dilemma as to how to get Chic Engine’s name circulating among the style conscious. If users cotton on to the fact his site may just be a super affiliate, filled with generic products, rather than a destination it it’s own right, he may quickly lose the interest of lurking trendsetters.

The index is going to need to be expanded and the natural ‘fairness’ of visual search means Chic Engine is sitting on an opportunity to leverage content from tons of boutiques that currently struggle to get visibility elsewhere.

What can he do? My suggestion would be to give something back to those he learned from.

What if part of the affiliate commissions were donated to a breast cancer charity? Now, that would be cool. Even among the trendy set.