You can read the more about Search Marketing on my blog .

Bing recently introduced the Visual Search Feature which is an evolution in the search world. This was first announced on 1st June 2017. With this feature a user can perform a visual search using any particular portion of the image. This feature is available both on the desktop and smartphones. According to Bing, this tool will work on both user generated images as well as the images available on the internet. The tool is also available to developers as an API.

How to do a Visual Search?

It’s pretty simple. Just follow these steps:

Firstly, you search for an image on Bing.com .

Then, click on the ‘Search within this image’ button which is on the top right corner. The logo has a magnifying glass enclosed in a dotted rectangle.

After that, the Image opens in a new window with a visual search selector which is more like a cropping tool.

Place the cropping tool on a particular portion of the image you want the search results for.

Example - Let’s say you are on the ‘Pirates of the Caribbean ‘poster and you use the cropping tool to focus on Johnny Depp’s face. Here’s the screenshot of the search result.

Focusing on Johnny Depp's Face in the poster -

The Image Search Results -

Looks pretty cool right? Try it out right away.





How accurate is the Bing Visual Search?

So I did a few tests to see how accurate this new feature is. There were a few interesting findings about this tool. The tests were more exploratory in nature, hence didn't consider any hard metrics.

Test One : A table set

Image Keyword – [table set]

Image –

Testing Various Portions using the Visual Search feature –

Cropping Tool on the 'Flower Vase':

Search Result -

Cropping Tool on the 'Glass of Wine' :

Search Result –

Results :

The outcome of focusing on the Flower Vase was incredible. It did show some flower vases of different shapes and sizes. Currently the product suggestions tab hasn't rolled out.

Next I focused on the wine glass , it showed me image results of different wine glasses.

Even though there were other elements in the frame of the cropping tool like some part of the flower vase was in the frame of the wine glass, the cropping tool still understood the prime focus object and showed results accordingly.

The table test looks like a high scoring test. Let’s move to the next test.

Test Two: Manchester United Squad

Image Keyword – [manchester united squad 2017]

Image –

Testing Various Portions using the Visual Search feature –

FA Cup Shield :

Search Result –

Wayne Rooney’s Face :

Search Result –

Manchester United's T-shirt Logo :

Search Result –

Results :

First I focus on the FA Cup shield, it started showing random coins and silverware.

Second , I focused on Wayne Rooney’s face, the search result showed me random people.

Lastly, I use the cropping tool to focus on the T-shirt logo, it shows random low resolution logos which isn’t even close to the Chevrolet logo.

Scope

Bing's new visual search has a lot of potential to say the least. They can easily use this to identify products from images and suggest users what to buy and where to buy the same.

Let's say you are searching for a tie which matches a grey colored suit.You come across one particular image with the most suitable tie. You can use the visual search image to find where the product is sold and but it online.

Sounds crazy right? Well it's happening.

Test Conclusion

The image search works perfect on objects like the Vase and the Wine Glass. It still needs a lot of work on facial recognition. Through all the tests I did, I found that the image resolution is extremely important. The high definition table pictures made it easier to perform a visual search on a particular portion whereas the low resolution Manchester United Squad Tshirt Logo made it difficult for the visual search to work. One interesting thing, when I focused the cropping tool on the FA trophy even though it didn’t show me the FA shield in the search results, it was able to identify that it’s a silverware and showed me results accordingly. This can be enhanced for various use cases. When you focus the cropping tool on a particular text in a picture, it identifies the font and background color and shows some similar image results. The font of the image search results might not be exactly the same but similar.

Similar Tools by Competitors

This is not exactly the first tool of it's kind. Here's a list of the other existing tools :

Google Lens

Pinterest Lens

Google ‘Similar Items’

Instagram shoppable tags

Bing.com employs have done a great job creating this feature. Obviously, it needs a lot of work but it’s the beginning of a new era for Search. The tool has a lot of scope , provided the improvements are made timely. Even though Google remains a dominant search engine , it has some competition in terms of image search.

Did you try the new Visual Search on Bing.com? Did you like it? Do share your views and findings.