Xiaomi filed a patent to search real world objects using augmented reality on mobile phones

25,078 Views

Augmented reality on mobile phones is gaining traction among businesses.

Tech companies are leveraging augmented reality on mobile devices in the areas of gaming, social media, marketing, brick and mortar (IKEA Place), healthcare and more.

A patent application filed by Xiaomi hints that the Chinese smartphone maker is working towards a future phone that can search for real world objects like “glasses” and “a public transportation card” using augmented reality.

Xiaomi’s patent application with the title, “Method, apparatus and storage medium for searching for an object” was published by the European Patent Office on May 29th, 2019.

According to the invention, a search request is given to a smartphone using voice, text or a picture. In response to the search request, the smartphone acquires a picture of the surrounding environment and detects the object in the picture. The object is then highlighted using augmented reality.

“In our daily lives, we often cannot find an item, for example, cannot find glasses, a public transportation card and so on. In such cases, we need to look for the item with naked eyes. However, by searching for an item with naked eyes, the item tends to be missed by our eyes, resulting in a prolonged search process,” Xiaomi notes in the filing.

Object search using augmented reality on mobile phones

According to a first aspect of the embodiments of the Xiaomi’s invention, the method for searching an object includes:

“in response to a searching request with respect to an object to be searched for, acquiring a scene picture for surrounding environment through an image capturing apparatus of the terminal device; detecting whether a suspected object corresponding to the object to be searched for exists in the scene picture; and when a suspected object exists in the scene picture, highlighting the suspected object in an augmented reality (AR) manner.”

Fig. 1 from Xiaomi’s patent application shown below is a flowchart illustrating a method for searching for an object

Fig. 9 from Xiaomi’s patent application shown below is a schematic diagram illustrating an application scenario of a method for searching for an object

Xiaomi in its patent application describes Fig. 9 as follows:

“As shown in Fig. 9, in this application example, the setting page of the user’s mobile phone includes a searching-mode enabling control element and the user triggers the searching-mode enabling control element when the user’s mobile phone is turned on. In this application example, the user sends a voice request to his/her mobile phone.

For example, the user says to his/her mobile phone “please help me find the public transportation card.”



In this application example, the user’s mobile phone turns on the AR camera in response to the searching request of finding the public transportation card, and detects whether a suspected public transportation card corresponding to the public transportation card exists in the scene picture captured by the AR camera of the mobile phone.

In this application example, the mobile phone may detect the similarity between a reference object in the scene picture and the public transportation card, and when the similarity is greater than or equal to the similarity threshold, determine that a suspected public transportation card corresponding to the public transportation card exists in the scene picture.

In this application example, when a suspected public transportation card exists in the scene picture, the suspected public transportation card is highlighted in an AR manner. For example, the suspected public transportation card and the direction to the suspected public transportation card are displayed in an augmented manner in the scene picture.

For example, the suspected public transportation card is circled with a virtual circle and the direction to the suspected public transportation card is indicated with a virtual arrow.”



Xiaomi notes that the invention can be used in other devices also, such as smart glasses, Augmented Reality (AR) glasses, a tablet computer etc.

The patent application was initially filed on November 28th, 2018.

The size of augmented reality market by the year 2024 will exceed $50B, a research report says.

Let us know your thoughts in the comments section.

Subscribe to our newsletter to stay updated on latest inventions.

Please leave this field empty Email *

Like 1+