Apple wants to use Augmented Reality to help the visually impaired

25,966 Views

The United States Patent & Trademark Office granted a patent to Apple last week that relates to warping images in an augmented reality device to help the visually impaired see the full view.

Apple notes in its patent that, “Many types of visual impairment may result in partial loss of sight, or weakened sight. Visual impairments may be the result of any number of health issues, such as diabetes, old age, retina issues, and the like.

As an example, some people develop floaters or blind spots in their eye which cause obstructions in a person’s visual field. The result is that people often have to compensate for blind spots and other obstructions and impairments by viewing only portions of a scene at a time. Thus, the full field of view is never made available at once.”

Apple’s invention is summarized as, “A method for displaying a warped area includes obtaining an image of a real environment, obtaining a portion of a field of view comprising an obstruction, determining a portion of the image of the real environment corresponding to the obstruction, applying a warping function to an area surrounding the portion of the image of the real environment corresponding to the obstruction to obtain a warped area, and displaying the warped area.”



According to one or more embodiments of the invention, a camera may capture an image of a view of a real environment. A portion of the view of the real environment may be determined to be occluded to a user.

For example, a user may have an optical obstruction, such as a blind spot or a floater that makes a portion of the field of view occluded. As an example, a portion of the real environment may appear blurry, or not appear at all, based on the optical obstruction.

A warping function may be applied to the image surrounding the obstructed portion. In one or more embodiments, the warping function may warp the area surrounding the obstruction such that the real environment appears warped around the obstruction. Thus, when the warped area is displayed, the user can view the full field of view regardless of the optical obstruction.

Apple’s figure 2 shows, in flow chart form, the method for augmenting an image of a real environment

Referring now to Apple’s figure 4, a system diagram is shown for an example setup for utilizing a device for presenting a warped image of a real environment, according to one or more embodiments.

Apple’s figure 4 shows a user #410 utilizing an electronic device #100 that includes a front-facing camera #110, a back-facing camera #120. In one or more embodiments, the user may view a real environment #405 through the front-facing camera #110 of the electronic device #100.

In one or more embodiments, an image of the real environment #405 includes a field of view that is similar to a user’s field of view #425 if the user were to look directly at the real environment #405.



Apple’s figure 5 shows an example system diagram of an augmented reality device to warp images. Figure 5 shows the real world environment as depicted through a display #150 of the electronic device #100.

The real world environment as viewed by a user with an optical occlusion is depicted at #505. As shown, part of the image may be obstructed due to the optical occlusion. For purposes of this example, a real-world environment may include a series of numerals from one through eight.

A user gazing at the real world environment #405 without the use of the electronic device 100 may see an obstruction over the numerals 5 and 6.



As described above, the calibration module #160 may identify a location of an occluded portion #520 of the real world environment. Warping module #155 may generate a warped area #515 surrounding the obstruction.

As described above, the warped area is generated such that at least a part of the area within the obstruction #520 is visible within the warped area #515. As depicted, numerals 5 and 6 were not visible to the user without the aide of the electronic device in #505, but become visible when warped in the warped area #515.

The remaining portion of the real world environment may remain unchanged in order to be less distractive to the user.

This patent was initially filed in September 2017. It is not known at this point of time when Apple is going to implement this Augmented Reality invention in its products.

Let us know your thoughts in the comments section.

Subscribe to our newsletter to stay updated on the latest inventions in tech.

Please leave this field empty Email *

Like 0