The PlayStation 5 is expected to take Sony’s line of consoles to new heights with cutting-edge graphical capabilities, a wireless virtual reality headset, and roster of heavy-hitting exclusive titles when it eventually launches in mid-to-late 2020. To top it all off, the next-generation gaming system might also leverage the smartphones in users’ pockets to trail-blaze an all-new way gamers can play titles on the console.

Millions of Americans already play game like Fortnite on their smartphones, and soon enough, cloud gaming services like Stadia and xCloud will bring console games to tablets and phones. Sony wants to these transform these ubiquitous mobile devices into alternatives to its DualShock 4 controller, at least on certain games.

The company was recently granted a patent for the concept by the United States Patent and Trademark Office. First published on Tuesday, the filing describes how gamers could use devices like an iPhone or iPad to control in-game movements. It also looks like Sony is examining the potential to leverage smartphones to introduce augmented reality experiences in certain games. Here’s how the patent describes it:

FIG. 4 illustrates an avatar displayed as if it were co-located with a second user's mobile device from a vantage point of a first user in accordance with an embodiment. FIG. 5 illustrates a vehicle avatar displayed as if it were co-located with a second user's mobile device from a vantage point of a first user in accordance with an embodiment. USPTO

“Methods and systems for using a position of a mobile device with an integrated display as an input to a video game or other presentation are presented. Embodiments include rendering an avatar on a mobile device such that it appears to overlay a competing user in the real world. Using the mobile device’s position, view direction, and the other user’s mobile device position, an avatar (or vehicle, etc.) is depicted at an apparently inertially stabilized location of the other user’s mobile device or body.”

In other words, users might be able to wirelessly sync their iPhones to the PS5, and use their built-in accelerometers to drive a virtual car, pilot a digital airplane, or control a video game character. This could be especially handy when a DualShock controller needs to be charged but users want to continue playing wirelessly. They could plug in the controller and substitute it for their phones. It could also probably come in handy if you’re trying to play multiplayer.

To take the idea a step further, Sony also envisions using smartphones’ screens and cameras to overlay AR graphics onto the real world to make local multiplayer more exciting. The PS5 could take in-game assets and graphics and create a virtual space users could interact with by looking around with their camera.

FIG. 1 illustrates a first user holding a mobile device at a first position in space relative to a display in accordance with an embodiment. USPTO

Friends that get together to play a racing games, for example, could use their phones as steering wheels. The patent also states that they could point their camera at their friend sitting next to them and have the game transform them into an avatar of a race car driver. But it might take a while before Sony gets these features into living rooms.

The PS5 is still more than a year away, and a patent grant doesn’t always result in features that find their way into a final product. That said, it appears from the patent that this feature could still be implemented after the console’s launch. Game system’s lifecycles typically last around five to seven years, and a smartphone capability like this could be worked in with the creation of a mobile app or a PS5 software update.

Sony fans might one day be able to use the iPhones in their pockets to traverse digital worlds.