UPDATE: The original version of this article referred to the document as a "patent" rather than a "patent application." Ars Technica regrets the oversight.

A newly published patent application shows Microsoft potentially planning to take console gaming beyond the barriers of the TV screen by projecting a "peripheral image" around a room, providing a 360-degree view of a virtual scene.

Microsoft's patent application for an "immersive display experience" was published by the US Patent Office last week after being filed back in early 2011. It describes a standard video game system with a connected "environmental display" capable of projecting a panoramic image that "appears to surround the user."

Such a projector wouldn't replace the central TV display used in current consoles, but it would provide a "peripheral image" that would "serve as an extension" of that primary display. The purpose, of course, is to extend the gaming environment outside of the TV screen, so a player could, for instance, "turn around and observe an enemy sneaking up from behind."

The display device as described is integrated tightly with a depth-sensing camera system (read: Kinect) that could even be housed in the same casing as the environmental projector, according to the application. This device could be a standard two-camera, structured-light-sensing system like the current Kinect, or a more sophisticated model that could include "multiple image capture devices" to "stitch a panoramic image from a plurality of captured images" pointed in all directions around the room.

Regardless of the form, the depth-sensing camera described in the patent application aids the environmental projector by sensing the layout and topography of the room. This allows the projector to provide color and distortion correction, so the projected images look correct even when cast against different walls and pieces of furniture.

In addition, the environmental display could use player position information provided by the camera system to "adjust an apparent perspective of the peripheral image display," making sure the environment continues to look natural as the player moves around the room. The document even describes creating a "shielded" region in the projected image where the player is standing, so light doesn't shine directly in a player's eyes as they look in the direction of the projector.

Projected images in the system described by the document "may be displayed at a lower resolution than the primary image," but the document suggests this would not "adversely affect [the] user experience." The projected images wouldn't necessarily just sit flat against the wall, either—they could work with stereoscopic 3D glasses for an added sense of depth and realism, according to the application. That last part gels somewhat with the "Fortaleza glasses" concept for a heads-up, augmented reality display mentioned in Microsoft documents leaked in June.

Holodeck Version 0.5?

The system described in this patent application may never turn into a full-fledged product, but this kind of environmental system could be a real revolution in the way we play and think about video games. One of the main factors limiting the current potential of motion controls, as I see it, is the need to constantly stay facing the TV screen in order to know what's going on in the game. This leads to a lot of games where leaning and shuffling to the side serves as a relatively poor substitute for actually turning your body or walking parallel to the screen.

With a game projected on all four walls, however, you'd be free to really walk around the entire room and look in any direction without missing out on what's going on. Imagine a new version of Dance Central that projected actual Arthur Murray-style footprints on the floor, or an adventure game where you actually had to explore the room to find hidden treasures, or a first-person shooter where you physically turn toward a muzzle flash projected on the wall to your right, rather than simply twisting in response to a red flash on the right edge of the screen.

If this kind of environmental display worked as advertised, it could obviate the need for head-mounted virtual reality displays like the Oculus Rift, which use sophisticated head-tracking to simulate an environment that surrounds you. If the image of the game world is actually surrounding you, there's no need to calculate which sliver of that world to show the player based on where they're looking.

That said, I am a bit concerned about how well a single 360-degree projector can actually transform a room. Unless the projector described here was mounted on the ceiling, I don't see how it would avoid the significant problem of image-blocking shadows, not just from the player but from assorted furniture and detritus in the room. Color and image-distortion correction is a nice idea, but if a piece of the scene is supposed to appear in the back corner of the room, having it instead appear on the Lay-Z-Boy sitting in front of that corner is going to be a bit disconcerting.

Still, it's interesting to see Microsoft's R&D department thinking about ways to extend games past the black borders of our flat-screen TVs in a potentially revolutionary way.