The Next Step in Gaming Experience
Despite the huge technological advancements we have seen over the past years, the television still remains the main focus point of living room entertainment. While visual and audio quality have improved a lot, the content we watch (e.g., games, movies, TV shows) remains boxed in the frame of the display, thus restricted by the physical size of our screen. The whole experience is limited, as an entirely virtual world is trapped in the television, completely ignoring our physical environment (Jones et al., 2013).
Now imagine the following scenario. You sit down in your living room to play a video game on your television. When the game starts, the room magically transforms to look like a cartoon, matching the shading in the video game. The colors of the room become super-saturated and cartoon edges appear on your bookshelves. You come across an enemy in the game and suddenly a streaking bullet flies towards your character and then out of the television. The enemy throws a grenade towards you. The grenade rolls out of your television, bounces off your coffee table and explodes in your living room, throwing shrapnel across your bookshelf. The entire living room appears to shake as you take damage from the explosion.
So how can we make this scenario happen? How can we take our living room entertainment, especially our gaming experience to the next level? Microsoft has been working on a project which has the answer to these questions.
IllumiRoom is an idea created with the purpose of extending the visual experience outside the television screen. It uses the television to provide a traditional, high-resolution gaming experience and a projector, designed to cover a wide area of the surrounding physical environment. This projector provides low-resolution information for the user’s peripheral vision, which can negate, include or even augment the physical environment, thus enhancing the content displayed in the television screen (Jones et al., 2013). Such visualizations are called peripheral projected illusions by the project designers.
In that context, IllumiRoom can change the appearance of a room, induce apparent motion, extend the field of view and enable entirely new virtual-physical gaming experiences. And of course it does not assume the physical space around the display to be a white flat projection screen. The system is designed to capture the appearance and geometry of the surrounding room (such as furniture) and use the information to create novel, interactive visual experiences (Jones et al., 2013).
After the presentation and very good reception of the idea in 2013, and despite the fact that there was no commercial use of the system yet, Microsoft decided to take the design one step further. So this year, to be specific a couple of days ago, they presented IllumiRoom 2.0 or RoomAlive as they called it.
This new prototype is based on the idea that with gaming experiences, bigger is often better. So what RoomAlive does is that it transforms any room into an immersive, augmented entertainment experience and enables users to naturally interact with augmented content in their everyday physical environment (Jones et al., 2014). To achieve this, apart from the projector used in the IllumiRoom, it uses a wide range of depth cameras, or procam units. By tiling and overlapping multiple of these units, RoomAlive is able to cover the room’s walls and furniture with pixels. It also tracks the user’s movements and dynamically adapts the gaming content to the room. Combining these features, the users can touch, shoot, dodge and steer virtual content that co-exists with the physical environment.
While Microsoft’s explorations are performed in the context of interactive games, the same or similar illusions could be used to enhance movies and television content. If they ever achieve to turn this project into a commercial product (obviously now the costs are extremely high), we will be able to take the step into a new era for gaming and entertainment in general.
Get a sneak peek of the two projects:
Jones, B.R., Benko, H., Ofek, E. and Wilson, A.D. (2013), “IllumiRoom: Peripheral Projected Illusions for Interactive Experiences”, http://research.microsoft.com/en-us/projects/illumiroom/IllumiRoom_CHI2013_BJones.pdf
Jones, B.R., Sodhi, R., Murdock, M., Mehra, R., Benko, H., Wilson, A.D., Ofek, E., MacIntyre, B., Raghuvanshi, N. and Shapira, L. (2014), “RoomAlive: Magical Experiences Enabled by Scalable, Adaptive Projector-Camera Units”, http://delivery.acm.org/10.1145/2650000/2647383/p637-jones.pdf?ip=188.8.131.52&id=2647383&acc=OPEN&key=4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E6D218144511F3437&CFID=581742302&CFTOKEN=90305801&__acm__=1412847756_d269bac028ac0fcb203ef216cbccbed2