The Eyes, Ears and Brain of the All New Kinect
One of my favorite moments from our recent Xbox Reveal event was seeing our resident Kinect experts, Scott Evans and Kareem Choudhry, demonstrate some of the incredible technology that is shaping the new Kinect.
With Xbox One, the all new Kinect is an essential and integrated part of the platform. By having Kinect as a consistent part of Xbox One, game and entertainment creators can build experiences that assume the availability of voice, gesture and natural sensing, leading to unrivaled simplicity, premium experiences and interactivity.
In these two videos, Scott and Kareem show examples of how the “eyes, ears and brains” in the new Kinect are significantly advanced.
Check out Kareem in the “Eyes and Ears” video showing features like 3D vision, the larger field of view and the magic of the 2D color camera. The 3D vision of the new Kinect sensor has three times the fidelity of the first generation Kinect, so you can see great details such as the wrinkles in your shirt, facial expressions, and the fingers on your hand. These improvements will enable game developers to create games with heightened natural interaction and engagement for more realistic gameplay. And more accurate interaction with Xbox One across all scenarios.
With the larger field of view, the new Kinect will enable you to stand close up or far away, play with more people, and detect different heights. I love this feature because now my son and I can easily be detected in the same field-of-view. I had the opportunity to try this scenario with Kareem – who is 6’5” – and it easily detected both of us (I am 5’1”).
The 2D color camera has 1080p, which is great for full-screen Skype conversations. It also has active IR, so Kinect can detect your movements in dimly lit rooms – when you’re playing a game in the dark, or watching a movie, for example. The new Kinect can literally see in the dark, as well as remove ambient mood lighting like lamps or other variable lighting for strong accuracy.
The “ears” of Kinect are also dramatically improved. The video showcases the fidelity of audio accuracy by showing the real noise in a room – with loud game play and other noise – contrasted with what the Kinect sensor hears when audio processing is turned on and the full microphone array is at work. The new Kinect sensor removes room noise and reflection so your voice commands will simply work.
Next, Scott shows us more details on the “brains” of Kinect in the second video. Through a rich combination of science – ranging from computer vision, machine learning, speech recognition, facial recognition, and digital signal processing – Microsoft has harnessed the science with creative energy from the Xbox and Microsoft Research teams to deliver the new generation of Kinect. Two demonstrations showcase this at work. The first looks at skeletal tracking and demonstrates the fidelity of the joints and anatomical correctness the new Kinect can display. Everything from your hip, spine and shoulder movements to the tips of your hand can be incorporated into gameplay with Xbox One. Kinect has advanced to a true human understanding and fidelity of expression as people move around. And, Kinect can now easily distinguish up to six players at the same time, in the same field of view.
The orientation of body movement is also shown – affectionately known as Block Man. This demonstration shows how Kinect can accurately depict what is being done by the body, a further reflection of Kinect’s ability to recognize and reflect movement.
These are just a few of the ways we’ve completely re-engineered Kinect to take full advantage of all that Xbox One can do. The new Kinect is more precise, more responsive, and more intuitive. It’s an entirely new Kinect for a new generation of entertainment. These videos showcase just a glimpse of what is to come when game and entertainment creators deliver magical experiences that bring this advanced technology to life. I can’t wait.