The “Kinect Sports Rivals” Champion scan – the process of scanning your face and body to create a digital, stylized version of yourself that will represent you in the game – is an amazing new-gen feature that is made possible only with the new Kinect. After scanning your body, Kinect focuses on your face and captures 93 measurements to build a 3D model, then adds features, such as skin color and hair style, to create the finished character. No two Champions look alike, unless you have an identical twin.
We sat down with the Champion feature’s lead engineer, Andy Bastable, and lead artist, Iain McFadzen, to learn more.
Xbox Wire: What came first, the new Kinect technology or the idea to scan players and create a stylized version of themselves?
Iain McFadzen: We actually had the idea for quite a while, but it was only when we saw the power of the new Kinect sensor that we truly felt it was something that we could come to fruition. That said, although the hardware was in place, the underlying software needed to scan and recognize the various elements that make up a human face had to be built from scratch.
Xbox Wire: Can you tell us about the initial pitch meetings? Did people think the idea was crazy/couldn’t be done?
Andy Bastable: The great thing about the Champion feature is people just get it. We didn’t need a well-crafted “elevator pitch” or snazzy images to get the concept across. All we said is “we’re going to create technology that can scan you and put you at the center of the game.” Everyone we talked to thought it was a fantastic idea and would really show off the power of Xbox One and the new Kinect sensor.
Iain: I think the biggest concern we faced was, as you can imagine, not everyone wanted a 100% reflection of themselves in game. So, this is where we coined our code name for the feature, “Awesome You,” because basically, we are taking your essence, your basic likeness and building this awesome character, your Champion. We want to make everyone look great in “Kinect Sports Rivals.”
Xbox Wire: How many people only worked on Champion scanning? How many hours did it take to create the scanning process?
Andy: The Champion feature was truly a global effort. The vision and technical leadership was driven by Rare in the U.K., while a lot of the under the hood “magic” was created by our research and development teams at the Redmond campus in the U.S. In addition, we had to run a number of user research tests with an array of people, which had us travelling through North America, Europe and Asia to gather the data we need. All-in-all, you’d need to measure the effort in man-years rather than hours to get the real total.
Xbox Wire: Can you tell us about the science that powers the “magic” behind the Champion creator
Andy: Sure, It may feel like magic, but there is a whole lot of science behind Champion scanning!
The whole scan process is made up of little systems called “pickers” or “classifiers” that analyze the data from the Kinect, and try to determine certain features of a person’s appearance, like height, skin color, hair style, etc. The first of these to run is the “body” classifier that determines physique. It takes lots of tiny measurements from the body, but, in keeping with the goal of creating an awesome you, the scan tries to make the result attractive and flattering! Next, we assess whether a player is wearing glasses or has facial hair by sampling pixels from the face, and we store this data for later. Then, we build the face model out of 93 different measurements of the face, add some stylish features, like a slightly chiseled jaw line for men, and try to subtly exaggerate the face to make it look as much like the person as we can.
After we have the face model, we then run our most challenging classifier, to choose the correct skin color. People often assume that picking the skin color would be rather easy, because of the color camera feed that the Kinect has, but in reality, people’s faces are often in shadow or have light shining on them, so it’s really tricky to get this right! For this, we use another feature of the new Kinect, the Active IR feed that can see in the dark, to help us here. This gives us a measure of the skin color, even if all the lights in the room were off.
The final classifier we run is to pick the best hair style for the person. We use the depth information from Kinect to see how much hair volume there is around the face, measure the amount of fringe (or what our American friends call ‘bangs’) we can see, and try to pick the best color. Once we have all the information from the pickers, we put it all together in the big reveal and present you with your finished Champion!
Xbox Wire: How many different Champions can it create?
Iain: We are actually in the realms of billions, if you factor in all of the different settings we feed in to the system. Just the result of our classifiers results in 44,000 different combinations, and that’s before we add in the different face models. With 93 different values to set (effectively like sliders in the kind of character creators you see in other games) to alter the face, this gives us the possibility to create potential Champions in the ballpark of the number of stars in the universe. The challenge for us is to make sure we pick the right combination for the person standing in front of the camera.
Xbox Wire: Any ridiculous early scanning memories? Did anyone appear with two noses or four eyes?
Iain: We had a few instances where the outcome wasn’t quite as expected! Early on, though, we managed to train the Xbox One to understand what a basic human head should look like, which stopped the really weird occurrences. We did still have what we called the “Klingon” for a while where, if you had fringe, the Xbox would give you a bumpy forehead. The engineers had to write a particular piece of clever code to take care of that particular problem.
Xbox Wire: What major milestones did the team hit along the way?
Iain: I remember a key one where we were able to take the facial data of team members we had captured and passed it through our pipelines and see the results. At this early stage, it was still outside of the game, but you could clearly recognize the individuals. This proved our methodology was going to work and we were on the right path to get great results.
Andy: Another key milestone was when we unveiled the feature to the public for the first time at gamescom in August last year. Although we’d done a fair amount of user research and testing, there’s nothing quite like putting a feature such as this in front of the general public and journalists for the first time. The response we received was amazing and really spurred the whole team to continue on.
Xbox Wire: What was the biggest challenge your team ran into when building this tech?
Iain: For me, it was actually the issue with how people perceive themselves. As we went through the journey of creating Champions, our user research would show that the people weren’t happy with the results. It wasn’t that it didn’t look like them, it was more that people wanted a character that represented how they wanted to be in the world of “Kinect Sports Rivals.” So with that, all the Champions in our world have a very distinct art style and they’re super athletic. Even if you’ve put on a few pounds in the real world, you’ll be a sporting titan in ours.
Xbox Wire: After the initial scan, “Kinect Sports Rivals’” players can customize their Champion. How many customizations are there? How did you land on these specific customizations?
Iain: In total, we have over 200 customization options in-game, including hairstyles, beards, body size, and we even have options for make-up, eyelashes, and eyebrows! We tried to strike a balance that would cover the majority of the population, but still give people enough options to express themselves how they want.
Xbox Wire: The Champions are stylized. How was the art team able to create Champions that resemble the player, but still fit into the “Kinect Sports Rivals” world?
Iain: Through a lot of hard work! Seriously, my team had to iterate constantly through development to try to strike the balance of being recognizable and overall Champion awesomeness. Go too far one way and the likeness is amped up, but the results are underwhelming, too far the other way the Champion looks great, but everyone starts to look the same.
Xbox Wire: A lot of people have already had the chance to create their Champion. Have you seen any creative uses for the Champions? Are they used as profile pictures in social media? Has anyone created a life-sized cutout of themselves?
Andy: The Champion creation experience is really great, truly magical, and almost everyone who’s gone through it has been keen to share out their Champions via Facebook, Twitter and a myriad of other social media. At the studio, our email accounts carry our pictures, so, as you can imagine, over the past few weeks at Microsoft, Champions have been popping up all over the place.
Recently, at a press event at the Rare studio, a number of the team printed out their life-size Champions. The overall winner of the competition that day had a scale 3D print of their Champion as the prize. It looked fantastic.
Xbox Wire: Anything else you want to share about Champion scanning?
Andy: Just give it a try. It’s a whole load of fun, especially when you have your friends and family with you. Also, not only does the Champion resemble you, we also look at the skill level you play at, the outfits you wear, and the Power-Ups you’ve equipped, and store all this information with your Champion in the Cloud. What this means is that your Champion will turn up in the games of other people, even if you aren’t playing. So, almost everyone you play against in “Kinect Sports Rivals” is a Champion of real person. Also, anyone on your friend list is able to challenge you to a Rivals Challenge to gain extra experience points and coins to unlock the next cool piece of equipment.
Iain: It really does put an extra edge on competition when the person you’ve just pulled of the rock wall in climbing, or barged in to a mine in wake racing, looks and plays like one of your friends or family. And, of course, as they’re Champions, they’re always available to play against, 24/7.
You can still join in the battle for leaderboard supremacy right now in “Kinect Sports Rivals” Preseason. Also, if you want to enter the world’s largest motion-gaming competition, “Kinect Sports Rivals World Champions,” and have the chance to win the grand prize of $10,000 and a trip of a lifetime for two to one of four amazing destinations, you can register today and get ready to compete when the game launches in April. Still haven’t pre-ordered your copy of “Kinect Sports Rivals”? Get it now!
“Kinect Sports Rivals” launches for Xbox One on April 8, 2014 in North America, Latin America, Australia and New Zealand, and on April 11, 2014 in Europe. Remember, it is you vs. the world!