What’s not exactly new but has a dizzying number of applications? You know it! Motion capture or “mocap” for short. The technology is now becoming integral to many human products, services, and research ranging from the entertainment and video game industries to the fields of robotics and healthcare. And of course, professionals are always finding new applications for motion capture itself or its data. (We provide it ourselves, check out our motion capture services.)
Why is it so diverse in usage? We think it’s rather simple: we still move! And as long as we move, we’ll look to capture, understand, and upgrade our movements. (And we’ll even develop AI mocap!) Here’s a review of motion capture across various industries!
Films & Video Games: Motion Capture in Origin
We’re going for the entertainment industry and the video game industry in one section because they share motion capture’s most famous usage: realistic animation!
Realistic Animation
Motion capture has traditionally been used to bring real-life movements to characters in animations, movies, or video games.
Take Hellblade: Senua’s Sacrifice (2019), for example. Melina Jurgens, the actor who plays Senua, would get into a mocap suit and simply act in every scene and fight. The crew at Ninja Theory would record the acting using sensors and cameras, then map everything out on Senua herself. The great mocap-powered acting, the story, the artistry, and a serious attention to psychological phenomena made the game incredibly memorable.
Not a gamer? No problem. Remember Gollum in The Lord of The Rings movies? What makes the character extremely memorable is the fact that Andy Serkis didn’t just do the voice. He wore that motion capture suit, got in the studio, and did everything we see Gollum do. Yes, my precious!
Let’s also look at some more recent examples! The latest installments in the FIFA game series use camera footage from real soccer games to make players’ movements as realistic as possible, all thanks to motion capture technology. Movies or series like Avengers: Endgame (2019), Arcane (2021-), or Avatar: The Way of the Water (2022), etc. power some or all characters’ movements using mocap.
Stunt Performances
Speaking of movies and series, motion capture is also widely used to make stunts safer and more realistic! A stunt performer in a mocap suit would go through the dangerous moves in the studio, a controlled environment. Then, their movements would be used to power a 3D model of the original actor with the actor’s own face and physical characteristics! Neat, isn’t it?
The final animations are usually so realistic and mixed up with many virtual effects that a non-professional probably couldn’t tell them apart. You can look for such performances in action movies, namely the Mission: Impossible series or Mad Max: Fury Road (2015)!
Gameplay Analysis
Game studios, especially those dealing with cameras or VR games, could use the players’ raw movement data to analyze and improve their own gameplay parameters. Players don’t have motion capture suits or sensors, but their movements in front of a camera is data enough for mocap systems to process.
Some companies like Ubisoft (the creators of the “Just Dance” franchise) track player movements and analyze the data in real-time to provide improvements for the players themselves!
Immersive VR Gameplay
Every mocap session yields a mountain of data. (We should know, we specialize in mocap!) So, all motion capture solutions combine and put the captured data to good use. Two of these good uses appear in VR games, available firsthand to gamers!
One use is how cameras or inertial sensors track the player’s movement and provide an accurate simulation in-game. Job Simulator VR is a great example. It’s a humorous VR game that lets you “job” in a world where robots have taken over all our jobs! For another example, you can take a look at Superhot VR, the live shooter game.
Another use is similar to the one that we’ve already discussed. Various VR games like Asgard’s Wrath (2019) or Lone Echo II (2021) use motion capture data to provide more lifelike animations for characters that you see in the virtual environment, including other players’ characters and non-playable characters or NPCs.
A short video from TechRadar about Hellblade: Senua’s Sacrifice
Sports & Athletics: Motion Capture for Better, Safer Sports
As entertaining as motion capture technology could be, it’s time to dive into more serious uses. Mocap is used nowadays to analyze sportspeople’s movements to improve performances, prevent and treat injuries, and even judge games better. Let’s take a look!
Performance Analysis
The technology is used extensively to analyze performance and give feedback. It’s similar to what Just Dance does for gamers, but these systems are extremely precise and there’s a lot more at stake than your groove!
Take Vicon, for example. The British company’s mocap systems such as Vicon MX or Vicon Nexus are widely used across sports teams, universities, and research institutions. (Stay tuned to the rest of the article for motion capture in research!) These systems consist of various software, cameras, inertial sensors, and other devices that come up to very high-precision package.
The same goes for Catapult, a sports analytics company based in Melbourne. Their video analysis solutions rely on motion capture to track and optimize performance in many sports from soccer to lacrosse.
Injury Prevention & Rehabilitation
Sports injuries are often the result of unnatural movements. No-one likes them, but they occur anyways. The good news is they’re more treatable than ever thanks to numerous technologies including motion capture!
Let’s take a look at Noraxon as an example. This American company combines various hardware (including electromyography (EMG) devices and mocap cameras) to capture to provide gait, fatigue, range of motion, running, and many more analyses in real time. This plethora of data could be used in injury prevention, physical therapy, and research.
Bonus Sports Example: Hawk-Eye
Before we get to research, here’s another interesting motion capture application! Hawk-Eye is a computer vision system used in sports from Tennis and cricket to volleyball and union rugby. It relies mainly on several cameras situated around the sports venue with clear lines of sight, which record the ball’s movements at all times until, for example, someone scores.
The system, knowing the game’s rules, triangulates the footage from at least the three nearest cameras, calculate where the ball went or would go, and provide an opinion in under 10 seconds. It’s accurate and considered an impartial judge by many governing bodies in sports, but it’s still usually used as a second opinion to that of the judges.
A motion capture case study from Vicon
Healthcare & Research: Motion Capture for Human Movement
We talked about how motion capture helps with sports performance and injuries. Can it be used for healthcare outside that context? Absolutely! The movement data captured using mocap is a wealth of information if you know what to look for.
Biomechanics & Movement Studies
Biomechanics refers to the study of the living body in terms of mechanics, that is both outside forces acting on the body and the body acting on the outside world. Movement studies, on the other hand, is more self-explanatory. Motion capture for both of these fields is featured in hundreds of universities and labs around the world.
Many of the companies that we’ve already mentioned (like Vicon) don’t specialize solely in sports. But for there are tens, if not hundreds, of companies out there making use of mocap. So, let’s take a look at a new one!
Qualisys, “the world’s only provider of underwater motion capture cameras” according to their website, is a Swedish company specializing in motion capture solutions for biomechanics, gait analysis, and sports. They stand out because of the sheer variety of fields they’re in, from cricket batting analysis to animal motion capture!
Ergonomics & Workforce Safety
We do research so we can make a substantial improvement in ourselves or the world around us. So, applications always matter in research. Ergonomics (the science of how things are designed) and workplace safety are two substantial applications in movement studies.
While looking at companies in the field, AnyBody Technology stood out to us. They’re a Danish company specializing in biomechanical modeling. Their main software, AnyBody Modeling System, combines motion capture data and musculoskeletal modeling to let the user simulate and analyze movement in any given task. What’s more, it could also be used to analyze the movements of prehistoric and imaginary creatures!
A clinical application video from Qualisys
Robotics & Animatronics: Motion Capture for Machines
That’s right! Motion capture data is used today to enable robots to be more like their living counterparts. Let’s look at two obvious examples: humanoid robots and animatronic figures.
Realistic Robots
Who doesn’t like us to have better robots? (Well, we’re guessing many are still iffy about those, but it’s a rhetorical question!) Nowadays, especially as AI continues to simulate human intelligence, many companies around the world are trying to push the boundaries of robotics. One such boundary is to make robots move like humans or animals, and mocap data is a big help. (Many companies are also going for the boundaries of AI motion capture, but let’s talk robots for now!)
Softbank Robotics is an international company pushing the boundaries on “everyday robotics”. They’re the ones behind Pepper, the semi-humanoid robot introduced in 2014. Today, Pepper robots have a track record of being receptionists, research assistants, and even cheerleaders!
For another example, let’s have a look at Boston Dynamics, an American company. Atlas, their latest humanoid robot, is a prime example of human motion in robotics. It can leap, do somersaults, and run at 2.5 m/s, outdoing the average human 1.3 m/s!
Better Animatronics
Animatronics is a multidisciplinary field. To simplify, you can think of it as robotics in exhibitions and theme parks, resulting in animatronic figures. (Or automatons if you will, but that’s an older term.) These figures aren’t traditionally known to be very lifelike, but that’s quickly changing.
You know exactly what we mean if you’ve seen Na’vi, the Shaman of Songs, and her lifelike movements in Disney’s “Pandora: The World of Avatar” exhibition in Walt Disney World. You just know it’s powered by massive amounts of motion capture data!
A video about Atlas, the humanoid robot from Boston Dynamics
Extended Reality & VTubing: Motion Capture at the Limits
We’ve already mentioned how motion capture is used to bring a lifelike quality to VR games, but it doesn’t end there. Mocap applications, technology, and data are becoming more and more widespread. To understand the scope, let’s take a look at some more immersive experiences and a new YouTube sensation!
Augmented Reality
Augmented reality or AR refers to a combination of the real world and the digital. Some AR experiences could be sitting in your phone right now, whereas others might need more equipment.
Let’s start with those that are probably sitting in your phone: social media AR filters! These algorithms rely on motion capture data to track, analyze, and adapt filters that correspond with your facial expressions. Popular examples of this feature include Snapchat and Instagram AR filters.
A more sophisticated use occurs when you put on an AR headset, like a Microsoft HoloLens. The headset tracks and recognizes your movements thanks to previous motion capture data it has access to, so you can seamlessly pick up something virtual or interact with the environment in a myriad of ways.
Virtual Reality
Virtual Reality or VR is different from AR in that here, everything that you see is digital. Gesture tracking, the technology we just mentioned, is also used in VR experiences. Another use is in VR games, which we talked about at the beginning of the article. One use remains: VR for military training and simulation.
Motion capture data could be used to effectively track soldier movements and gestures, so they could practice tactical scenarios and coordinated maneuvers. Virtual Battlespace 4 is a prime example of software developed in the area, leveraging numerous technologies including mocap. According to its creators, Bohemia Interactive Simulations, the software is now used by more than 50 defense organizations around the world.
VTubing
VTubing, short for Virtual YouTubing, is a fun online phenomenon when you see a virtual avatar or character instead of a real person in videos or livestreams. These avatars’ movements and expressions are in sync with those of the person behind them, usually thanks to motion capture technology. (There are manual controls at work, but the bulk of the job is usually done using a phone and facial motion capture software.)
The phenomenon started in Japan, probably in 2016 with the introduction of Kizuna AI. In the recent years, however, it’s spread to all over the planet as a global, cross-cultural phenomenon. We thought a fun video from Bao The Whale, a VTuber, would be sufficient closing for this piece!
Bao The Whale introduces herself in a Q&A song
Final Word
Motion capture is overwhelmingly useful. (You can review its pros and cons here!) It’s used everywhere from video games to Snapchat AR filters, either directly with cameras, sensors, and suits, or indirectly with the data from previous cameras, sensors, and suits! We look forward to seeing what more it could bring to the human experience.
Need on-site or remote motion capture? We excel at them. Get in touch here and we’ll get back to you!
Author
