As you settle into your seat to watch the NBA action on the court below, you take a sip of your beverage and gaze through your augmented reality (AR) device. Players’ names and box scores appear and you watch replays on demand. When your favorite player makes a highlight-reel move, you share the play with other fans, annotating it with visuals and selected stats.
This seamless integration of the digital and real world is the new immersive fan experience — coming soon to an arena near you. Immersive sports analytics, or SportsXR, will not only give fans access to real-time digital data throughout games, it will collect and analyze in-game data, helping athletes and coaches optimize their strategy during games.
The collection and analysis of big data have transformed professional sports over the last decade. Many MLB, NBA, and NFL teams today employ in-house analytics teams that pick apart plays and tendencies to find an advantage over their opponents. They collect game data and use it to evaluate current line-ups, predict players’ performance, and adjust game strategy. But most data are collected through time-intensive, manual tagging of videos. This post-processing step results in low-fidelity and a time delay, especially at the college and recreational level and for many international teams.
By incorporating advanced sensing technology, computer vision, machine learning, and data visualization, SportsXR is poised to disrupt data analytics, taking sports decision-making to the next level. SportsXR will bring the power of sports analytics from backroom data science experts into the hands of the wider audience of coaches, athletes, and fans, where it can have the most direct impact. To improve player performance, it will supply coaches with real-time information that enables athletes to reach their full potential. To enhance the fan experience, it will provide augmented, real-time shared data displays; attending sports events at the stadium will be more engaging and customized. By blurring the boundary between the digital and real worlds, SportsXR has the potential to create a new sports fan culture, changing both media consumption and content creation. The opportunities for SportsXR at the intersection of sports analytics, broadcasting, mobile device applications, and augmented reality devices are tremendous.
In-game analytics for fans
In our SportsXR research team at the Harvard Visual Computing Group, we are working with college basketball teams to develop a SportsXR experience for fans, athletes, and coaches. Towards the goal of creating an immersive fan experience, we are researching ways to allow users to seamlessly collect in-game data, to analyze them in real-time, and to give fans the ability to create and share their own data stories during live games. For example, a fan could assemble and share team line-ups, shot types, play outcomes, and team strategies. Augmented reality displays of sports analytics data will give fans unique game insights around the events taking place on the court. We are also researching intuitive ways for social content creation and sharing of SportsXR data.
Training
We are also researching SportsXR technologies that will give direct visual feedback to coaches and athletes in realistic training settings. In basketball, shooting is one of the most valuable skills. By showing a player information such as the ball’s trajectory and the optimal shooting angle from the current position on the court, we can help athletes improve their performance by giving them immediate visual feedback. Combining virtual information and real-world actions sharpens players’ physical and mental practice. For coaches, we are researching ways to immersively display analytics data, such as shooting percentages visualized as a heat map overlaid on the court, for real-time evaluation and decision-making.
The foreseeable future
The set of technologies that enable SportsXR will be part of the emerging AR cloud. The AR cloud is a real-time, virtual information platform based in the real world that, just like the internet, or the Internet of Things, promises to change the way we communicate, make decisions, and consume media. Industry and academia are developing this next-generation information system. Our research in SportsXR is a pioneering effort to bring the AR cloud to sports.