At this year’s National Association of Broadcasters (NAB) convention, the IBM sports and entertainment team accepted an Emmy® Award for its advancements in curating sports highlights through artificial intelligence (AI) and machine learning (ML).

The Technology & Engineering Emmy awards recognize individuals or organizations whose work provides “extensive improvement on existing methods” or whose contributions are “so innovative in nature that they have materially affected television.”

IBM was granted the award for an ensemble of AI and ML technologies that help people consume broadcast content in new ways. These include computer vision and sound recognition systems that help automate curation and editing of exciting moments in sports; systems that extract summaries or factoids from vast amounts of structured and unstructured natural language data; and systems that forecast and predict player performance and winners. These technologies have also enabled “second screen” synchronization that now routinely complements broadcast television on linear channels.

These innovations create a symbiotic relationship between content and the audience, on-air talent and production and post-production workflows for many of the world’s most high-profile sports and entertainment events. These include the Masters, the GRAMMYs, US Open Tennis, Wimbledon and ESPN.

Long-term partnerships foster innovation

Though organizations can quickly realize business value by adopting high-impact technologies such as hybrid cloud or intelligent automation, the kind of industry-changing transformation that the National Academy of Television Arts & Sciences (NATAS) has recognized doesn’t happen overnight. It requires forethought, planning and collaboration.

“The IBM sports and entertainment team got to take home the Emmy, but credit for the award should be shared with the many client organizations who trusted us and collaborated with us to build and implement their technology roadmaps over the longer term,” said Noah Syken, Vice President of Sports and Entertainment Partnerships at IBM. “When clients set a goal two or three years down the line, we can bring all the pieces together incrementally and tap into technologies as they emerge. They also get to collaborate with the great minds on our team who understand how to bring these capabilities to life in new and amazing ways.”

Over decades of long partnerships with its clients, IBM Consulting has developed a proven collaborative approach that delivers similarly impactful results in many sectors, though most go unrecognized by a statuette. The IBM Garage™ methodology integrates business strategy, design and technology to help organizations transform their business from ideation to building to scaling.

Dreaming big with the Masters

Many of the technologies recognized by NATAS were embodied in IBM’s work with the Masters Tournament. For example, in the Masters app, the My Group feature lets viewers create a personalized highlight reel of their favorite golfers, powered by AI.

How did this come about? Syken recalls a pivotal collaborative session with the Augusta National Golf Club. “In one of our IBM Garage sessions we had with the Masters team, around 2015, we encouraged them to dream up a crazy idea, no idea too outlandish, let’s put them up on the board.”

 One idea was creating an AI director that could understand what was happening on the course and who was competing, and then contextualize and prioritize the highlights for broadcast viewers. This idea became a North Star that helped the IBM Consulting and Masters teams build out their technological capabilities. They realized the system would not only need to capture and analyze every shot from every hole, it would also need to capture other data that improved understanding, such as the level of crowd noise.

Though these capabilities didn’t exist in 2015, the IBM team, led by Stephen Hammer, Sports CTO and Distinguished Engineer at IBM Consulting, Monica Ellingson, Sports Practice Lead, and Aaron Baughman, Distinguished Engineer of AI and Cloud at IBM Consulting, could begin guiding the research to make the dream a reality.

Exploring trends and leveraging technologies for clients

Another integral role that the IBM team takes on in its partnerships is to identify marketplace trends: how other organizations are aggregating data and adding value, and how IBM can incorporate these advances in unique ways.

“Through our partnership with ESPN we recognized that fifteen million people play on their fantasy football apps every week, and we could add value with AI,” says Syken. “Aaron Baughman and the team started to think about how we can enrich that experience, and that led to our ability to provide analysis and predictions and context to fantasy football players to help them make better selections.”

“Our work on ESPN Fantasy Football ended up advancing the broadcast TV experience, because the ESPN commentators would use some of our work to figure out what to say on-air,” says Baughman. “They would look at the boom-and-bust projections, figure out which players were most likely to play with hidden injuries or play in meaningful minutes, what their score spread would be, which players were most likely to be traded—then put commentary around it.”

“The technologies we invented and created together became a beachhead that helped IBM change the way broadcast TV is consumed and ultimately enjoyed,” says Baughman. “The innovations around AI and automatic workflows that aid in predictive analysis for football can also apply to coverage of other sports. The computer vision and sound technology that we developed for golf could be applied to tennis and soccer broadcasts.”

Pioneering use of unstructured text data

For over a decade, IBM has been gathering insight from unstructured data (such as that used in large language models) to provide real-time insight to its sports and entertainment clients and to enhance predictive analysis.

“Sports fans are very accustomed to the numbers: the facts and figures, how far did the ball go, how fast did the athlete run, how much do they win or lose? And those are the things that determine ranking,” says Syken. “That’s only good to a point, because when you get up to the tournament time, how they’re playing right now is really important. The reporter who says or writes, you know, I watched a player in practice yesterday and her ankle’s hurting—that can have a real impact. Our systems let us incorporate that report with structured data to make better predictions.” Such systems that combine structured and unstructured data are used in the US Open and Wimbledon apps to deliver better predictions about player performance.

These technologies also improve the second screen experience, where viewers refer to their mobile device or computer while watching a live broadcast. “A fan watching the US Open can look at a player’s likelihood to win and say, wait a second, why is this player supposed to lose this match?” says Baughman. “They can read media mentions or look at ‘by the numbers’ where we write sentences based on past performance in the current tournament. All these insights are put together to tell the story around a tennis player to augment their broadcast TV experience.”

“We also use unstructured data for the GRAMMYs, using a technology called extractive summarization,” says Baughman. “We burn factoids into the live stream that pop up in the video as the artists walk down the red carpet.”

An honor beyond technology

“Though the Emmy award IBM received is for technology and engineering, we also see it as a validation of our human-centered processes and the IBM Garage approach,” says Syken. “As teams from IBM Consulting build capabilities through each client engagement, they also build trust that enables long-term partnerships. And that leads to innovations that can change an industry.”