• The NFL is using GenAI to streamline its content creation process, significantly reducing the time it takes to find specific plays and media assets. Previously, searching for specific plays could take upwards of 10 minutes using traditional filtered searches. By leveraging GenAI, this process has been reduced to approximately 30 seconds. This allows content creators to spend more time on the creative aspects of their work and less time on tedious searches.
• One of the key challenges in developing this system was translating natural language queries into a format that the underlying database could understand. To address this, the NFL and AWS teams developed a custom solution that combines traditional natural language processing techniques with a large language model (LLM). This system identifies relevant entities (players, teams), actions (touchdowns, interceptions), and constraints (yardage, quarter) from the user's query. The LLM then translates these elements into API calls that the database can understand.
• To ensure accuracy and user confidence, the system incorporates a "reasoning step" that allows users to see how the LLM arrived at its API call. This transparency allows users to validate the system's logic and provide feedback, which further improves the system's performance.
• The system has been designed to learn from past executions, making it faster, cheaper, and more reliable over time. This is achieved through a semantic search that identifies similar past requests and reuses the associated API calls and processes. As the system encounters more queries, the space of unknown tasks becomes smaller, requiring less processing time and resources.
• The system was developed and deployed in a remarkably short timeframe. The initial proof-of-concept was developed in just a few weeks, followed by a rapid iteration process involving beta testers and end users. This agile development approach enabled the NFL to get the system into production in August, just before the start of the season.
• The NFL is already considering expanding the system's capabilities and applications. This includes incorporating additional data sources (such as the NFL API) to answer a wider range of questions, expanding the scope of searchable data to include historical players, and potentially developing a fan-facing application.
• The NFL is using Amazon Web Services (AWS) cloud computing solutions, specifically for LLM they use Anthropic Claude 2, and has progressed to using Anthropic Cloude 3.5 throughout their project.
• The NFL has developed a system capable of searching for very specific game scenarios, like finding instances where a player has a less than 50% probability of completing a pass with a defender within one yard in the fourth quarter. This is possible due to the integration of the system with the NFL’s statistical data.
• The database used in this NFL system is updated overnight, so data from games is accessible for analysis by 5:30 or 6:00 AM the following morning. This highlights the rapid turnaround time for data processing and analysis.
• The system uses a semantic search approach but pre-processes queries to remove context-specific information, like team and player names, replacing them with generic terms like "team" and "player." This ensures that semantically similar queries, even if they involve different teams or players, are treated as the same, improving search accuracy.
• The NFL opted to use a pre-processing step to normalize team and player names instead of training a custom embedding model. This approach, they explain, avoids the complexity and time investment of training a specialized model.
• The user interface for this system was custom-developed in collaboration with UI developers at the NFL using React, while the backend operates as an API. This indicates a separation of concerns in the system's architecture.
• The NFL initially utilized Streamlit to create a quick prototype of the application to get it into the hands of users as quickly as possible. This demonstrates an agile development approach.