This resource offers a comprehensive overview and technical guidance for setting up a movie summarization pipeline utilizing multiple AWS services, including the AWS suite of Large Language Models (LLMs) provided through Bedrock. Intended for solution architects, developers, and system administrators, it serves as a blueprint for creating and deploying a modern, efficient, and cost-effective solution tailored to process and summarize movie content.
Contents
Solution Overview
The pipeline process begins when a video file is uploaded to an Amazon S3 bucket and triggers a series of processes orchestrated by Amazon EventBridge and AWS Step Functions. The pipeline utilizes AWS Elemental MediaConvert and Amazon Rekognition for comprehensive video analysis and for extracting transcripts and contextual data from the video.
For users focused on reducing costs and processing time, a simplified version of the pipeline can also be deployed to streamline the process by focusing only on extracting transcripts from the video and excluding contextual data extraction.
The output of the pipeline is accessible through a user-facing API, built with Amazon API Gateway and AWS Lambda. The API Gateway provides endpoints to access lists of available movies and retrieve the summaries. The summarization of movies is performed using the Anthropic “Claude v2” model offered within Amazon Bedrock, which takes into account user-specified parameters for the summary language and the type of summary to be written. This allows for customization of the summary, such as not revealing the movie’s ending, including the actors’ names, or crafting the summary in an entertaining manner to encourage readers to watch the movie.
AWS Services Used
- Amazon S3 (Simple Storage Service): Provides scalable object storage for movie files. S3 is a highly durable and available solution for storing and retrieving any amount of data.
- Amazon EventBridge: A serverless event bus service that connects application data from various sources. In the pipeline, EventBridge triggers workflows in response to events such as movie uploads to S3.
- AWS Step Functions: Coordinates multiple AWS services into serverless workflows. In the pipeline, a Step Function is used to orchestrate tasks such as transcription extraction and video frame analysis.
- AWS Elemental MediaConvert: A file-based video transcoding service with broadcast-grade features. MediaConvert is used to process and convert video files (full pipeline version only).
- Amazon Rekognition: Offers video and image analysis using machine learning. Rekognition is used to analyze video frames for contextual data extraction (full pipeline version only).
- Amazon DynamoDB: A fast and flexible NoSQL database service for any scale. DynamoDB is used to store and retrieve processed data such as transcriptions and contextual information.
- AWS Lambda: A serverless compute service that runs code in response to events. Lambda is used to handle API requests and invoke summarization processes.
- Amazon API Gateway: A fully managed service that makes it easy to create, publish, maintain, monitor, and secure APIs. API Gateway provides the front-end interface for the summarization API.
- Amazon Bedrock: A fully managed service that simplifies the process of building and scaling AI applications using Foundation Models (FMs). Utilizing Anthropic’s Claude v2 model, Bedrock helps provide the core large language model (LLM) capabilities for sophisticated and nuanced movie summarization.
Movie Summarization Pipeline Solution Architecture
Architecture Diagram for the Complete Pipeline
Architecture Diagram for the Simplified Version of the Pipeline
The primary difference between the full and simplified versions of the pipeline are found in the AWS Step Function that handles video processing and transcription. The full version includes a media converter in the step function to process video frames, and uses Amazon Rekognition for extracting detailed contextual information from the video, enabling more nuanced summarizations. In contrast, the simplified pipeline, audio transcriptions are stored in an S3 bucket, with their paths noted in a DynamoDB table. When the summary creation API is called, it fetches the transcription from the bucket to create the summary. The simplified version efficiently delivers effective summaries more quickly and at a lower cost.
API Overview
This API provides access to the movie summarization pipeline, allowing users to list available movies and retrieve customized summaries. It is designed for easy integration and use, supporting multiple summary types and lengths.
List Movies
Endpoint: /list
Method: GET
Description: Retrieves a list of movies available for summarization.
Parameters: None
Response Format: JSON
Summarize Movie
Endpoint: /summarize/:id
Method: POST
Description: Returns a summary of the specified movie.
Parameters:
- id (path parameter): Unique identifier of the movie.
- type (body, optional): Type of summary (ex: Synopsis as default ).
- language (body: optional): Language of the summary (ex: english as default).
- custom (query, optional): Add a custom summary type following the format.
- Response Format: JSON
Summary types:
- Synopsis:
- Length: Medium to long
- Characteristics: Provides a complete overview of the plot, including the setting, main characters, and major developments of the story. May reveal the ending.
- Tagline:
- Length: Very short
- Characteristics: Designed to quickly capture attention. Often used in movie posters or DVD covers. Must be impactful and memorable.
- Back Cover Summary:
- Length: Short
- Characteristics: Gives an overview of the story without too much detail. Often used on the back of DVDs or in online catalogs. Should arouse interest without revealing key plot elements.
- Logline:
- Length: Very short
- Characteristics: Summarizes the plot by focusing on the central element or conflict of the film. Used to give a quick idea of the main story.
- Press Summary:
- Length: Medium
- Characteristics: Used in press kits and releases. May include information about the cast, director, and details about the film’s production.
- Festival Program Summary:
- Length: Short to medium.
- Characteristics: Used in film festival programs. Provides enough information about the plot to generate interest, while including details about the directors, actors, and sometimes artistic intentions.
- Streaming Platform Summary:
- Length: Short
- Characteristics: Must be concise and appealing to encourage instant viewing. Often focuses on the most intriguing or unique elements of the film.
Summary Examples for The Movie ‘Harry Potter and the Philosopher’s Stone’
Movie Summary in English
Movie Summary in French
A Streaming Platform-Friendly Summary of the Movie
A Kid-Friendly Summary of the Movie
Conclusion
The movie summarization pipeline outlined in this article highlights the potential for synergies between AWS services and advanced Large Language Models. Integrating Bedrock with the Anthropic “Claude v2” model has led to the creation of a scalable and efficient solution for summarizing movie content. This innovative system, utilizing AWS Elemental MediaConvert, Amazon Rekognition, and other services, presents a robust pipeline for extracting meaningful insights from video content.
With its user-friendly API interface and customizable summarization options, this technology unveils exciting possibilities for a future where accessing and enjoying movie summaries is both convenient and personalized.
About TrackIt
TrackIt is an international AWS cloud consulting, systems integration, and software development firm headquartered in Marina del Rey, CA.
We have built our reputation on helping media companies architect and implement cost-effective, reliable, and scalable Media & Entertainment workflows in the cloud. These include streaming and on-demand video solutions, media asset management, and archiving, incorporating the latest AI technology to build bespoke media solutions tailored to customer requirements.
Cloud-native software development is at the foundation of what we do. We specialize in Application Modernization, Containerization, Infrastructure as Code and event-driven serverless architectures by leveraging the latest AWS services. Along with our Managed Services offerings which provide 24/7 cloud infrastructure maintenance and support, we are able to provide complete solutions for the media industry.