Contents
- Link to Previous Volume
- Broadcast and Live Remote Production
- Hybrid Broadcast and Live Production Solutions
- Generating Closed Captions for Live Streams
- Adding Captions to Amazon IVS Live-to-VOD
- Adding Captions to Amazon IVS Live Streams
- vMix on AWS
- Solution Overview: Terraformed vMix on AWS
- Working with an AWS M&E Partner Integrator
Link to Previous Volume
Volume 1: AI/ML and Data Analytics
Broadcast and Live Remote Production
AWS, with its robust suite of cloud and hybrid solutions, provides head-on solutions to the challenges associated with Broadcast and Live Remote Production. These challenges include the need for high-quality, low-latency video transmission over unreliable networks, the complexity of managing remote production workflows, the demand for scalable and cost-effective infrastructure, and the increasing pressure to deliver real-time insights to enhance viewer engagement.
AWS offers a comprehensive set of tools and services that streamline remote production workflows, enabling efficient content creation from diverse locations. Its scalable and cost-effective cloud resources alleviate the burden of managing on-premises infrastructure, while AI and analytics capabilities empower broadcasters to extract real-time insights, elevating the overall quality and impact of live broadcasts.
Hybrid Broadcast and Live Production Solutions
Hybrid broadcast and live production workflows can be effectively established using three key AWS services: AWS Elemental Live, AWS Elemental MediaLive, and AWS Elemental Link. AWS Elemental Live, an on-premises video encoding appliance enables broadcasters to create live outputs for broadcast and streaming delivery. AWS Elemental MediaLive, a cloud-based service, complements this by facilitating the encoding and packaging of live streams with high reliability and scalability, offering a flexible cloud-based solution for content delivery. Lastly, AWS Elemental Link provides a reliable and cost-effective means to transport live video signals from on-premises locations to the cloud, enhancing the agility and versatility of the overall hybrid production workflow.
Generating Closed Captions for Live Streams
Closed Captions are essential to ensure accessibility and inclusivity for individuals with hearing impairments. Captions also are important when audio is unavailable or not clearly audible. Unfortunately, captioning audio in live streams can be tricky. The following workflow describes the process of adding captions to Amazon IVS Live-to-VOD and Amazon IVS Live Streams
Adding Captions to Amazon IVS Live-to-VOD
Subtitles can be generated for Live to VOD (DVR) using Amazon IVS, Amazon S3, the auto-record-to-s3 feature, Amazon EventBridge, and Amazon Transcribe.
Live-to-VOD Closed Caption Generation
Workflow Overview
When recording begins, video segments and metadata files are stored in the designated S3 bucket configured for the channel. These files can be later accessed for post-processing or on-demand video playback.
As illustrated in the diagram below, when video segments are added to Amazon S3, it triggers a state machine. This in turn starts a MediaConvert Job, which extracts an MPEG-III (MP3) audio file using the m3u master playlist file from Amazon IVS. After this process is complete, a Transcribe Job is employed to generate WebVTT files using speech recognition.
State Machine Diagram
Amazon Simple Notification Service (SNS) is utilized to send a notification promptly upon the completion of subtitle generation.
SNS Notification
Adding Captions to Amazon IVS Live Streams
The example above described the process of generating captions for assets coming from Live-to-VOD. However, Amazon IVS can also be used in conjunction with Amazon Transcribe to deliver real-time captions for live streams.
Workflow Overview
In this setup, Nginx runs on Amazon ECS and acts as a stream service instead of directly streaming to Amazon IVS. A validation server, also running on Amazon ECS, is responsible for checking streaming keys.
Nginx forwards the RTMP live feed source to Amazon IVS and an additional service (also running on AWS ECS) for transcription and translation (translate server and and transcribe server on the diagram above).
For transcription, the system uses ffmpeg to extract audio from video streams. This audio is then processed by a Node.js script, which connects to Amazon Transcribe to convert the spoken words into text. The resulting transcriptions are sent via WebSocket to the IVS player.
Additionally, the Node.js script connects to a DynamoDB table to retrieve overlay configuration. This configuration is sent along with the transcriptions using Amazon IVS Timed Metadata, which allows images related to what is being said to be displayed on top of the video player.
Generating Closed Captions for a live stream using Amazon IVS and Trascribe
vMix on AWS
vMix is a professional live video production and streaming software that allows users to mix multiple video sources, apply transitions and effects, add graphics, and stream live to various platforms. It serves as a versatile and feature-rich live video production software, facilitating the creation of professional live productions with ease.
Ease of Provisioning Infrastructure
One notable benefit of utilizing vMix on AWS is the remarkable ease with which infrastructure can be provisioned and de-provisioned. This feature is particularly valuable in the context of live video production, where rapid turnarounds are often required. This agility can be particularly advantageous when new productions show up unexpectedly, as the infrastructure can be seamlessly scaled up to accommodate additional requirements without causing disruptions.
Integration with AWS Services
The comprehensive suite of integrated services offered by AWS further amplifies vMix’s functionality. Amazon CloudFront aids in content delivery, ensuring that the produced content reaches viewers with low latency and high performance. Amazon S3 offers secure and durable storage for media assets, simplifying content management. The AWS Elemental family provides a suite of tools and services designed specifically for video processing and delivery, enhancing the quality and reliability of video streams.
Reduced Carbon Footprint
The solution aligns harmoniously with the Sustainability Pillar of the AWS Well-Architected Framework by significantly reducing the carbon footprint associated with traditional live video production practices. Historically, live video productions necessitated the transportation of substantial hardware and often required the deployment of outside broadcasting (OB) vans to event locations. This entailed not only the consumption of considerable fuel resources but also contributed to air emissions and increased road congestion.
With vMix on AWS, the need for physical hardware transport and OB vans is substantially diminished. Instead, the infrastructure resides in the cloud, allowing video production teams to harness its power remotely. This transformative shift minimizes the carbon emissions attributed to travel and logistics, as operators no longer need to commute to event locations with cumbersome equipment. Consequently, there is a notable reduction in the overall environmental impact associated with live video production.
vMix Key Features
- Multiple Inputs: Offers compatibility with a diverse array of video and audio inputs, ensuring flexibility in capturing various sources for live productions.
- Real-time Mixing and Effects: Facilitates the creation of complex multi-camera productions, smooth transitions, overlays, and other visual effects, elevating the quality of live video content.
- Live Streaming: Enables live streaming, allowing the broadcast of productions to a global audience in real time.
- Recording and Playback: Allows recording live productions for future use or playback, providing content longevity and reusability.
- Advanced Audio Mixing: Offers advanced audio mixing tools, ensuring optimal sound quality and balance for a professional live production experience.
Solution Overview: Terraformed vMix on AWS
vMix on AWS – Architecture Diagram
The solution illustrated in the architecture diagram above offers a comprehensive platform for live video production and streaming. It provides an integrated ecosystem that seamlessly incorporates AWS Media Services, enabling a smooth transition from live streaming to Video-on-Demand (VOD) content. The solution stands out for its scalability, flexibility, and customization capabilities, allowing it to adapt to a diverse range of workflow requirements.
AWS Services Used
- Amazon Elastic Compute Cloud (EC2): Provides the compute resources necessary to run vMix software and manage the live video production process on the AWS cloud.
- Amazon S3: Serves as a storage solution for vMix, storing recorded content, media assets, and other data generated during live video productions.
- AWS Elemental MediaLive: Used for encoding and compressing live video streams, ensuring efficient delivery and compatibility with various devices and platforms.
- AWS Elemental MediaPackage: Offers secure and reliable packaging and protection of live video streams, optimizing them for distribution to different endpoints.
- AWS Elemental MediaConvert: Handles the conversion of live streams into Video on Demand (VOD) content, supporting the transition from live to on-demand video.
- AWS Lambda: Leveraged for serverless, event-driven tasks within the vMix workflow, such as automated content processing or scaling based on demand.
- Amazon CloudFront: Serves as a Content Delivery Network (CDN) to efficiently distribute vMix-produced content globally, reducing latency and enhancing viewer experiences.
- Amazon API Gateway: Serves as the gateway to manage API requests, facilitating communication between vMix and external systems for enhanced control, automation, and extensibility of live video production workflows.
Use Cases
Live Events: vMix excels in covering live events by seamlessly integrating various audio and video sources, enabling real-time mixing, transitions, and effects. Events such as concerts, sports matches, or corporate conferences are ideal scenarios in which vMix can be leveraged to deliver high-quality live streams.
Webcasting and Webinars: A valuable tool for webcasting and webinars, vMix offers features such as multi-camera support, overlays, and live streaming capabilities to ensure a professional and engaging presentation. vMix is particularly useful for educational webinars, product launches, or any content requiring interactive online broadcasting.
Live-to-VOD Workflows: When used in conjunction with AWS Media Services, vMix facilitates the transition from live streams to Video-on-Demand (VOD) content. By capturing and storing content in real time, it enables the creation of on-demand content libraries, making it ideal for platforms requiring both live and archived video content.
Solution Benefits
- Scalability: The solution dynamically adapts computing resources to cater to high-traffic events, ensuring uninterrupted live video production.
- Flexibility: Flexibility in instance type selection based on CPU, memory, and GPU capabilities helps tailor the vMix workflow to meet specific performance requirements.
- Reliability: AWS offers a highly reliable infrastructure with inherent redundancy and fault-tolerant features, guaranteeing consistent and dependable live video streaming experiences.
- Global Reach: Leveraging AWS’s extensive global infrastructure, the solution allows the deployment of vMix instances in regions worldwide, ensuring a wide-reaching audience for live productions. CloudFront strategically uses Edge locations (distributed data centers closer to end-users) to reduce bandwidth consumption. This leads to reduced data transfer costs and reduced latency for end-users.
- Cost Optimization: Aligning resource usage with fluctuations in demand helps achieve cost efficiency. Expenses can further be reduced through the strategic utilization of Reserved Instances.
- Integration with AWS Services: Seamlessly integrates with other AWS services, enabling the creation of customized workflows that enhance the overall efficiency and functionality of live video production setups.
Working with an AWS M&E Partner Integrator
AWS M&E partner integrators can assist in navigating the various challenges associated with the implementation and maintenance of cloud-based media workflows on AWS.
With deep expertise in AWS, M&E partners not only offer guidance on best practices and migration strategies but also provide comprehensive managed services including monitoring, support, and optimization, allowing media companies to focus on their core business while outsourcing the technicalities of cloud management to experts.
M&E partners also play a crucial role in facilitating the relationship between AWS and end-users. Partners have extensive knowledge of AWS-based media workflows and can help clients identify funding opportunities provided by AWS. These funding programs help offset the costs associated with migrating to the cloud, making it more accessible for companies operating with stringent budgets.
Process of Engagement
The basic process of engagement typically involves an initial consultation to understand client needs and objectives, followed by a design phase to develop a customized solution. Implementation and testing are then carried out, followed by ongoing support and maintenance to ensure that solutions remain up-to-date and effective.
CloudWise – AWS Managed Services
CloudWise, TrackIt’s AWS Managed Services offering includes a suite of services such as monitoring, optimization, and support, enabling companies to manage their AWS infrastructure with ease and efficiency. CloudWise allows customers to stay focused on their core business while TrackIt experts handle all the technicalities of cloud infrastructure management.
Built on in-house custom monitoring software, the offering includes real-time monitoring, customized dashboards, monthly cost analysis and coverage reports, annual architecture reviews, and quarterly security assessments. Customers also benefit from 24/7/365 global support from AWS-certified TrackIt engineers working to ensure that their cloud investments are optimized to their full potential.
Managed Services Offering Provided by AWS M&E Partner TrackIt
About TrackIt
TrackIt is an international AWS cloud consulting, systems integration, and software development firm headquartered in Marina del Rey, CA.
We have built our reputation on helping media companies architect and implement cost-effective, reliable, and scalable Media & Entertainment workflows in the cloud. These include streaming and on-demand video solutions, media asset management, and archiving, incorporating the latest AI technology to build bespoke media solutions tailored to customer requirements.
Cloud-native software development is at the foundation of what we do. We specialize in Application Modernization, Containerization, Infrastructure as Code and event-driven serverless architectures by leveraging the latest AWS services. Along with our Managed Services offerings which provide 24/7 cloud infrastructure maintenance and support, we are able to provide complete solutions for the media industry.