Amazon Web Services has made TwelveLabs Marengo Embed 3.0 generally available as an embedding option in Amazon Bedrock Knowledge Bases, giving developers a managed way to search video, audio and images with natural-language queries. Customers can now build media retrieval applications without separately assembling frame extraction, transcription, embedding, vector storage and synchronization components.
Marengo Embed 3.0 jointly represents video, audio, images and text in a 512-dimensional vector space. Within a managed knowledge base, AWS says the ingestion process captures visual, textual, speech and audio signals in unified embeddings, while Bedrock handles storage, ingestion, reranking and retrieval.
How the managed workflow operates
AWS documented the setup using a 10-minute clip of the 2022 FIFA World Cup final. The prerequisites include an AWS account, access to Amazon Bedrock with Marengo Embed 3.0 enabled in a supported Region, an Amazon S3 bucket, and the necessary Bedrock and S3 permissions through AWS Identity and Access Management.
Media files are first uploaded to S3. According to AWS, no preprocessing is required because the managed service performs segmentation, frame sampling and transcription internally. Bedrock Knowledge Bases supports MP4 and MOV video, JPEG and PNG images, and audio tracks; AWS also lists S3, SharePoint and Confluence among its native data connectors.
In the Bedrock console, users create a Managed KB, replace the default Amazon Titan Text embedding selection with TwelveLabs/Marengo Embed 3.0, and connect the S3 location containing their media. Audio and video segmentation durations are configurable, with both set to four seconds by default in the walkthrough.
Starting a sync causes the service to extract frames, transcribe audio, create a Marengo embedding for each segment and write the resulting vectors to the index. This consolidates the media-processing pipeline, but developers still need to choose segmentation settings appropriate to their material and retrieval needs.
Searching and application integration
The console’s Test feature lets developers choose how many source chunks to retrieve, apply metadata filters and submit semantic queries. In AWS’s soccer example, a request for penalty kicks returned ranked moments where kicks were attempted. Each result included the chunk’s start and end times, source URI and embedding type, providing the information needed to locate and extract a relevant passage from the footage.
For production applications, developers can query the knowledge base with the Amazon Bedrock Retrieve API through the Boto SDK. AWS also describes using the knowledge base as an Amazon Bedrock Gateway target in Bedrock AgentCore. Potential uses identified by AWS include locating plays in sports footage, searching media archives, finding incidents in security video, retrieving lecture segments by concept and locating feature demonstrations in retail videos.
Availability and costs
Managed Knowledge Bases with Marengo Embed 3.0 is available in US East (N. Virginia), identified as us-east-1, and US West (N. California), identified as us-west-1. AWS says Managed Knowledge Bases charges for stored and retrieved data, while Marengo embedding generation is billed at the standard Amazon Bedrock model invocation rate.
Further implementation details are available in the Amazon Bedrock Knowledge Bases documentation and the Amazon Bedrock samples repository.
Source: AWS Machine Learning Blog
Definition. Marengo Embed 3.0 is a multimodal embedding option that represents video, audio, images and text in a shared 512-dimensional vector space within Amazon Bedrock Knowledge Bases.
| Media type | Supported formats or handling |
|---|---|
| Video | MP4 and MOV; managed segmentation, frame sampling and transcription |
| Images | JPEG and PNG |
| Audio | Audio tracks with managed segmentation and transcription |
| Text | Represented with other modalities in the shared vector space |
Key takeaways
- Marengo Embed 3.0 is generally available as an embedding option in Amazon Bedrock Knowledge Bases.
- The model represents video, audio, images and text in a unified 512-dimensional vector space.
- The managed workflow performs segmentation, frame sampling and transcription without separate preprocessing.
- Developers can retrieve ranked media segments through the Bedrock console or the Retrieve API using the Boto SDK.
- Availability is limited to us-east-1 and us-west-1.
- Storage and retrieval charges are separate from Marengo embedding-generation charges.
FAQ
What can Marengo Embed 3.0 search in Amazon Bedrock Knowledge Bases?
It supports semantic search across video, audio, images and text using natural-language queries.
Does media require preprocessing before ingestion?
According to AWS, no separate preprocessing is required because the managed service performs segmentation, frame sampling and transcription internally.
Which media formats are supported?
Bedrock Knowledge Bases supports MP4 and MOV video, JPEG and PNG images, and audio tracks.
How can production applications query the knowledge base?
Applications can use the Amazon Bedrock Retrieve API through the Boto SDK, or use the knowledge base as an Amazon Bedrock Gateway target in Bedrock AgentCore.
Where is the Marengo-managed knowledge base option available?
It is available in US East (N. Virginia), us-east-1, and US West (N. California), us-west-1.
How is the service billed?
Managed Knowledge Bases charges for stored and retrieved data, while Marengo embedding generation is billed at the standard Amazon Bedrock model invocation rate.
