|
Developer-first security for your entire codebase (Sponsored)
 |
Ready to embrace secure coding without slowing down? SonarQube Advanced Security helps you find and fix quality and security issues across your entire codebase—from first-party and AI-generated code to your open source dependencies.
SonarQube addresses security challenges head on, offering a unified solution for:
Proactive vulnerability identification and management across all code
Streamlined compliance with SBOM generation and license policy management
Integrated code quality and code security solution for comprehensive code health, eliminating tool sprawl
Disclaimer: The details in this post have been derived from the official documentation shared online by the Meta Engineering Team. All credit for the technical details goes to the Meta Engineering Team. The links to the original articles and sources are present in the references section at the end of the post. We’ve attempted to analyze the details and provide our input about them. If you find any inaccuracies or omissions, please leave a comment, and we will do our best to fix them.
Meta AI’s “animate” feature turns a generated image into a short, dynamic video using AI.
It wasn’t a matter of simply introducing another creative effect, but an attempt at something far more complex: delivering video synthesis at a global scale and in near real time.
Billions of users might click “animate” on any given day, and each request comes with critical constraints. The system must produce results in just a few seconds, maintain a consistently high success rate, and operate within tight GPU budgets that leave little room for waste. What looks like a playful feature in the product UI is a carefully engineered balancing act between cutting-edge AI research and large-scale distributed systems.
The challenge is twofold:
First, the underlying diffusion models must be optimized to run fast enough for interactive use without sacrificing quality
Second, the service must be deployed across Meta’s global infrastructure in a way that minimizes latency, contains failures, and scales to meet demand spikes without degrading user experience.
These two domains (model/inference-level acceleration and planet-scale traffic engineering) form the backbone of Meta’s solution. In this article, we will look at how Meta implemented this feature and the challenges they faced.
 |
Help us Make ByteByteGo Newsletter Better
TL:DR: Take this 2-minute survey so I can learn more about who you are,. what you do, and how I can improve ByteByteGo
Model and Inference Optimizations
The “animate” feature is powered by diffusion models, which normally take a lot of steps and computation to produce good results.
For reference, a diffusion model is a type of AI that creates images (or videos) by starting with pure random noise and then gradually “cleaning” it step by step until a clear picture appears. It learns how to do this by studying lots of real images and figuring out patterns. Think of it like an artist who begins with a messy canvas and slowly refines it.
To make animation practical for billions of users, Meta had to speed up the models without lowering quality. They achieved this with several optimizations as follows:
1 - Half-precision with bfloat16
Most AI models run in 32-bit floating point (FP32), which uses a lot of memory and processing power. FP32 has a wide range and fine-grained precision, which helps when weights or gradients are very large or very small.
Meta switched to a lighter format called bfloat16 (a type of 16-bit precision) for both training and inference. Unlike FP16, it keeps the same range as FP32 but sacrifices some precision. This means it can represent very large and very small numbers safely. It cuts the memory requirement in half and allows GPUs to process data faster. The model’s accuracy remains high, but performance improves immediately.
 |
2 - More Efficient Temporal Attention
When creating a video from images, the model needs to understand how frames relate to each other. Normally, this involves copying the same context information across all frames before running attention layers, which wastes memory and compute.
Meta changed the process so that the copying happens after the linear projection step in cross-attention. Since the context is identical, delaying expansion avoids unnecessary work and lowers the memory footprint.
3 - Fewer Sampling Steps with DPM-Solver
Diffusion models usually need to run through many tiny steps to turn random noise into a clear picture or video.
The more steps we use, the better the result, but it takes longer. Meta used a faster method called DPM-Solver, which can get almost the same quality in far fewer steps (about 15 instead of dozens or even hundreds).
This makes the process much quicker while still keeping the output sharp.