Architecting for 6 Billion Daily Requests: Inside Wix's Media Platform
At Wix, "Media" isn't just about storing files. It's about processing streams of information - images, video, and audio - at an immense scale. We host over 300 million websites and serve more than 6 billion media requests every single day.
To handle dozens of petabytes of data while keeping costs low and latency minimal, we had to treat media handling as a complex engineering discipline. Below are five specific technical challenges we faced and the architectural solutions we implemented to solve them.
Challenge 1: The "Cgo" Bottleneck in Real-Time Encoding
The Problem: We recently migrated from WebP to AVIF (based on the AV1 video codec), which offers 50% better compression for the same quality. However, AVIF encoding is CPU-intensive.
Our image manipulation service is written in Go, but the high-performance encoding libraries (libaom/libavif) are written in C. Using Go's standard C interoperability (Cgo) to pass image data between two different C libraries (one for manipulation, one for encoding) was too slow. Every time data crossed the Go/C boundary, it incurred significant overhead due to memory copying and safety checks.
The Solution: Unsafe Pointers To achieve real-time performance, we bypassed standard memory management. We utilized Go’s unsafe package to pass raw C pointers (32-bit pointers) directly from the first C library to the second, tunneling strictly through Go without processing the data there. This required rewriting code flows to bypass existing library wrappers, but it eliminated the memory copy overhead, making the AVIF encoding viable for production traffic.
Challenge 2: CPU-Expensive Resizing Operations
The Problem: Downscaling high-resolution images is computationally expensive. Resizing a 4,000px image to a 200px thumbnail requires the CPU to average a massive amount of pixels. Doing this calculation from the raw source file for every request would crush our CPU resources and increase latency.
The Solution: The "Masters" Caching Strategy We implemented a tiered caching strategy called Masters. Instead of resizing from the original source every time, we generate constant intermediate sizes (e.g., 1000x1000, 500x500) stored in the cache. When a user requests a 450px image, the system automatically fetches the closest larger Master (the 500px version) and resizes from there. This minimizes the "pixel distance" the CPU needs to process, drastically reducing load while maintaining high quality.
Challenge 3: Cache Fragmentation and Duplication
The Problem: Our architecture relies heavily on CDNs. However, in an unopinionated platform where developers define manipulations via URL parameters, different parameter orders can theoretically produce the exact same image. For example, a request to crop then resize might result in the same pixel output as a different variation of those commands. If the URL is different, the CDN treats it as a cache miss, forcing our backend to re-process the same image unnecessarily.
The Solution: Request Normalization We implemented an internal normalization layer. Before processing, the system analyzes the requested operations and maps logically identical requests to a single, optimized canonical URL structure. This ensures that regardless of how the developer constructs the query, if the output is the same, we hit the same cache key.
Challenge 4: Validating Visual Quality at Scale
The Problem: When updating underlying encoder libraries the binary output of an image often changes slightly. This breaks standard regression tests that rely on byte-equality, even if the image looks perfect to the human eye.
The Solution: "Golden Eye" Testing with SSIM/VMAF We use a multi-stage testing approach:
1. Golden Tests: We maintain a baseline of "correct" images. Our primary tests check for byte-by-byte identity against these "Golden" files.
2. Algorithmic Fallback: If a byte-check fails (due to a library upgrade), we fall back to perceptual metrics like SSIM (Structural Similarity), PSNR, or Netflix’s VMAF. These algorithms mathematically simulate human vision to ensure the visual integrity is maintained, allowing us to deploy optimizations safely.
Challenge 5: Dynamic Access Control for Digital Assets
The Problem: Many Wix users (like photographers or artists on DeviantArt) sell digital content. They need to display a preview to the public but restrict the high-quality original to paying customers. Managing two separate files is inefficient and error-prone.
The Solution: Token-Based Manipulation We integrated authorization directly into the media manipulation pipeline. We generate signed tokens that carry metadata about allowed manipulations. A "free" token might only authorize the rendering of a blurred or low-res version of the image ID.
A "paid" token authorizes the full-resolution render. This allows us to serve secure, context-aware versions of the same asset dynamically without duplicating storage.
Watch the full podcast - "Wix Media Platform: The Engine Behind Your Media"
Yes, serving media for 300M+ websites and handling 6B+ requests a day completely changes how you think about images and videos. Watch Tomer Salton and Eli Ben-David open up Wix Media Platform behind the scenes - the systems that quietly serve images, videos, and audio at MASSIVE scale:
This post was written by Tomer Salton
More of Wix Engineering's updates and insights:
Join our Telegram channel
Visit us on GitHub
Subscribe to our YouTube channel
Comments