1. Introduction: The Challenge Understanding the engineering need for AirBot starts with recognizing the scale of the ecosystem it operates within. Wix operates a massive cloud-based development platform serving 250 million users. This generates over 4 billion HTTP transactions daily, feeding a data lake that currently holds 7 petabytes of data. To manage this volume, Wix Data Engineering maintains over 3,500 Apache Airflow pipelines (DAGs) . These pipelines handle everyth

A Year of Building at Scale: Inside Wix Engineering During 2025, Wix Engineering shared deep dives, playbooks, and real-world lessons from building and operating one of the largest web platforms in the world. These stories don't start as content. They start as engineering challenges - some of the most complex, high-scale, and unforgiving challenges modern software teams face. Problems that emerge only when thousands of developers, hundreds of teams, and hundreds of millions o

Imagine starting your day as a senior engineering manager at a fast growing tech company. You’re barely through your first coffee when your phone lights up with a critical ticket. The Kubernetes environments don’t match across stages. DevOps is in place. CI/CD, automation, monitoring, on-call rotations. And still, developers spend more time fighting infrastructure than building products. This is the moment many organizations mistakenly try to "add more DevOps." In reality, th

Packed with engineering insights, developer stories, and tools to help you build better, faster, and smarter. Don't miss it, subscribe...

Hi, I’m Ben Chen, an Engineering Group Manager at Wix. Over the years, my team and I have faced countless challenges managing one of the...

Introduction In the first part of this article (Introducing PlatySpark: How Wix Built the Ultimate Spark-as-a-Service Platform - Part 1)...

The Challenge Managing Apache Spark applications at scale can be a daunting task, especially for large organizations with multiple data...