Wix engineering blog tells the story of our daily challenges and our way of solving them. Read about architecture, scaling, mobile and web development.
1. Introduction: The Challenge Understanding the engineering need for AirBot starts with recognizing the scale of the ecosystem it operates within. Wix operates a massive cloud-based development platform serving 250 million users. This generates over 4 billion HTTP transactions daily, feeding a data lake that currently holds 7 petabytes of data. To manage this volume, Wix Data Engineering maintains over 3,500 Apache Airflow pipelines (DAGs) . These pipelines handle everyth
