Updated: Jul 4
We love telling the story of our daily challenges and how we solved them.
Throughout 2020, we shared our experience and insights on engineering culture, architecture, scaling, mobile, frontend and backend and open source (yes, Covid-19 as well). All written by our very own Wix engineers. Here’s a quick recap of our 2020’s blog posts, podcasts and meetups -
This year we celebrated our 13,000 members on Meetup! - join us here
Continually Growing Your Engineering Skills While Scaling the Organization: Wix Engineering Culture
Wix is an engineering-focused company: our founders are engineers and about 50% of Wix’s staff is R&D. So to keep our engineering in best shape we spend 20% of work time enhancing our software engineering skills and building a strong, innovative and collaborative engineering community both within and outside of the organization.
How we do it, what we spend all that time on and the results we get (all during a global pandemic) - all told from the inside by the Server Guild Manager, Aviva Starovolsky-Peisach.
Helping Seniors During the Covid-19 Crisis
The Israeli Ministry of Finance and Welfare asked us to build a system to organize their outreach efforts to senior citizens and those in other at risk groups. Their volunteers would be calling to check on one million Israeli individuals: do they need food delivered; home assistance; any other form of help? The very next day we assembled a team.
In a week the system was ready for testing. And in another week, the system itself and both the user-facing website and app for volunteers was available to users. This is the story of how we built this web application so fast using Corvid. Oh, and don't miss the special podcast interview about it with Yoav Abrahami.
Contributing Code in the Age of COVID-19: How We Came Together For an Open Source Hackathon
3 hours and 35 participants equals 1,158 lines of code, 19 completed tasks and 21 pull requests. We transformed a traditionally offline Goodness Squad event - a meetup focused on contributing to open source projects - completely online utilizing some pretty cool tools and solutions. And yes, we even had an online afterparty to celebrate this event.
Want to get a taste and learn how we did it? Check out this article by Daria Tkhorevskaya.
International Women in Engineering Day 2020
On June 23 we marked the International Women in Engineering day, and this year we wanted to celebrate it by bringing you the stories of some of the women working as engineers across Wix Engineering R&D sites around the globe.
From ways to engage more young women to pursue careers in engineering to sharing best career advice and inspiration - Bringing the voices of our women-engineers with short and insightful interviews with Adi Marantz, Anastasiia Chepka, Brigita Pliška, Gal Sharir and Mariia Borovyk.
Meet Asaf Yonay, Head of FED Infrastructure at Wix Engineering
By his own accord, the three biggest challenges Asaf Yonay, Head of FED Infrastructure at Wix Engineering, faces are: Understanding and supporting the company vision, balancing support for existing products with creating new innovative tools, and Scaling up Wix, the FED guild, and the FED Infra group. Read the full interview with Asaf to learn how he's tackling all that, while also doing what he loves most.
Leading Server Software Infrastructure Group at Wix Engineering: Meet Yuval Perry
Here, at Wix Engineering, we’re always on the hunt for technical challenges and personal growth. So, it’s no wonder that our Server Guild Manager, Yuval Perry, joined us as a result of his own journey of microservices challenges.
We sat down with Yuval to hear more about his take on software infrastructure management, development velocity, propagating knowledge, and his strategy for dealing with technical debt. The result was an insightful interview. You can also hear Yuval's story here:
- Backend Engineering -
How We Implemented Idempotency in a Billing System at Scale
So you have a system processing hundreds of millions of dollars annually, and each transaction needs to go through smoothly and perfectly. Not to mention it has to happen exactly once.
At Wix Engineering we solved this with Idempotency and the exactly-once approach.
And created an infrastructure which allows developers to take any method, or function, and add to it idempotency or exactly-once semantics with near zero effort. Read Lior Asher's blog post to learn more.
How to Build 1,000 Microservices with Kafka and Thrive
In today's environment of globally distributed large scale web applications, reactive, event-driven design is mandatory. In this talk, Natan Silnitsky showcases how we implemented a reliable, scalable, event-driven, distributed Mega Microservices architecture on top of Kafka but without Kafka streams.
We, at Wix Engineering, have built powerful (soon-to-be-os) libraries for data streaming and data storage using Kafka features like compacted logs and exactly once semantics. These libraries offer low latency, high throughput event processing and also high volume data-storage with very simple and easy to understand APIs that work seamlessly together with other technologies such as gRPC.
Natan presents the challenges faced while creating and fine-tuning such a system. An architecture that is comprised of a 1,000 micro-services that need to handle incoming traffic from multiple data centers while preserving eventual consistency and high throughput:
Lessons-learned from Migrating our Build System to Bazel
The Bazel User Conference is an annual gathering of people passionate about the Bazel build system. It is the perfect place to get your dose of great tech talks, demos, and lightning talks presented by members of the Bazel community.
This time we had a lot to share: How we’ve made (and are making) local development with Bazel great, how we used Bazel to build a parallel and asynchronous artifact publishing pipeline, and 6 lessons we learned from migrating Wix build system to Bazel (with no downtime).
Watch our very own Ittai Zeidman, Or Shachar and Liron Tal share their insights on the stage of BazelCon 2019. You can also hear our Bazel-journey story here:
Principles on How to Build Fast at Scale
Three years ago, we were given a big challenge - Transform Wix Build System, then based on Maven and Teamcity, to a new system that will support our exponentially growing scale. We ended up choosing a build tool that originated at Google and completely revolutionized our builds.
Nowadays - the system builds in a fracture of the time - even with our largest codebases. In this talk, Or Shachar breaks down the key-principles that made the new build system so fast, how each one enables a faster, more efficient, and more stable build system, and touches upon how this was implemented using Bazel. For more engineering updates and insights:
Making Order in CI/CD Mess
CI/CD is a complex process involving code, services, and developers - stitching all these entities together is essential, especially when managing a huge scale CI/CD, like the one we have at Wix.
In this article Igal Harel describes the reasoning behind, and the challenges that lead us into building a CI/CD portal. He also explains how we implemented it and what the impact was of this unique portal on our engineering processes.
NPM Flattening for Zero Cold Start Serverless
Have you ever waited five or more minutes for 1Gb of NPM packages to download just to write three lines of code? We have. It doesn’t scale.
We built a freemium serverless platform and we wanted to let our users use NPM. We wanted to allow them to use any NPM package, with zero cold start while keeping sane download and deployment times while minding our storage. Now here’s the engineering challenge: The naive ‘npm install’ approach has several disadvantages. It’s slow, which prolongs deployment time.
It keeps the same package over and over again for different users which increases storage usage. It would force us to either have a long cold start if we choose to copy the node_modules when the container starts, or keep dedicated user containers online, which costs a lot.
In this talk, Schachar Levin unveils the solution we implemented to face this challenge and support complex npm, Inc. dependencies trees allowing multiple versions support, deduplicating packages, minimizing storage and download time, while making node able to import packages in a new and efficient way:
How (Not) To Add Deadlock To Your Critical Flow - 3 Simple Steps
The story of a Deadlock - How a library ended up causing a deadlock six months after it was released, and what a new feature that was recently developed by Wix, combined with java's ConcurrentHashMap, has to do with it. More on this and other lessons learned, alongside some practical advice, in this blog post by Guy Nahum.
How We’re Able to Host 1 Million Sites per MongoDB Cluster
MongoDB is a very capable database tool. But as with most things, there are strengths and weaknesses. In this article we explain how we got to discover one of its limitations and what we changed in our data model to compensate for it. This enabled us to host 1 million sites in a single non-shared MongoDB cluster. Read all about it in Giedrius Graževičius' article.
MySQL JSON Data Type and Migration
At Wix Engineering, we are using JSON in blobs Data Type as far back as MySQL 5.5. In this talk, Baruch Assif Osoveskiy shares both our wins and losses, our successes and our challenges, when it came to migrating our blob JSON data to JSON data types, and how we use the new data type to improve search time and reduce IO cost:
Working with NoSQL in a SQL Database
The X Dev API is a new protocol for non-blocking, asynchronous calls to MySQL. In this talk, Lior Altarescu explores the benefits of working with this protocol and connectors, and the challenges we encountered during the process of adopting X Dev API in Wix Engineering. We will share how we are incorporating the protocol in our massive multi dc architecture, and how it helps us, at Wix Engineering, rollout to production faster:
- Front-End Engineering -
Is Stencil a Better React?
A potential rival to React, a framework, capable of getting certain things done even faster, one that can make your coding life easier? Intrigued? So was Yoav Abrahami when he discovered Stencil - a React-inspired web component library. It is using the same JSX and some of the same concepts as React.
Yet, Stencil compiles to an optimal bundle, creating Virtual DOM that is consolidated directly into the DOM itself (no VDOM to VDOM comparison). The result is standard web components with optimal performance. In his article Yoav walks us through building the TodoMVC app from scratch with Stencil and dives deep into what the framework is capable of, comparing and contrasting it to React and Svelte. Read here
A Promise of a Bright Future With Async Iterators, Generators, and Pipes, Part 1
Tipping the Scale
Times Square is (at least before Covid-19...) the most visited place globally with about 380,000 pedestrian visitors a day - roughly the same amount of people who go to Wix’s dashboard. Building a page which receives such high traffic requires having great performance and intelligent architecture.
In this talk, Eyal Eizenberg goes over the challenges of rebuilding Wix’s dashboard, the architecture and how we got it to load under 1 second:
My Website is Slow, Now What?
You built a great-looking, feature-rich website. But it also has a big problem: it's just too slow. As a result, your visitors are frustrated, and your bounce-rate is high. So now, what do you do?
In this session, Dan Shappir explains what web performance actually means, and how to measure the performance of your website. Dan shows tools and techniques you can use to locate performance bottlenecks, and various methods and technologies you can use to overcome them. Mostly he provides a list of action-items for improving the performance of your websites:
In this blog post Dan explains his decision to use the pipeline operator, and the whole motivation to add this operator to the language. He then goes on to show how additional research indicates that this operator isn’t actually necessary, and that the same benefits can be achieved without it. Read here.
React Hooks: SOLID View
About a year ago, we were celebrating the release of the new React API. ‘Rewrite everything with hooks’ was pretty much the mainstream idea. The community was in awe - certain use cases seemed to have become much simpler, there were new patterns, and the approach to designing components was changing.
Yet, when the dust settled, we all had to face the truth: there are no silver bullets out there. Turns out, we’re paying a certain price for those improvements. Watch Roman Yakobchuk clears things out and compare the pros and cons:
Problem Solving Using the TypeScript Compiler
TypeScript’s type system is very powerful and can express very exciting things. This technical-yet-humorous talk is done almost entirely hands-on, in an editor, inside a .d.ts file. By using a certain amount of generics, recursive types, and conditional types, we code together.
Watch Gal Schlezinger reveal how to solve FizzBuzz in the type system itself, by getting the solution with no runtime code whatsoever.