When we needed a company-wide toolkit, the front-end infrastructure team decided to build it openly, letting developers around the company participate in its development, similar to the way the open source works. Hundreds took up the task. Was it chaos, or a new and better way of developing software?
Listen to the full story with Ran Yitzhaki.
You can also listen to this episode on Apple podcast, Spotify, Google or on Wix Engineering site. And you can also read the full episode here:
INTRO: DEMOCRACY
Hi, I’m Ran Levi, welcome to the Wix Engineering podcast.
It seems so natural to us today but, in early human history, the concept of representative democracy would’ve been utterly inconceivable. In fact, even for centuries after it was invented, most people on Earth wouldn’t have known anything other than monarchy. If you tried explaining the concept of representative government to any one of them, you might’ve received some strange looks.
Traditional power structures just come so naturally to us. You’ve got one leader--a king, a god, a CEO--then their inner circle--the royals, the apostles, the executive board--then you go down the chain until you reach the average Joe Schmos. Even today, most organizations of most kinds are shaped like this.
But it’s interesting to wonder what life would be like if that weren’t the case. For example, if religions worked like democracies. Or companies did. Or even individual teams.
This podcast is about a kind of experiment to that end. It’s about an engineering project which, instead of operating like you would expect--with a leader, a team, and assigned roles and tasks--it was totally in the open. Hundreds of people were allowed to take part, if they wanted to - and they did. Kind of like a representative government.
It sounds like it wouldn’t work. But that’s what people thought about democracy, too, until it did.
…
IMPETUS FOR YOSHI
Ran: So my name is Ran and I’m a team leader at the Frontend Infrastructure Group at Wix.
Recently, Ran Yitzhaki took part in an infrastructure project that would impact every developer at his company. To follow along, you first have to understand how Wix is built.
Ran: So Wix had a micro frontend architecture in which every application needed to be developed on its own Git Repository and each app had its own build script.
Basically, instead of one, big app--one codebase where everything is mushed together--Wix utilizes a microfrontends architecture, where thousands of smaller components combine together, but are built independently of one another. There are all kinds of benefits to this model, but one of the drawbacks is that it requires developers to spend much more time on the build process (basically, the infrastructure side of their code).
Ran: The developers just invested a lot of time in dealing with these build-related problems and the opportunity cost of such thing is very high - because this means that they cannot use this time to build products for our users. They actually deal with the same problems over and over.
It takes a lot of time for developers to master and maintain the infrastructure in each of their projects. Wouldn’t it be convenient if, instead of having individual developers do such repetitive, redundant work, they had a tool to do it automatically?
Ran: To just centralize all of these problems into a single place.
That was the impetus for the new project, called “Yoshi.”
YOSHI EXPLAINED
Ran: Yoshi is a library, which we also call a toolkit. It consists of all of the build constraints such as building the applications for production, running a local dev server, that includes incremental changes and AGMR and it includes running the tests and running the linter. So all of these things are an integral part of the development environment of the user and now that we put all of these constraints in a single library, the developers do not need to deal with them.
Or, put simply:
Ran: We moved all of the build constraints from the developers to the developers of Yoshi.
A feature-packed toolkit, taking the load off developers so they could spend less time on builds and more time writing better code. It sounds good, in theory. Here’s the problem:
NEED TO PLEASE EVERYONE
Have you ever tried to design anything for any large-enough group of people? It’s impossible. I don’t care what you’re doing: toolkits, podcasts, you name it. If you’re a parent, you know what I mean. Even getting two kids and a spouse to agree on dinner is a daily struggle. People just have their own preferences, which usually collide with most other people’s preferences.
Developers are no different. As a developer myself, I can confirm: in many ways, we’re like children.
Ran: We understood that many developers have their own opinions regarding how these tools should work. Developers can be very opinionated about what their tooling are doing.
Even within the same company, where everybody was working on the same apps, Ran and his colleagues knew that they would have a tough time pleasing everyone. They needed a way to figure out what people would like and, more importantly, what they would agree on.
Ran: So we wanted to be completely transparent regarding how we do stuff and our own decision-making process. We wanted to introduce new features before we actually create them and listen to the feedback and also having the other developers in the company being able to contribute to us.
Every new feature, every little change--open for anyone to see, comment on, or contribute to themselves. Call it democracy. Or, by its proper name:
Ran: “internal open source”.
Interviewer: Can you describe what internal open source means? Because it kind of seems like a contradiction, right? Open source means that it’s not internal but in this case it is.
Ran: Right. So in open source, your users or the people that can develop and use the product are the entire world. But in an internal open source, the users are the developers in Wix. So in fact we had a smaller community of around 700 developers which also used the product and contributed to it.
HOW TO DO INTERNAL OSS
So, how can you create a project where up to 700 people can be involved? Complicated though Yoshi was, this was the tougher problem to solve. As Ran and his colleagues opened the project to everyone, they laid out some ground rules for how it would work.
Ran: We use GitHub issues for how to handle our own tasks and we also used PRs for every new change.
“PRs” means “pull requests.”
Ran: So every new change was introduced with a PR that explains exactly why we’re doing this change and, you know, we’re starting by explaining the why of this change and then describing what we’re trying to do and how we’re going to test it of course.
This is very important if you try to create something that you want other developers to be able to understand why you made some decisions just by going through the code and reading why a PR was added.
In addition to the log of pull requests…
Ran: We started a process of creating big features using RFCs, which are request for comments, when we publish what we want to do, describing it in a paper before starting to develop it and then hearing a lot of feedback from people and changing the product accordingly.
We also introduced big changes with blog posts. If we release a new version, we also have a blog and this blog is completely internal in the company.
TOWN HALLS
Even the boring updates got their own blog post. On top of that…
Ran: Pre-corona, we used to go to different sites and sit with users (our users are Wix developers of course) and just listen to what they want us to do. And that didn’t scale well with the corona because we couldn’t really move between sites and we started this kind of doing townhall event in Zoom.
Interviewer: What do these townhalls look like? How many people are participating on the calls? Is it just a bunch of developers talking about really high level technical stuff?
Ran: So each of those townhalls usually has around 100 developers and it’s all recorded. So other people can listen to it in the future. But yeah, it’s a very technical discussion, we’re showing all of the new features, what are plans for the future and they get to ask us questions and suggest what they want us to do. Because again, if we don’t listen to them, we don’t understand what we should do next and this is a critical part.
ENCOURAGING PARTICIPATION
With all these systems in place, the Yoshi team was capable of incorporating as many developers as possible into the building process. But they still had one more problem to solve.
Interviewer: How is it that Wix developers who already have things to do have time to get on townhalls and have time to develop for the toolkit?
Ran: That’s an amazing question and this is actually a problem that we had because we wanted to get this meaningful contribution. And I think that we mostly got people that like fixing small stuff.
Developers could offer minor help here and there, but the project required bigger, more meaningful contributions from the community. That required time and energy from people who didn’t have much of it to spare.
Ran: We couldn’t get the real features, like real meaningful features, and for that, they needed to invest a few days, maybe a week or two.
There was only one time of the month when anybody could make that kind of commitment.
Ran: We decided to use the concept of a guild week.
During guild week at Wix, developers are free to work outside of the usual bounds of their jobs, with developers and teams they’re not otherwise a part of. It was during those weeks that some of the best features were developed for Yoshi.
Ran: So actually there’s one story from around a month ago. We have a Slack channel - when we talk about stuff, and some developer asked for a feature and he said “It would be so cool if you can do that”. In that case, it was dynamic routing, filesystem-based dynamic routing.
It’s technical stuff, but you don’t need to understand it fully to get the gist.
Ran: He just wanted to put a specific file that means that you can have a dynamic route and then we just answered, “We don’t have this feature yet. But if you want to join us for a guild week and contribute it, that would be wonderful,” and next week, he joined us for a week and delivered this feature and I think it was super cool because it wasn’t an easy feature at all.
Over time, more and more developers from around the company started to make contributions just like this.
Ran: So this project is up for around four years and right now there’s around 160 developers or 160 contributors in total.
PUSHBACK
As you can imagine, with 160 contributors and hundreds more making comments and suggestions, there were a few conflicts to be worked out. Rarely in a democracy does everything go over perfectly smoothly.
Ran: So I will take one example. I think that a few months ago - our company is moving to be totally typescript. We used to work with some JavaScript and some typescript but now we’re trying to focus on typescript.
A simple change: instead of two programming languages for developers to choose from, just one. But, of course…
Ran: What we get is an angry response from the community, from the people that actually wanted to use JavaScript. We got some people opening issues about that and we needed to explain all of the reasons that we had at the time.
The reason for eliminating Javascript was simple: it was more efficient and cost-effective for the company if everyone was on the same page, using the same syntax.
Ran: If we need to maintain two different templates for JavaScript and typescript, it means that we can deliver less features. We also outlined the benefit of having everyone at Wix using the same syntax.
So we try to make these kinds of decisions from the most pragmatic point of, you know, looking at everything in the company. And we also showed data.
Ran and his colleagues showed the Javascript defenders what was really going on: that 90% of their colleagues were already, by their own volition, using typescript. The efficiency they could gain from eliminating Javascript would outweigh the inconvenience to that other 10% of developers. It wasn’t ideal for everyone, maybe, but it only made logical sense for the company.
After some debating…
This person that we were discussing the matter with just said, “Right. I’m convinced. This is – I understand now”.
YOSHI COMPLETED
After four years, by the hands of hundreds of contributors, through thousands of comments and a few spirited debates, Yoshi began to come to fruition.
Ran: So today, Yoshi is not a single toolkit that knows how to build everything. We actually have different development flows. We have development flows for editor applications and one for full stack applications and one for business manager applications.
Each development flow has its own team that takes care of creating the best developer experience for this specific use case and today you just put a react component as a file and you can just start – get up and running.
There’s little doubt that without their democratic system of developing the project, it would’ve turned out quite a bit different. Probably for the worse.
Ran: Since I've been doing it for the last four years, I can’t really imagine a different way to build such a product. I sometimes see products that are being built in that way and I’m talking specifically about developer-facing products and you can easily see that products that listen to the users and are completely open in the way that people write everything enables you to just achieve more with less people.
So in fact, I would say that the capacity of the team is much larger since we have the community with us.
TAKEAWAY
They say that two heads are better than one. In Yoshi’s case, 700 heads were better than five or six. By using an internal open source model, Ran and his initial team didn’t have to come up with every solution--they didn’t have to do everything right or risk disappointing anyone with the final result. Instead, they got help at every step of the way from people who had expertise, motivation and creativity to offer.
That doesn’t mean every project should be internal open source. But it’s an alternative worth considering.
Ran: If we talk about internal open source and you feel that you have a big enough community, which you can benefit from and you’re building tools for developers, you can definitely use this model.
It is different, granted. Sometimes, the various external pressures from the many contributors can lead to compromises which are less than ideal. Also, If you’re used to the comfort of being in charge--of being amongst only your own people--opening up to so many strangers can be difficult. Suddenly, your mistakes have so much new weight. Like a good politician you’ll need patience, and a strong stomach.
Ran: So you need to be ready to make mistakes. You need to be able to get feedback and criticism outside when everybody can see. And if you’re willing to do that, you can grow really fast as a developer and as a person.
For almost all of human history democracy was an alien concept. In the past few hundred years, we’ve come to realize just how useful it is to have lots of voices in the room. How much better things are when all kinds of different people are around to offer expertise, commitment and creativity.
The next time you’re building something--whether it be a government, a company, or a piece of software--you might consider democracy for yourself.
That’s it for this episode, thank you for listening. For a full list of our previous episodes - visit wix.engineering/podacst. The Wix Engineering Podcast is produced by PI Media - written by Nate Nelson, produced by Yotam Halachmi and narrated and edited by me, Ran Levi. Special thanks to Moard Stern from Wix. See you again next episode, bye bye.
For more engineering updates and insights:
Join our Telegram channel
Visit us on GitHub
Subscribe to our YouTube channel