It’s not every day that we get to see the inner workings of such a fundamental tool like a bundler. These workhorses help to bridge the gap between writing modular code that is easy to maintain and still end up with modules that run smoothly in the browser.
For you to get the gist of how bundler works, our very own, Ronen Amiel (@ronenamiel) goes a step further than just taking a look under the hood. Instead, in his lecture at You Gotta Love Frontend conference, Kyiv, he actually codes from scratch a simplified JavaScript example of a modern bundler, live on stage. (Scroll down to the bottom of this blog post to watch it)
Bundlers like webpack, Rollup, and Parcel are so commonly used these days that most of us take them for granted. However, this demo will probably provide enough Aha moments to realize that bundlers do much more than just file concatenation and that they’re not as scary as you might think.
The inner workings of a bundler
Generally speaking, bundler implementation consists of three main parts. At first, the entry file is parsed and its dependencies are extracted. Next, a dependency graph is built recursively as depicted in the example below. Finally, everything is packaged into a single file that can run properly in the browser. Accordingly, Ronen implements those three functions during the lecture and uses the resulting bundler to process a sample input, which consists of a linear chain of dependencies.
A sample dependency graph
Looking a little closer, at the first stage the input is parsed and an Abstract Syntax Tree (AST) is generated. The AST is then traversed in order to extract the dependencies, such as those denoted by import statements. This part is implemented with the help of quite effective tools, some of which are mentioned below.
Ronen then uses a queue-based implementation to create the dependency graph. An efficient queue is used instead of recursion, which might be one of the immediate techniques that come to mind when approaching such a task. The queue maps dependencies by keeping pairs consisting of dependent files as keys and files that they depend on as values. In this process every such value child is also pushed to the queue, leading to the processing of all child assets as well, and eventually mapping the dependencies of the entire input.
Based on that dependency map the bundler then combines all files into a single consistent file. It’s important to note though that it is not enough just to concatenate the files. For example, before the bundling, each of the files had its own separate scope. As this might lead to conflicts in the bundled output, the bundle function resolves it by emulating a file-level scope using an empty wrapper function. This function simply wraps the code of each file, so when all files are combined it turns what used to be a file scope into a function scope with a similar effect.
Useful tools for parsing, AST traversing, transpiling and more
During the live coding, Ronen uses quite a few tools that are worth knowing as they can be useful in everyday work. Some of those tools are described here and if any of them has piqued your interest you can see in the video how they were put to use when implementing the bundler.
Babylon JS parser
Babylon parses JavaScript and creates an AST out of it. In this demo, it is used for parsing the input JS files and creating an AST that can then be traversed for extracting the dependencies.
AST explorer
This useful utility displays ASTs such as the ones generated by the Babylon JS parser. Its convenient GUI allows exploring the AST and pinpointing the exact fields that contain the dependency information needed by the bundler.
The import declaration as presented by AST explorer at the bottom right is what we’re after
Babel-traverse
Babel-traverse is used for traversing the AST generated by the Babylon JS parser. The AST explorer mentioned above can be used to identify which fields need to be accessed to extract the dependency information.
Path
This native Node.js module handles file paths. In this case, it is used to retrieve the absolute path of each input file based on its relative path.
Babel
The Babel transpiler transforms the code, which in this case is formatted as an AST, to make it compatible with any browser.
Putting it all together
With the help of those effective tools and some swift JavaScript coding, it is time to put the bundler to the test. After bundling the input example and running the bundled output, it should print “Hello Wix Engineering” if all works well. While this is probably one of the more complicated “Hello World” examples, it is a very straightforward implementation of a tool that many web developers use on a daily basis.
Watch Ronen, live on YGLF's stage, codes from scratch a simplified JavaScript example of a modern bundler. You can also check out Minipack on GitHub to see the annotated code.
Photo by Tim Mossholder on Unsplash
For more engineering updates and insights:
Visit us on GitHub
Subscribe to our YouTube channel