As the saying goes — Knowledge is power, and power is… improved site performance?

The frontend development process often involves tradeoffs — how much code we want to add to achieve desired functionality versus the performance impact. The more code we include, the more our users’ browsers will need to retrieve to run our applications.

Nowhere is this felt more than with the addition of external packages. Consider the task of date formatting: a package called moment covers a variety of date manipulation use cases, and is one npm install away. However, adding it will increase your bundle size by a significant amount, potentially impacting site performance. Many use cases will be sufficiently served by a significantly smaller package such as date-fns. Bundle size is just one metric which can be used to gauge the impact of code changes on site performance, and many more can be exposed.

This kind of knowledge is most useful to developers at the point where code changes are made — what better place than pull requests? Throughout my internship with the frontend infrastructure team at NerdWallet, I worked on helping frontend developers become more aware of the performance impact of their code. The chosen method of accomplishing this task was exposing pertinent information in automated PR comments. The first such metric happened to be built bundle sizes.

Initial Implementation

The requirements for the bundle size tool were as follows:

• Run after a webpack build on the CI server

• Obtain the sizes of emitted asset files

• Output the asset sizes in a human-readable format suitable for a PR comment

The tool had to be flexible and easily runnable by NerdWallet’s internal build system in CI, so a command line form was chosen. The tool was built with ES7 and Node to easily integrate with existing frontend infrastructure.

Frontend apps at NerdWallet take advantage of a central build package which incorporates common webpack configurations. As part of the build process, this package runs

webpack --profile --json > webpack-stats.json

which generates a webpack stats file containing a plethora of post-build information. The piece we care about is the assetsByChunkName property, which contains the paths to all asset files generated by webpack.

In its initial incarnation, the asset sizes tool read the asset paths and determined the parsed and gzipped sizes of each corresponding file (for an application employing code splitting such as the one below, there are quite a few).

The tool generates the pretty console table above, as well as a Markdown table if used with the --markdown flag.

The newly obtained asset size data is saved along with other post-build information for the project, in JSON format.

Comparison of Asset Sizes

What could be more useful than clearly displayed built asset sizes? — A comparison of asset sizes to those on your upstream branch!

The next step was adding the ability to compare the sizes on the current working branch to those on a different branch. This was accomplished by adding a --compare mode, which would take an existing asset-sizes.jsonfile and compare its information to the current built assets. Below is the console output when running the tool in this mode:

The output includes the size difference for each asset. Cheery green checkmarks are displayed for positive reinforcement, when a size drops. Severe red crosses are displayed for punishment, if there is a size increase beyond a certain threshold. Of course, Markdown output is just a --markdownflag away.

The Final Result

Undoubtedly, the comparison functionality is most useful when running on the CI server, outputting a comparison to asset sizes on master in a PR comment.

An outline of this process:

1. The NerdWallet build system saves asset sizes for every branch (including master) in S3 after its CI build

2. During a build triggered by a PR, the system runs the asset sizes CLI tool in comparison mode, passing in the asset size data file corresponding to the upstream branch

3. The Markdown-format comparison output is captured and posted in the form of a PR comment

Here is the output of the tool on a couple of PRs, both implementing an analogous single-line change pertaining to date formatting — one using moment, the other using date-fns :

The comments include the table seen in the console comparison output (unchanged assets are not displayed), as well as an HTML <details> tag which can be expanded to show the parsed and gzipped sizes for all assets.

From the output of the tool, the impact of both code changes on bundle size becomes immediately clear at PR time. This is not to say that moment is a bad library. The point here is making developers aware of the performance impact of their changes, so they can incorporate this knowledge into choosing the right tool for any particular use case.

Notice that the PR comments above include an extra detail — webpack-bundle-analyzer output! Through feedback gathered from developers, we realized that the visualization provided by this tool would be very valuable. This is especially true for apps which do not employ code-splitting and only output a single js-css asset pair. Here is how this works:

• The central build package uses the webpack-bundle-analyzer plugin to save the interactive HTML file at build time

• The NerdWallet CI build system maintains webpack-bundle-analyzer files in S3 for each project and exposes links to them

Metric Tool Generalization and Future Functionality

The addition of asset analysis to the build process encouraged the NerdWallet DevOps team to establish a general system for gathering and maintaining CI build-time metric data. This allows for useful functionality such as the cross-branch comparison described above, and is already being used for test coverage analysis. The system can potentially be leveraged to implement metric visualization across time in the future.

With the asset sizes tool complete, attention turned to generalizing metric retrieval and display functionality so that other performance metrics could be easily exposed. I realized that most frontend metric tools would have the following common functionality:

• Gathering, outputting, and saving metric data for the current branch

• Outputting a comparison of gathered data to upstream data

The specific functionality required for exposing a particular metric would be:

1. A function to retrieve JSON metric data

2. A function to output a representation of a single metric dataset to stdout

3. A function to output a visual comparison between 2 datasets to stdout

4. Any extra CLI arguments required to accomplish the prior 3 tasks

Thus, I developed a programmatic interface that allows for relatively simple creation of metric CLI tools, requiring only the specification of the 4 items described above.