View profile

A Practical Guide to istanbul/nyc Code Coverage Internals

A Practical Guide to istanbul/nyc Code Coverage Internals
By Mastering JS Weekly • Issue #83 • View online
Code coverage tools track what lines of code are executed during tests. They help you identify which functions and code paths aren’t tested. There are numerous code coverage tools for JavaScript: istanbul and its more modern CLI wrapper nyc were the de facto standard in Node for many years, but tools like Jest also ship their own code coverage tools. Most notably, the V8 JavaScript runtime now has native code coverage.

Example code coverage output
Example code coverage output
Over the last couple of weeks we’ve been working on supporting code coverage with Temporal’s TypeScript SDK. Supporting code coverage has been a challenging project, but it’s also been rewarding.
Code coverage tools like nyc mostly “just work.” In most applications, you just add `nyc` before your test command and you automatically get code coverage. With Temporal’s TypeScript SDK, code coverage is a bit trickier because Temporal Workflows run in a `vm.runInContext()`, which means you need to unify code coverage results from the individual Workflow runs.
Istanbul/nyc vs V8 Coverage
The first decision we had to make was whether to use nyc or V8’s native code coverage. We chose nyc for 2 reasons. First, we’re more familiar with nyc and istanbul.
Second, and more importantly, istanbul is more “hackable”, because code can access the current state of code coverage in the `global.__coverage__` variable. Your code can actually inspect the current coverage state. And, more importantly, you can transfer code coverage state between processes or between sandboxes. V8 native coverage doesn’t support that.
Print current code coverage state
Print current code coverage state
To get code coverage for Workflows, what I needed to do was:
  1. Get global.__coverage__ from every Workflow
  2. Merge the Workflow’s global.__coverage__ into the main test file’s global.__coverage__.
Googling for Solutions
Getting `global.__coverage__` is fairly easy, the hard part is merging the code coverage in memory. For niche problems like this one, there often isn’t an exact solution on Google. But there’s plenty of related info. This blog post from Cypress.io was very helpful, and reading the source code for istanbul-merge provided some insights.
In particular, reading the code for istanbul-merge pointed me to the istanbul-lib-coverage library, which provides some utilities for working with istanbul coverage maps. Below is an example coverage map POJO from istanbul:
Code coverage POJO
Code coverage POJO
In istanbul, a CoverageMap is a collection of FileCoverage objects. And CoverageMaps have a nice `merge()` function that you can call to merge a `global.__coverage__` object.
The `merge()` function merges `global.__coverage__` into a CoverageMap
The `merge()` function merges `global.__coverage__` into a CoverageMap
Unfortunately, we’re not quite done. Because `global.__coverage__` is not an instance of a CoverageMap. So while you can merge `global.__coverage__` into a CoverageMap, you can’t merge a CoverageMap into `global.__coverage__`.
Fortunately, transforming a CoverageMap into the same format as a `global.__coverage__` is easy. A CoverageMap has a `data` property that contains a bunch of FileCoverages, and a FileCoverage object has a `data` property that contains the file’s coverage data. To transform a CoverageMap into `global.__coverage__` format, you just need to get rid of the `data` properties as shown below.
Get rid of `data` fields
Get rid of `data` fields
Bonus: Clearing Coverage State
Another detail I needed to implement was zeroing out the code coverage state after `merge()` calls. That’s because Temporal can suspend a Workflow, which would wipe out the coverage data, so we can’t wait until the end of the Workflow to pull coverage data. But, if we merge the same CoverageMap multiple times, then the line execution counts will be wrong.
Each istanbul FileCoverage has a `s` property that contains the number of times the given line has been executed. Resetting all the values to 0 resets code coverage.
Need to set all `s` values to 0 to reset code coverage
Need to set all `s` values to 0 to reset code coverage
Below is our `clearCoverage()` function.
Loop over every file's `s` value, set all the values to 0
Loop over every file's `s` value, set all the values to 0
Thanks for reading! Check out our sponsor, Vue School!
Thanks for reading! Check out our sponsor, Vue School!
Most Recent Tutorials
Using limit() with Mongoose Queries - Mastering JS
Capitalize the First Letter of a String Using Lodash - Mastering JS
What We're Reading
Integer math in JavaScript | James Darpinian
Code Coverage | Cypress Documentation
Did you enjoy this issue?
Mastering JS Weekly

Pragmatic web development. No bloatware allowed!

In order to unsubscribe, click here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Powered by Revue