Automated Visual Regression Testing

A step-by-step guide on keeping your UI stunningly predictable with Jest & Storybook.

Rainbow of Lollipops — Amy Shamblen, Unsplash

I just wanted to change this one thing, not to break a hundred others!

Sounds familiar? To me it definitely does.

That’s why I would like to share an idea I came up with to test for unwanted visual changes using Storybook & Jest. The sample application uses React, but the solution itself can be used in an ecosystem of your choice. Ready to set off on a journey towards a soothingly dependable UI? I thought so, let’s roll.

1. Application bootstrap

Let’s start with a fresh instance of a React application using Create React App:

npx create-react-app visual-regression-testing

It would be nice to have a sample component to focus on. Add a src/components/Button/index.jsx:

and some styling at src/components/Button/styles.css:

2. Unit testing

CRA comes with a sample unit test and Jest as a test runner built in. Going to the application directory and running:

npm test

will get it rolling. Together with React Testing Library this is a great setup to test the functionalities of your application. Let’s add a unit test for the Button in src/components/Button/index.test.jsx:

At this point we could check if the button has the correct classes depending on the passed variant, and be done with it. Sure. But keep reading.

3. Storybook

According to their page,

Storybook is an open source tool for developing UI components in isolation.

That’s exactly what we want. Let’s add it by running:

npx sb init

in the project folder.

Storybook comes with samples within the `src/stories` folder and some baseline configuration. You can go through those if you wish, but for the sake of this example we will keep it simple. Let’s modify the .storybook/main.js file to contain only:

Now let’s add some stories to illustrate the three variants (default, primary, and secondary) the Button comes in. Create a src/components/Button/index.stories.jsx with the following content:

Enough of that blind following! Let’s take a look by running:

npm run storybook

New Storybook tab should open showing three… very unimpressive buttons. Uh oh, despite all the correct classes, the button looks nothing like we’d expect.

Ah, the styles were not imported. In src/components/Button/index.jsx add

and watch Storybook update on the fly. Now you are able to check for visual regression manually.

4. Snapshots

Let’s first add some tooling:

npm install jest-image-snapshot puppeteer jest-puppeteer start-server-and-test lodash rimraf --save-dev

Then let’s add a separate Jest setup for visual regression testing. Make a visualRegression/jest.config.js with:

and a visualRegression/setupTests.js with:

Then in the scripts section of your package.json add:

Finally, add a visualRegression\__cases__\Button.test.js file, containing of:

and run:

npm run visualRegression:test

Whoa, what just happened? Storybook was run, then during the test a page of the Button’s Basic story was visited in an iframe mode (to avoid Storybook’s UI in all the snapshots). A print screen of it was taken, and saved (as there was no existing one to compare to). So if you go to visualRegression\__cases__\__image_snapshots__, you’ll find a snapshot with your basic button there.

Should the story change visually — let’s say, by changing ‘Hello World’ to ‘Basic Button’ in src/components/Button.index.stories.jsx— the next time the test is run it will create a file in visualRegression\__cases__\__image_snapshots__\__diff_output__ to outline where exactly the differences lay.

Adding scripts:

and running

npm run visualRegression:update

will update the snapshots to match the current state.

Yet this is just one story of one component. Adding a test case for each and every one would be, to put it mildly, tiresome. Let’s automate.

5. Automation

After testing some other approaches, I came up with one allowing a decent amount of control in terms of the naming and which stories to include.

Firstly, add src/components/Button/index.visualTest.js with:

Add such a file for each component you wish to keep track of. The ‘name’ should match the first part of the ‘id’ in the story’s URL (relates to the title). ‘Stories’ are the names of the exports within the ‘*.stories.tsx’ file). There could also be an ‘as’ field, which would be a pretty name to print out in the tests.

Secondly, create visualRegression\generateTestCases.js:

and visualRegression\clear.js:

Then in ‘package.json’, replace all the pre-existing visual-regression-related scripts with:

Finally, run:

npm run visualRegression:update

to remove the old and generate the new. Open visualRegression\__cases__\Button.test.js and notice there’s a test case for each story.

6. Check

So now if you were to break your application by a careless override, you will be warned.

Let’s simulate that by changing the value of the `color-primary` variable at the end of src/components/Button/styles.scss to a reversed hex color:

Then run:

npm run visualRegression:test

The visual outcome of the stories has changed, so the tests are failing. It is now possible to go through the diffs and identify what’s actually changed.

Should you come to a conclusion the changes are acceptable — in this case, the new color is nicer than the previous (which it is) — run:

npm run visualRegression:update

and commit both the change, and the snapshots.

No more changes popping up unexpectedly. From now on, you are covered.

If at any point you need a recap, all the steps are available as commits in this repo.

Happy hacking!

Tap the 👏 button if you’re already feeling safer (and/or you found this article useful).