Headless Browser Testing With Xvfb

Jan 09 ’15·javascripttesting

These days, when the phrase "headless browser" is mentioned, you immediately think of PhantomJS, but - there are alternatives. In this article, I am going to introduce a nice alternative you can use on Linux which allows you to run real browsers - Firefox and Chrome, headless - using Xvfb.

What's Wrong With PhantomJS?

PhantomJS is great tool and is widely used by companies and developers around the world. It has particularly been widely adopted for running JavaScript test suites. However, using PhantomJS isn't without drawbacks. Although it is built on top on WebKit - the same rendering engine both Safari and Chrome use, it still behaves in subtly different ways from those browsers, which can cause false positives (tests failing when there is not a real defect) and in some cases performance degradation of your test suite. Since PhantomJS isn't a real browser in the sense that no end-user actually use it, fixing these issues specifically for PhantomJS becomes an upkeep cost of the test suite, rather than something that directly improves the product. Getting developers to write tests consistently is hard enough, but if maintaining the tests becomes annoying - which false positives definitely are - it can become demoralizing.

Run Real Browsers Headless

Because of the above reasons, at one point in my career I decided to use real browsers only to run JavaScript test suites. This has meant opening up a separate browser window to run the tests during development which I would simply hide in the background. I am okay with this. However, in many cases the continuous integration servers we use are Linux servers, and do not have displays. This is where Xvfb comes in.

Xvfb is a virtual display framebuffer for X - the display system used by Linux. It provides a fake display buffer for graphical programs to write to, thus allowing any program to run headlessly.

Installing Xvbf

If you are running Debian or Ubuntu and are using apt-get, installing is simply as:

apt-get install xvfb

If you are on CentOS and using yum, it's

yum install xorg-X11-server-Xvfb

Installing the Browsers

On Ubuntu, I found installing Chrome and Firefox painless.

• For Chrome simply go to the Chrome download page, download the .deb file, then do dpkg -i <the path to the .deb file> to install.
• For Firefox, it's just apt-get install firefox.

On CentOS, I have had more trouble.

• For starters, Chrome is not supported on CentOS and I haven't even found a simply way to build Chrominum and I gave up.
• Installing firefox also had a little hiccup: I ran yum install firefox, but then running firefox resulted in an error, which I solved by also installing gdk-pixbuf2 via yum install gdk-pixbuf2.

If you are on other Linux distros I haven't mentoned, checkout the Chrome download page and the Firefox on Linux page.

Xvfb Usage

The usage of Xvfb is:

xvfb-run <some command>

That's all you need to know! For example, you can run Firefox within Xvfb:

xvfb-run firefox http://google.com

This should just start a firefox process in the background. You may see a cryptic warning in the terminal, but otherwise it's not very exciting because you don't actually see anything.

If you use Testem to run tests, you can do:

xvfb-run testem -l firefox

to run your tests in Firefox or

xvfb-run testem -l chrome

to run them in Chrome - headless!

Of course, this is not limited to Testem, you can use xvfb-run with Karma, Grunt, Gulp or any other test runner that spawns a real browser and it will just work.

Implications

If you use Linux for development, this is an attractive solution. It allows you to run your tests without having an unwanted window and it's easy to setup and use. However, even if you don't use Linux for development, you may still use Linux for continuous integration, in which case, maybe you are currently using PhantomJS to run tests. If that's working well for you, great. But if PhantomJS is starting to create unwanted overhead for you, consider switching to this setup.