We're developing a library[1] for this use-case. The goal is to have the same simple API for multiple browsers. Right now it supports both Headless Chrome and PhantomJS.
We think we'll begin work on Firefox headless soon. PRs welcome :)
I'm currently evaluating testcafe which is pretty nice, and headless chrome was easy to setup. I'm interested to see what people recommend for browser testing, I'm also using preact.
Writing end-to-end tests is always a sucky and flaky experience, and it always ends up slowing down CI. With that said, I've used testcafe without any big complaints.
My general rule of thumb is to try having as few end-to-end tests as possible. It's fine for an e2e test to cover multiple aspects of your application. Trying to write e2e tests as if they were unit tests just leads to sadness and infinite CI builds.
I don't think there's any value to running all your tests on the browser. It's much faster and simpler to run the bulk of your tests on node, with a fake DOM environment.
If you're interacting directly with the DOM, you might want to consider breaking that off as a separate lib. From the lib's repo you can have all of its tests run on a real browser.
Totally agree. I am a fan of Test Pyramid model, where at the top you have a very small number of long running, complex tests (usually e2e), and at the very bottom you have a lot of small, isolated and very fast unit tests.
Testing everything in a browser/e2e provides a rather poor cost/benefit ratio, especially as the project grows and more features and tests are added, so it should be reserved for cases where it is really important.
I like having e2e tests that cover the very core functionality and involve multiple parts of the stack. For example, if you have your rather typical app with a login modal, it's probably good to have a test that clicks on the button, makes sure that the modal is visible, and the user is redirected to the expected page after logging in. But testing if error message is displayed if email address is invalid? That's too much.
I currently have setup two CI jobs running Webdriver.IO with different configurations (different in what browser is used). Locally I just run it on all installed browsers using selenium-assistant - not headless, because I want to see what's going on when running locally.
(And yeah, cross-platform Edge would be nice for being able to test in CI without needing a third party.)
Can't wait for a cross-platform headless Edge :-)