I’ve been working on a Vue.js application, and I was thinking about how I can test it easily. The app has a couple of interesting libraries with custom directives and widgets, some of them with scoped-slot. Additionally, I have 10-12 Vue components with hierarchical structures in different .vue files, which means I need to test features that (as usual) imply > 1 component. And in this case, I have 4 stores.
My first approach was using Jest, it is pretty easy to install and configure. I tried 2 small and simple tests in the beginning and the result was good. But when I started to think to test larger units with 2 or more components, adding scoped slots + store + custom libraries with custom components +… each test was incredibly big and unmaintainable. Everything is for a really small application with 10-12 components. It doesn’t make sense to me. And an important issue: scoped-slot only can be emulated, but you can’t add those to the tests.
The second and great option was to create E2E tests with Nightwatch.js. I installed it and configure it locally and with BrowserStack pretty fast which allowed me to test on my Firefox and Chrome local installation but most important run my tests on different OS (like Windows and macOS) and Browsers (like Chrome, Firefox, Safari), but most importantly I could create tests really easy. Let me show quickly how you can configure it locally on my laptop (Ubuntu 19.04) (the installation process is available in docs):
In my root path I have this:
In the test folder, I have my tests, in conf my Nightwatch config files (for BrowserStack and local), in output I have the JUnit XML files (those are the results of Selenium tests). If some of the tests don’t result as expected, a screenshot is saved in the screenshots folder. And finally, in the log folder, there are log files with issues about tests and selenium server results.
Let’s see how looks my local Nightwatch settings:
module.exports = {
src_folders: ["e2e_tests/tests"],
selenium: {
launch_url: process.env.HOST,
start_process: true,
server_path: require('selenium-server').path,
cli_args: {
'webdriver.gecko.driver': require('geckodriver').path,
'webdriver.chrome.driver': require('chromedriver').path
},
log_path: './e2e_tests/log'
},
test_settings: {
default: {
launch_url: process.env.HOST_TEST,
screenshots : {
enabled : true,
on_failure : true,
on_error : false,
path : "./e2e_tests/screenshots"
},
output_folder: "./e2e_tests/output"
},
chrome: {
desiredCapabilities : {
browserName : 'chrome',
alwaysMatch: {
acceptInsecureCerts: true
},
chromeOptions: {
w3c: false
}
},
webdriver: {
port: 4444,
start_process: true,
server_path: require('chromedriver').path,
log_path: './e2e_tests/log'
}
},
firefox: {
desiredCapabilities : {
browserName : 'firefox',
alwaysMatch: {
acceptInsecureCerts: true
}
},
webdriver: {
start_process: true,
port: 4444,
server_path: require('geckodriver').path,
log_path: './e2e_tests/log'
}
}
}
}
These settings will open my local Chrome and Firefox browsers and run tests in port 4444.
Just to test the app quickly I can create a simple test in the e2e_tests/test folder like this:
// e2e_tests/tests/FirstTest.js
module.exports = {
'Checking title must contains Hello World! string in HTML title' : function (browser) {
browser
.url(process.env.HOST)
.waitForElementVisible('body')
.assert.titleContains('Hello World!')
.end();
}
};
If I add this script in my package.json "e2e": "./node_modules/.bin/nightwatch e2e_tests/tests/ -c ./e2e_tests/conf/nightwatch.conf.js"
I can run my first test on Chrome and Firefox running: $ npm run e2e -- --env firefox,chrome .
And tatan!! after opening both browsers simultaneously and then close automatically, I got something like this:
firefox [App] Test Suite
firefox ================
firefox - Connecting to localhost on port 4444…
firefox
firefox ℹ Connected to localhost on port 4444 (7131ms).
firefox Using: firefox (77.0.1) on linux 5.3.0-59-generic platform.
firefox Results for: Checking title must contains Hello World! string
firefox ✔ Element was visible after 142 milliseconds.
firefox ✔ Testing if the page title contains 'Hello World!' (10ms)
firefox ✔ firefox [App] Checking title must contains Hello World! string (5.429s)
firefox [Filter] Test Suite
firefox ===================chrome [App] Test Suite
chrome ================
chrome - Connecting to localhost on port 4444…
chrome
chrome ℹ Connected to localhost on port 4444 (2494ms).
chrome Using: chrome (83.0.4103.116) on Linux platform.
chrome Results for: Checking title must contains Hello World! string
chrome ✔ Element was visible after 58 milliseconds.
chrome ✔ Testing if the page title contains 'Hello World!' (12ms)
chrome ✔ chrome [App] Checking title must contains Hello World! string (1.944s)
chrome [Filter] Test Suite
chrome ===================OK. 2 total assertions passed (6.023s)
Now let’s see how looks my Nightwatch config file for BrowserStack, so we can run our tests in multiple browsers and OS remotely:
// e2e_tests/conf/browserstack.nightwatch.conf.js
nightwatch_config = {
src_folders: ["e2e_tests/tests"],
selenium: {
start_process: false,
host: "hub-cloud.browserstack.com",
port: 80,
log_path: './e2e_tests/log',
output_folder: "./e2e_tests/output"
},
test_settings: {
default: {
launch_url: process.env.HOST,
screenshots : {
enabled : true,
on_failure : true,
on_error : false,
path : "./e2e_tests/screenshots"
},
log_path: './e2e_tests/log',
output_folder: "./e2e_tests/output"
},
chrome_win10: {
desiredCapabilities : {
'browserstack.user': process.env.BROWSERSTACK_USER,
'browserstack.key': process.env.BROWSERSTACK_ACCESS_KEY,
'os': 'Windows',
'os_version': '10',
'browser': 'Chrome',
'browser_version': (process.env.CHROME_W10_VERSION) ? process.env.CHROME_W10_VERSION : '83.0',
'resolution': '1280x800'
},
webdriver: {
log_path: './e2e_tests/log'
},
log_path: './e2e_tests/log',
output_folder: "./e2e_tests/output"
},
chrome_catalina: {
desiredCapabilities: {
'browserstack.user': process.env.BROWSERSTACK_USER,
'browserstack.key': process.env.BROWSERSTACK_ACCESS_KEY,
'os': 'OS X',
'os_version': 'Catalina',
'browser': 'Chrome',
'browser_version': (process.env.CHROME_CATALINA_VERSION) ? process.env.CHROME_CATALINA_VERSION : '83.0',
'resolution': '1280x800'
},
webdriver: {
log_path: './e2e_tests/log'
},
log_path: './e2e_tests/log',
output_folder: "./e2e_tests/output"
},
firefox_win10: {
desiredCapabilities : {
'browserstack.user': process.env.BROWSERSTACK_USER,
'browserstack.key': process.env.BROWSERSTACK_ACCESS_KEY,
'os': 'Windows',
'os_version': '10',
'browser': 'Firefox',
'browser_version': (process.env.FIREFOX_W10_VERSION) ? process.env.FIREFOX_W10_VERSION : '77.0',
'resolution': '1280x800'
},
webdriver: {
log_path: './e2e_tests/log'
},
log_path: './e2e_tests/log',
output_folder: "./e2e_tests/output"
},
firefox_catalina: {
desiredCapabilities : {
'browserstack.user': process.env.BROWSERSTACK_USER,
'browserstack.key': process.env.BROWSERSTACK_ACCESS_KEY,
'os': 'OS X',
'os_version': 'Catalina',
'browser': 'Firefox',
'browser_version': (process.env.FIREFOX_CATALINA_VERSION) ? process.env.FIREFOX_CATALINA_VERSION : '77.0',
'resolution': '1280x800'
},
webdriver: {
log_path: './e2e_tests/log'
},
log_path: './e2e_tests/log',
output_folder: "./e2e_tests/output"
},
safari_catalina: {
desiredCapabilities: {
'browserstack.user': process.env.BROWSERSTACK_USER,
'browserstack.key': process.env.BROWSERSTACK_ACCESS_KEY,
'os': 'OS X',
'os_version': 'Catalina',
'browser': 'Safari',
'browser_version': (process.env.SAFARI_CATALINA_VERSION) ? process.env.SAFARI_CATALINA_VERSION : '13.0',
'resolution': '1280x800'
},
webdriver: {
log_path: './e2e_tests/log'
},
log_path: './e2e_tests/log',
output_folder: "./e2e_tests/output"
}
}
};
// Code to copy seleniumhost/port into test settings
for(var i in nightwatch_config.test_settings){
var config = nightwatch_config.test_settings[i];
config['selenium_host'] = nightwatch_config.selenium.host;
config['selenium_port'] = nightwatch_config.selenium.port;
}
module.exports = nightwatch_config;
In this file, config will run my tests in BrowserStack on Chrome and Firefox in Windows10 and Catalina OS (Mac) and Safari on Catalina OS. Results of those tests will be in e2e_tests/output, but in BrowserStack App Live Dashboard as well with a nice Video and Screenshots. Awesome, eh?
In my package.json I added another script for BrowserStack tests: "e2e:browserstack": "node ./e2e_tests/conf/local.runner.js -c ./e2e_tests/conf/browserstack.nightwatch.conf.js"
So now I can run my tests just running: $ npm run e2e:browserstack -- --env firefox_win10,safari_catalina
I will get in my console something similar to:
Connecting local!
Connected. Now testing…
firefox_win10 Using insecure HTTP connection on port 80. Consider using SSL by setting port to 443 in your Nightwatch configuration.
firefox_win10 [App] Test Suite
firefox_win10 ================
firefox_win10 - Connecting to hub-cloud.browserstack.com on port 80…
firefox_win10
firefox_win10 ℹ Connected to hub-cloud.browserstack.com on port 80 (10779ms).
firefox_win10 Using: firefox (77.0) on windows 10.0 platform.
firefox_win10 Results for: Checking title must contains Hello World! string
firefox_win10 ✔ Element was visible after 1999 milliseconds.
firefox_win10 ✔ Testing if the page title contains 'Hello World!' (418ms)
firefox_win10 ✔ firefox_win10 [App] Checking title must contains Hello World string (7.624s)
firefox_win10 Using insecure HTTP connection on port 80. Consider using SSL by setting port to 443 in your Nightwatch configuration.
In my BrowserStack App Live build for Firefox on Windows 10 I will see something like this:
So finally, now I just need to add more tests and add them to my CI/CD pipeline.
Aaaaaaaaaand, today we have a super classic Argentine musician who wrote incredible rock music since the ’80s. One of him masterpiece is “El amor después del amor” (it’s Love after love).