Posted by Martijn Versluis on 24-4-2019
TL;DR The final (refactored) solution is at the bottom.
The modern Ruby/Rails web developer often has to deal with a complex frontend, which means that a passing Selenium test does not guarantee a 100% working web application. There can be JavaScript warnings or errors that popped up in a headless browser, but those are not visible in the test output.
Of course, it is possible to have a frontend (unit) test to catch those errors. But to make sure a feature really works end-to-end you need a feature spec (integration test) that covers the whole stack. There must be a way to handle frontend errors with Selenium…?
🎉 There is!
You have to dig a bit into the page object:
1
page.driver.browser.manage.logs.get(:browser)
It is a collection of Selenium::WebDriver::LogEntry objects. Let’s see what’s in there. Below is a pretty print of page.driver.browser.manage.logs.get(:browser). The message property is what we want to show up in our test output.
1
2
3
4
5
6
7
[#<Selenium::WebDriver::LogEntry:0x00007fab652d29f0
@level="INFO",
@message=
"http://127.0.0.1:3002/static/js/1.chunk.js 80889:20 \"%cDownload the React DevTools for a better development experience: https://fb.me/react-devtools\"",
@timestamp=1554883156962>
....
]
So, by default the logs aren’t really useful. Let’s see what happens when there is a frontend error. We will randomly invoke function foobar, which we didn’t define. Afterwards, we inspect the logs again. It now contains an entry for our JavaScript error:
1
2
3
4
5
#<Selenium::WebDriver::LogEntry:0x00007fbfb4335b28
@level="SEVERE",
@message=
"http://127.0.0.1:3002/static/js/1.chunk.js 82408:19 \"./src/components/ivr/IVRSettingsForm.jsx\\n Line 71: 'foobar' is not defined no-undef\\n\\nSearch for the keywords to learn more about each error.\"",
@timestamp=1554883526785>
This entry is actual valuable information. If there exists such an entry, the Selenium test should fail. Note the @level="SEVERE" part. We can actually filter the logs and select the entries where level == 'SEVERE'. Let’s create our own SeleniumBrowserErrorReporter!
1
2
3
4
5
6
7
8
9
10
11
12
13
class SeleniumBrowserErrorReporter
def initialize(page)
self.page = page
end
def report!
# The magic needs to happen here...
end
private
attr_accessor :page
end
First, we select only severe errors, and return early when there are none:
1
2
3
logs = page.driver.browser.manage.logs.get(:browser)
severe_errors = logs.select { |log| log.level == 'SEVERE' }
return if severe_errors.none?
Then we format the entries by grabbing the message and join them with a blank line in between:
1
2
3
report = severe_errors.map { |error| error.message.gsub('\\n', "\n") }.join("\n\n")
raise "There #{severe_errors.count == 1 ? 'was' : 'were'} #{severe_errors.count} "\
"JavaScript error#{severe_errors.count == 1 ? '' : 's'}:\n\n#{report}"
We attach the reporter in our RSpec configuration:
1
2
3
4
5
6
7
8
9
RSpec.configure do |config|
...
config.after :each, type: :feature do
SeleniumBrowserErrorReporter.new(page).report!
end
...
end
The test failure looks like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
1) Failure/Error:
raise "There #{severe_errors.count == 1 ? 'was' : 'were'} #{severe_errors.count} "\
"JavaScript error#{severe_errors.count == 1 ? '' : 's'}:\n\n#{report}"
RuntimeError:
There were 3 JavaScript errors:
http://127.0.0.1:3002/static/js/0.chunk.js 82408:19 "./src/components/SettingsForm.jsx\n Line 53: 'foobar' is not defined no-undef\n\nSearch for the keywords to learn more about each error."
http://127.0.0.1:3002/static/js/0.chunk.js 82408:19 "./src/components/SettingsForm.jsx\n Line 53: 'foobar' is not defined no-undef\n\nSearch for the keywords to learn more about each error."
http://127.0.0.1:3002/static/js/main.chunk.js 4611:2 Uncaught ReferenceError: foobar is not defined
# ./spec/support/selenium_browser_error_reporter.rb:12:in `report!'
# ./spec/rails_helper.rb:120:in `block (2 levels) in <top (required)>'
There is one annoying thing: newline characters (\n) are escaped. Let’s replace them with actual newlines:
Now we have a nice report, just like our browser console would display it:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
1) Failure/Error:
raise "There #{severe_errors.count == 1 ? 'was' : 'were'} #{severe_errors.count} "\
"JavaScript error#{severe_errors.count == 1 ? '' : 's'}:\n\n#{report}"
RuntimeError:
There were 3 JavaScript errors:
http://127.0.0.1:3002/static/js/0.chunk.js 82408:19 "./src/components/SettingsForm.jsx
Line 53: 'foobar' is not defined no-undef
Search for the keywords to learn more about each error."
http://127.0.0.1:3002/static/js/0.chunk.js 82408:19 "./src/components/SettingsForm.jsx
Line 53: 'foobar' is not defined no-undef
Search for the keywords to learn more about each error."
http://127.0.0.1:3002/static/js/main.chunk.js 4611:2 Uncaught ReferenceError: foobar is not defined
# ./spec/support/selenium_browser_error_reporter.rb:12:in `report!'
# ./spec/rails_helper.rb:120:in `block (2 levels) in <top (required)>'
Final solution
Finally, I’d like to refactor a bit, because our report! method grew large and got a bit unreadable.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
class SeleniumBrowserErrorReporter
def initialize(page)
self.page = page
end
def report!
severe_errors = logs.select { |log| log.level == 'SEVERE' }
return if severe_errors.none?
report = error_report_for(severe_errors)
raise "There #{severe_errors.count == 1 ? 'was' : 'were'} #{severe_errors.count} "\
"JavaScript error#{severe_errors.count == 1 ? '' : 's'}:\n\n#{report}"
end
private
attr_accessor :page
def error_report_for(logs)
logs
.map(&:message)
.map { |message| message.gsub('\\n', "\n") }
.join("\n\n")
end
def logs
page.driver.browser.manage.logs.get(:browser)
end
end
Wrap up
Now we have a nice error reporter that makes us aware of frontend issues. Please let me know if you like this solution, or if you have questions or feedback.
At Kabisa we know a lot about writing good tests. Leave us a message if you would like to get in touch.
