Robotframework browser automation tools — a review for 2021

In a previous article I discussed the benefits of robotframework and given an example of how easily you can extend robotframework using the python lib core.

In this followup article I’ll go over the tools robotframeworks’ community has to offer for browser automation.

The testing pyramid

the testing pyramid

According to Martin Fowler, tests should be sorted into 3 different layers by their cost (the cheapest at the bottom), execution speed (the fastest at the bottom), stability (most stable at the bottom), complexity (the least complex at the bottom), and validity (least valid at the bottom)

Unit Tests

At the bottom of the pyramid we find our unit tests, in which we test the different components of our software in isolation.

Unit test are fast, easy to develop and low on cost, however they are not realistic scenarios and they do not reflect a real users use-case in any sense.

Unit tests are our first line of defense against regressive code changes, they should be ran most frequently (probably with every commit or even local developers code change), and they are the main volume of test coverage.

Integration Tests

At the middle of the pyramid we find integration tests — aka API tests.
This time around we test our api as a who service and the interrelation between the different components of our software.

To perform api tests you need an api running with data sets, so they require more setup and they are slower to run when compared with unit tests.
Which is why they are more costly and complex.

API test are quite stable most of the time and they are closer to user use-cases then unit tests, but they are still not quite the user use-case.

They are performed more rarely then unite tests (probably every merge commit or pull request approval) and they are secondary in volume and coverage.

E2E Tests

At the top of our pyramid, we have E2E tests and these are the closest thing to a user that we can get.

Product managers love these tests because they give full coverage to the system (hence they are End-2-End).

despite the clear advantage, UI tests come at a cost, literally.
They require a fully operational system, they are slow to execute, they take more time and efforts to write and maintain and they are unstable — in other words they are flaky.

Since UI tests are costly, unstable and complex, they are performed infrequently (at most at nightly test runs) and they have the lowest volume and test coverage.

Robotframework Selenium Library

Link — https://github.com/robotframework/SeleniumLibrary

This is arguably the most commonly used library and the one you’re going to find in most tutorials.

Selenium is probably the most famous browser automation framework and is being used widely across different enterprises.

Selenium design decouples the API from the browser actions with the webdriver that is specific to the browser, this way selenium can easily support many different languages (C#, Java, Python, Rubi, Javascrip, PHP, R…) and different browsers as well (Firefox, chrome, IE, Edge, safari, Opera).

Selenium also supports distributed test runs with selenium grid.

The robotframework selenium library offers easy use of the selenium webdriver to perform browser automation.

*** Settings ***
Documentation     Simple example using SeleniumLibrary.
Library           SeleniumLibrary

*** Variables ***
${LOGIN URL}      http://localhost:7272
${BROWSER}        Chrome

*** Test Cases ***
Valid Login
    Open Browser To Login Page
    Input Username    demo
    Input Password    mode
    Submit Credentials
    Welcome Page Should Be Open
    [Teardown]    Close Browser

*** Keywords ***
Open Browser To Login Page
    Open Browser    ${LOGIN URL}    ${BROWSER}
    Title Should Be    Login Page

Input Username
    [Arguments]    ${username}
    Input Text    username_field    ${username}

Input Password
    [Arguments]    ${password}
    Input Text    password_field    ${password}

Submit Credentials
    Click Button    login_button

Welcome Page Should Be Open
    Title Should Be    Welcome Page

The library offers a suite of keywords for managing webdriver instances and costumed assertions for common scenarios.

For example, keyword assertions for HTML table objects.

*** Settings ***
Library           SeleniumLibrary

*** Test Cases ***
Table Example
    table should contain  table-locator  My-Name
    table cell should contain  table-locator  row=3  column=6     My-Name

Another example would be common “waiters” for automation timing.

*** Settings ***
Library    SeleniumLibrary

*** Test Cases ***
Test Case
   Open Browser    https://robotframework.org/    firefox
   Wait Until Element Is Visible    introduction
   Close Browser

The library is also easily extendable with plugins.

Robotframework-PuppeteerLibrary

Link — https://pypi.org/project/robotframework-PuppeteerLibrary/

Puppeteer was originally developed by google as a javascript based framework for browser automation.

The Idea was, using the Chrom-Devtools-Protocol (CDP) and this way we can have our browser executing the automation code by itself rather then having a webdriver acting as a proxy.

The idea is that the browser can execute the automation faster and more reliably then web driver could, and the use of native browser protocol allows access to more advance browser features such as local cache, geo-locations and the http communication.

But puppeteer only supports javascript as scripting language, and that was harsh limitation for many automation developers, so later on Pyppeteer was introduced, although not officially by google.

So robotframework-puppeteer is actually a wrap around Pypeteer that allows easy keyword integration with robotframework.

*** Settings ***
Library    PuppeteerLibrary
*** Test Cases ***
Submit login form
    &{options} =    create dictionary   headless=${False}
    Open browser    http://127.0.0.1:7272/login-form-example.html   options=${options}
    Input Text    id=exampleInputEmail1    demo@qahive.com
    Input Text    id=exampleInputPassword1    123456789
    Click Element    id=exampleCheck1
    Run Async Keywords
    ...    Wait For New Window Open    AND
    ...    Click Element    css=button[type="submit"]
    Switch Window    NEW
    Wait Until Page Contains    Login succeeded
    Close Browser

Run Async Keywords is a killer feature!
It allows executing several operation concurrently, and since our modern browser applications are ruining concurrent routines, our automation should be able to hook in to these routines(or their by-products), a feature totally missing in selenium by design.

*** Settings ***
Library    PuppeteerLibrary
*** Test Cases ***
Submit login form
 Open browser ${HOME_PAGE_URL} options=${options}  
 Input Text id:username foo  
 Input Text id:password bar  
 Run Async Keywords Click Element id:login_button AND 
 ... Wait For Request Url ${URL_API}/login POST username=demo

In this example we are clicking a button and awaiting for the HTTP request to be sent by the browser.
This helps with better timing of the automation and avoid flakiness in our test execution, as well as validating the application at different stages.

Robotframework-Browser

link — https://robotframework-browser.org/

In January 2020, Microsoft announced the release of Playwright, a Nodejs web automation package.

Essentially, the two main contributors of puppeteer moved from google to Microsoft and began to work on playwright.
just like puppeteer, playwright uses CDP to automate the browser and it supports either javascript or type-script.

Robotframework Browser uses type-script functions that will be executed by the browser, and then python functions provide the API for robotframework keywords, and the inter communication between them is done via gRPC

gRPC, being bidirectional and binary based rather then text based, is a perfect protocol for this purpose.

In addition to that, browser supports amazing assertions syntax that make your tests super readable and maintainable.

*** Settings ***
Library   Browser
*** Test Cases ***
Example Test
    New Page    https://playwright.dev
    Get Text    h1    ==    🎭 Playwright

In this test case we are navigating to playwrights home page and asserting that it contains an h1 element with the text Playwright.

*** Settings ***
Library   Browser
*** Test Cases ***
Example Test
  New Page    http://somewebsite.com
  ${promise}=    Promise To    Wait For Response
  Click    \#delayed_request
  ${body}=    Wait For    ${promise}

In this example we are running a promise asynchronously.
It is similar to the Run Async Keywords we saw with the puppeteer library.

In this case specifically, we are clicking a button and then awaiting for an http response to return and catch it’s body.

Once again, help validating the web application and time our automation for a more reliable automation.

*** Settings ***

Library          Browser
*** Test Cases ***
Positive Test
    New Page    http://homepage.com
    Fill Text  #Username   my-user
    Fill Text  #Password   1234
    Click      #Login
    Wait Until Network Is Idle

Wait Until Network Is Idle is a killer feature!
Some times you want to make sure that the network is idle after you complete a set of action.
This keyword allows you to do that and add more robustness to your tests!

Another awesome feature of browser library is the ability to extend it with javascript.

async function myGoToKeyword(page, args) {
  await page.goto(args[0]);
  return await page.title();
}
exports.__esModule = true;
exports.myGoToKeyword = myGoToKeyword;

and now in robotfroamework:

* Settings *
Library   Browser  jsextension=${CURDIR}/module.js
* Test Cases *
Hello
  New Page
  ${title}=  myGoToKeyword  https://playwright.dev
  Should be equal  ${title}  Playwright

Playwbot

Link — https://pypi.org/project/playwbot/

As Playwright javascript package gained popularity, Microsoft fired off yet another bomb — they developed a python package for playwright!

So Playwbot is kind of the new kid on the block, providing a wrap to the python implementation of playwright.

*** Settings ***
Library  playwbot.Playwbot
*** Test Cases ***
Positive Test
 ${page}= New Page ${context} 
 ${selector}= Convert To String xpath=/some-selector 
 ${element}= Query Selector ${page} ${selector}
 ${status}= Is Visible ${element} 
 Should Be True ${status}==True

Playwbot is still new and under development, but I’m personally curious to see how it will go :)

Conclusion

Robotframework is a rich eco-system providing many different tools for different tasks.

For browser automation, robotframework offers an amazing tool belt for you to pick, each has it’s strengths and weaknesses.

As more tools are introduced, robotframeworks community keep extending it and you as a developer will enjoy the benefits.