End-to-end testing

End-to-end browser tests are a great way to ensure the functionality of self-developed apps or bundles. The mapapps-4-developers project already contains some end-to-end tests based on the Playwright framework in the file sample_camera.spec.ts. You can use the tests as a template to write your own tests.

The following sections describe how you can execute the tests in mapapps-4-developers.

Installation of Playwright

To install Playwright and its dependencies run the following command:

Install Playwright

$ npx playwright install --with-deps

We also recommend that you install the VS Code extension for Playwright, as this allows you to run and debug tests directly from your IDE, among other things.

Test execution

After installing Playwright, you can run the tests as described below.

Start the development server

$ mvn compile -Denv=dev -Pinclude-mapapps-deps

Run the tests

npm run e2e-test

A test report is printed to the console, as shown in the following example:

Run the tests

Running 6 tests using 2 workers

  ✓  1 [Desktop Firefox] > src/test/end-to-end/sample_camera.spec.ts:31:5 > expect 3D widget shows zoom, heading and tilt (5.5s)
  ✓  2 [Desktop Firefox] > src/test/end-to-end/sample_camera.spec.ts:15:5 > expect 2D widget shows zoom and rotation (6.2s)
  -  3 [Desktop Firefox] > src/test/end-to-end/sample_camera.spec.ts:52:6 > expect map extent is correct when zoomed in
  ✓  4 [Mobile Chrome] > src/test/end-to-end/sample_camera.spec.ts:15:5 > expect 2D widget shows zoom and rotation (4.3s)
  ✓  5 [Mobile Chrome] > src/test/end-to-end/sample_camera.spec.ts:31:5 > expect 3D widget shows zoom, heading and tilt (4.1s)
  -  6 [Mobile Chrome] > src/test/end-to-end/sample_camera.spec.ts:52:6 > expect map extent is correct when zoomed in

  2 skipped
  4 passed (13.7s)

To open last HTML report run:

  npx playwright show-report target/end-to-end/reports/html

If you run the npx playwright show-report target/end-to-end/reports/html command described above, you will see an HTML report produced by Playwright. If any errors occurred, the report will help you to analyze them.

You might have noticed that no test browsers were opened during the test execution. The reason is, that the e2e-test script runs tests in headless mode.

To show the browsers during the test execution you can run the tests in headed mode using the following command:

Run the tests headless

npm run e2e-test:headed

You can also run the tests via the Playwright UI, which can be started as follows:

Open Playwright UI

npm run e2e-test:ui

If you are using the Playwright VS Code Extension, you can also start the tests directly from the IDE.

Test configuration

The configuration of the tests is contained in playwright.config.ts. Among other things, it defines on which browser engines and on which viewport the tests are to be executed:

playwright.config.ts

{
projects: [
        {
            name: "Desktop Firefox",
            use: {
                ...devices["Desktop Firefox"],
                headless: headless
            }
        },
        {
            name: "Mobile Chrome",
            use: {
                ...devices["Galaxy S9+"],
                headless: headless,
                launchOptions: chromiumLaunchOptions
            }
        }
    ],
}

If left unchanged, tests are executed on Firefox using a desktop device and on Chromium using an emulated mobile device.

The file is extensively commented, take a look at it.

Screenshot comparisons

Playwright offers the possibility to compare web applications or parts of them against expected screenshots. The file sample_camera.spec.ts contains a test that performs such a comparison.

test.skip('expect map extent is correct when zoomed in', async ({ page }, testInfo) => {
    ...
    await expectToMatchScreenshot(page, "expect-map-extent-is-correct-when-zoomed-in.png");
})

However, the test is deactivated on delivery because expected screenshots are not contained in the project, which would cause the test to fail on the first run. If you want to try out the test, remove the suffix .skip from the test. When the test is executed for the first time, a screenshot is taken which is used as the expected result for subsequent test executions.

If you use screenshots in your tests, please note that screenshots may differ depending on the runtime environment. They should therefore be used with caution. Playwright offers a number of possible assertions that may be a better alternative.

Prepared utility functions

In the file testUtils.ts you will find some other useful functions:

You can use the waitForMap function to wait until the map has finished loading. This function is used, for example, in the test which uses screenshots for comparison.
The function expectToMatchScreenshot enables stable screenshot comparisons by using custom intervals