Menu
×
   ❮   
HTML CSS JAVASCRIPT SQL PYTHON JAVA PHP C C++ C# AWS W3.CSS HOW TO BOOTSTRAP REACT MYSQL JQUERY EXCEL XML DJANGO NUMPY PANDAS NODEJS DSA TYPESCRIPT ANGULAR ANGULARJS GIT POSTGRESQL MONGODB ASP AI R GO KOTLIN SWIFT SASS VUE GEN AI SCIPY CYBERSECURITY DATA SCIENCE INTRO TO PROGRAMMING HTML & CSS BASH RUST TOOLS MATH

Playwright Continuous Integration


Share

Playwright in Continuous Integration

Continuous Integration (CI) means a server runs your tests automatically, every time someone pushes code or opens a pull request.

To push code means to send your commits to the shared Git repository, and a pull request asks to merge a change into the main branch.

A broken change is caught before it is merged, not after it reaches users.

Playwright runs in any CI system: GitHub Actions, GitLab CI, Azure Pipelines, Jenkins, CircleCI and others.

This chapter uses GitHub Actions, because npm init playwright can set it up for you.


The GitHub Actions Workflow

When you create a project with npm init playwright@latest, it asks if you want a GitHub Actions workflow.

If you answer yes, it creates this file:

Example

.github/workflows/playwright.yml

name: Playwright Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v6
    - uses: actions/setup-node@v6
      with:
        node-version: lts/*
    - name: Install dependencies
      run: npm ci
    - name: Install Playwright Browsers
      run: npx playwright install --with-deps
    - name: Run Playwright tests
      run: npx playwright test
    - uses: actions/upload-artifact@v5
      if: ${{ !cancelled() }}
      with:
        name: playwright-report
        path: playwright-report/
        retention-days: 30

You can also create the file yourself.

Commit it, push, and GitHub runs it.



Workflow Explained

  • on: run on every push and pull request to the main or master branch
  • runs-on: ubuntu-latest: run on a fresh Linux machine from GitHub
  • actions/checkout: get your code
  • actions/setup-node: install Node.js (the latest LTS version, LTS means Long Term Support)
  • npm ci: install the exact package versions from package-lock.json
  • npx playwright install --with-deps: download the browsers, and the Linux system libraries they need
  • npx playwright test: run the tests
  • actions/upload-artifact: save the HTML report as an artifact (a file kept after the run), so you can download it from the run page
  • if: ${{ !cancelled() }}: upload the report even when tests failed, which is exactly when you need it

To read the report, open the workflow run on GitHub, download the playwright-report artifact, unzip it and run npx playwright show-report with the folder name.

Note: Commit package-lock.json.

npm ci needs it, and it makes CI use the same Playwright version as your computer.


CI Settings in the Config

Most CI systems set the environment variable CI.

The config made by npm init playwright uses it to behave differently in CI:

Example

playwright.config.js

// @ts-check
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: 'html',
  use: {
    baseURL: 'https://www.w3schools.com/playwright/practice/',
    trace: 'on-first-retry',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
  ],
});
SettingIn CIWhy
forbidOnlytrueFail the run if someone left a test.only in the code, which would skip all other tests
retries2Retry failed tests, and mark tests that pass on retry as flaky (see Retries)
workers1Run one test at a time, which is more stable on small CI machines
traceon first retryRecord a trace when a test is retried, so you can debug it (see Trace Viewer). This setting is the same everywhere, but only CI has retries

On your own computer CI is not set, so you get no retries and as many workers as your CPU allows.

If your CI machines are large, you can raise workers, for example to '50%' of the CPU cores.


Sharding

When you have many tests, one machine can be too slow.

Sharding splits the tests into parts, and runs each part on its own machine at the same time.

The --shard option picks one part.

You can try it on your own computer, here with the 5 tests from the Page Object Model chapter:

Example

npx playwright test --shard=1/2
npx playwright test --shard=2/2

Output:

Running 3 tests using 3 workers, shard 1 of 2

[1/3] [chromium] › tests/login.spec.js:12:1 › wrong password shows an error
[2/3] [chromium] › tests/login.spec.js:5:1 › login works
[3/3] [chromium] › tests/todo.spec.js:5:1 › can add todos
  3 passed (5.3s)

Running 2 tests using 2 workers, shard 2 of 2

[1/2] [chromium] › tests/todo.spec.js:14:1 › can complete a todo
[2/2] [chromium] › tests/todo.spec.js:25:1 › can delete a todo
  2 passed (2.0s)

Together the two shards ran all 5 tests, and no test ran twice.

In GitHub Actions, a matrix is a list of values, and GitHub starts one job for each value.

Here it starts one job per shard:

Example

    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    steps:
    # ... checkout, setup-node, npm ci, install browsers
    - name: Run Playwright tests
      run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}

fail-fast: false lets the other shards finish when one shard fails.

Each shard makes its own report.

To get one report for the whole run, use the blob reporter (see File Reporters) in the shards, upload the blob-report folders, and join them in a last job with npx playwright merge-reports.


The Playwright Docker Image

Microsoft publishes a Docker image with Node.js, the browsers and all their system libraries already installed:

mcr.microsoft.com/playwright:v1.65.1-noble

The tag is the Playwright version, and noble is the Ubuntu version (24.04).

Use the same version as @playwright/test in your package.json, or the browsers will not match.

In GitHub Actions, run the job inside the image with container.

Then you can skip the Node.js setup and the browser install:

Example

jobs:
  test:
    runs-on: ubuntu-latest
    container:
      image: mcr.microsoft.com/playwright:v1.65.1-noble
    steps:
    - uses: actions/checkout@v6
    - run: npm ci
    - run: npx playwright test

The image is also the easiest way to make screenshot baselines on Windows or Mac that match the Linux screenshots in CI.


Tips for CI

  • Start with the generated workflow, and change it only when you need to
  • Always upload the HTML report, and keep trace: 'on-first-retry', so you can debug failures that only happen in CI
  • If CI only needs one browser, install just that one: npx playwright install --with-deps chromium
  • Keep secrets, like test passwords, in your CI system's secret store, and read them with process.env
  • A flaky test (one that sometimes passes and sometimes fails) is a real problem: fix it instead of adding more retries


Free account

Track your progress

XP earned 0 Day streak 0 Spaces 0 League —

New

Learn in Adventure Mode W3Schools Adventure App

Coding fundamentals as bite-sized lessons and challenges.

×

Contact Sales

If you want to use W3Schools services as an educational institution, team or enterprise, send us an e-mail:
sales@w3schools.com

Report Error

If you want to report an error, or if you want to make a suggestion, send us an e-mail:
help@w3schools.com

W3Schools is optimized for learning and training. Examples might be simplified to improve reading and learning. Tutorials, references, and examples are constantly reviewed to avoid errors, but we cannot warrant full correctness of all content. While using W3Schools, you agree to have read and accepted our terms of use, cookies and privacy policy.

Copyright 1999-2026 by Refsnes Data. All Rights Reserved. W3Schools is Powered by W3.CSS.