November 3, 2024 • 12 min read

Playwright vs. Cypress - comparison of E2E testing frameworks

A detailed comparison of two popular testing E2E frameworks - Playwright and Cypress. What features are comparable, and which of them offers more?

Last updated: November 3, 2024

Recently, I moved my website to a new technological stack. I ditched my old Gatsby website and created a new project in Next.js. It was a great effort in itself. I wondered if it was the right time to rewrite my tests. E2E tests are abstract, so after the migration, I would only need to change tests a little (or not). However, instead of changing things incrementally - like a normal human being - I changed everything simultaneously, including tests. It took me a lot of time, but I ended up with two sets of similar E2E tests - in Cypress and Playwright. I decided to take advantage of this opportunity and compare those two frameworks.

Playwright

Playwright is a modern testing library developed by Microsoft and released in 2020. The team that initially worked on another automation tool, Puppeteer, created Playwright. "Hold on, isn't Puppeteer from Google?" It is, but the team moved from Google to Microsoft. It's another tool to your tech stack from Microsoft if you don't have enough already.

Playwright consists of two parts:

• Playwright Library that provides unified APIs for launching and interacting with browsers.
• Playwright Test that provides all those APIs plus a fully managed end-to-end test runner.

Even though it has a much shorter lifespan than Cypress, the library gained over 66k stars on GitHub (compared to 47k in the Cypress repo). There is a good reason behind this popularity. Playwright offers full feature parity with Cypress and more.

Cypress

Cypress, an end-to-end testing framework for web applications, was first released in September 2017. Since its release, it has gained significant popularity in the developer community, amassing over 40,000 stars on GitHub. The framework is developed and maintained by the Cypress.io team, a group of experienced developers and engineers including Gleb Bahamutov. I stumbled on his blog posts about testing multiple times - good stuff. Cypress is still a go-to choice for many developers looking for an E2E testing framework.

Playwright vs. Cypress

Let's move on to the meat of this post - comparison. I prepared a table listing different features you may expect from a modern E2E testing tool. The table may be helpful per se, but I'll review each feature and add context. I'll start with "more," where Playwright offers extra functionality, and I'll move to the feature parity.

Programming language

Playwright supports TypeScript out of the box. You can install it and start writing your type-safe tests. JavaScript is also supported (obviously). By default, you write tests in JavaScript using Cypress. However, you can use Cypress with TypeScript by providing additional configuration.

Other languages

Besides JavaScript, Playwright supports other programming languages like Python, C# or Java. It doesn't matter much to me - I'm creating this website in the JavaScript ecosystem. But having one testing framework for the whole team - front-end and back-end developers - may be a plus. While on the subject - I wondered if I should put Java support as a plus (JavaScript developer shitting on Java - oh, irony). Cypress does not (and probably won't) support other languages.

Parallelization

By default, Playwright Test runs test files in parallel. It spins several worker processes that run simultaneously. Tests in a single file run in order, but you can even parallelize tests in a single file. Parallelization is a paid feature in Cypress Cloud.

Browsers

Parallelization helps with cross-browser testing. The Playwright framework offers the concept of "projects," each of which can serve as a different browser, including Safari.

Defining projects in PlaywrightTS
1projects: [
2  {
3    name: 'chromium',
4    use: { ...devices['Desktop Chrome'] }
5  },
6  {
7    name: 'firefox',
8    use: { ...devices['Desktop Firefox'] }
9  },
10  {
11    name: 'webkit',
12    use: { ...devices['Desktop Safari'] }
13  },
14  {
15    name: 'Mobile Chrome',
16    use: { ...devices['Pixel 5'] }
17  },
18  {
19    name: 'Mobile Safari',
20    use: { ...devices['iPhone 12'] }
21  }
22]

A modern-day Internet Explorer may introduce exotic bugs, so it's crucial to support it. It's super easy to configure different projects with Playwright globally. On the other hand, Cypress is a little hassle. Here's a fragment of my CI GitHub action running Cypress with multiple browsers.

GitHub action running Cypress with multiple browsersYAML
1e2e-test-chrome:
2    runs-on: ubuntu-latest
3    strategy:
4      matrix:
5        node: [16]
6    needs: build
7    steps:
8      - name: Checkout commit
9        uses: actions/checkout@v3
10      - name: Use Node ${{ matrix.node }}
11        uses: actions/setup-node@v3
12        with:
13          node-version: ${{ matrix.node }}
14          cache: npm
15      - name: Install dependencies
16        run: npm ci --legacy-peer-deps
17      - name: Restore Cypress binary
18        uses: actions/cache@v3
19        id: cache-cypress
20        with:
21          path: ~/.cache/Cypress
22          key: cypress-binary-${{ hashFiles('**/package-lock.json') }}
23          restore-keys: |
24            cypress-binary-
25      - name: Download build artifacts
26        uses: actions/download-artifact@v3
27        with:
28          name: build-output
29      - name: Run Cypress
30        uses: cypress-io/github-action@v4
31        with:
32          start: npm run serve
33          wait-on: 'http://localhost:8000'
34          browser: chrome
35          install: false
36  e2e-tests-firefox:
37    runs-on: ubuntu-latest
38    strategy:
39      matrix:
40        node: [16]
41    needs: build
42    steps:
43      - name: Checkout commit
44        uses: actions/checkout@v3
45      - name: Use Node ${{ matrix.node }}
46        uses: actions/setup-node@v3
47        with:
48          node-version: ${{ matrix.node }}
49          cache: npm
50      - name: Install dependencies
51        run: npm ci --legacy-peer-deps
52      - name: Restore Cypress binary
53        uses: actions/cache@v3
54        id: cache-cypress
55        with:
56          path: ~/.cache/Cypress
57          key: cypress-binary-${{ hashFiles('**/package-lock.json') }}
58          restore-keys: |
59            cypress-binary-
60      - name: Download build artifacts
61        uses: actions/download-artifact@v3
62        with:
63          name: build-output
64      - name: Run Cypress
65        uses: cypress-io/github-action@v4
66        with:
67          start: npm run serve
68          wait-on: 'http://localhost:8000'
69          browser: firefox
70          headed: true
71          install: false

There is no Safari because Cypress hasn't supported Safari for a long time. Only recently did it get experimental support for WebKit (Safari's browser engine).

Other test runners

Playwright Test is a dedicated test runner for the framework. However, with a few lines of code, you can hook Playwright up to other existing test runners like Jest, Mocha, or AVA. Cypress does not support third-party runners.

Multiple tabs

Playwright uses BrowserContext, which provides a way to operate multiple independent browser sessions. You can effortlessly write tests that interact with numerous tabs. Cypress runs inside a browser - therefore, it won't support multiple browser tabs.

Code generation

I wouldn't call it a killer feature, but it's cool. Playwright allows you to generate code. You start recording, click around your app, and, for example, it will create locators for you (locators represent a way to find elements on a page similar to JavaScript selectors). It may not generate a production-ready code, but it can be an easy way to start. Cypress doesn't provide such a feature.

Performance

On the contrary, performance is important for most people. Playwright directly interacts with the browser using the Chrome dev tools protocol, which translates into better performance. I took advantage of my two sets of almost identical tests and did some performance testing. I did the testing using UIs for both frameworks and the Chromium browser. To avoid outliers, I ran each set of tests several times and took the average time. For Cypress, it is the overall time, and for the Playwright, I took the time of the longest test because they run in parallel.

Of course, it depends on many factors, but Playwright is roughly 3-4 times faster than Cypress.

iframe support

A page in Playwright exposes its current frame tree via specific methods. You have access to the iframe element out of the box. Cypress also supports iframes but requires an additional plugin.

Dialogs

A similar story is with native browser dialogs. Playwright auto-dismisses dialogs, but you can register a dialog handler and easily accept them. Cypress can also handle dialogs with events. However, the code depends on the type of dialog.

VSC plugin

Both frameworks offer a supporting plugin for Visual Studio Code. Microsoft develops an official plugin for Playwright. Using Cypress, you have to be satisfied with community plugins.

Syntax

The syntax preferences are subjective, so it's hard to point out the winner here. Playwright offers syntax similar to modern asynchronous ES6 JavaScript, with many async and await keywords. Cypress tests remind jQuery-like syntax with many chained methods. Below I present one E2E test written for both frameworks.

Subscription test in PlaywrightTS
1import { test, expect } from './fixtures'
2
3test.describe('Subscription tests', () => {
4  test('checks the successful flow', async ({ page, settingsPage }) => {
5    const { section } = await settingsPage.getDictionary('en')
6    const formURL = await settingsPage.getFormURL('en')
7
8    await page.route(formURL, (route) =>
9      route.fulfill({
10        status: 200,
11        contentType: 'application/json',
12        body: JSON.stringify({ status: 'success' })
13      })
14    )
15    await page.goto(settingsPage.link.home)
16
17    const input = page.getByLabel(section.newsletter.email)
18    const button = page.getByRole('button', { name: section.newsletter.button })
19
20    await expect(input).toHaveAttribute('required')
21    await expect(input).toHaveAttribute('type', 'email')
22    await expect(input).toHaveAttribute('autocomplete', 'off')
23
24    await input.fill(settingsPage.example.email)
25    await button.scrollIntoViewIfNeeded()
26    await button.click()
27
28    await expect(
29      page.getByText(section.newsletter.success.heading)
30    ).toBeVisible()
31  })
32
33  // More test cases
34})

Subscription test in CypressJS
1/// <reference types="Cypress" />
2import { icon } from '../fixtures/theme.json'
3import { form, user } from '../fixtures/subscription.json'
4
5describe('Subscription tests', () => {
6  beforeEach(() => {
7    cy.visit('/blog/hello-world/')
8    cy.findByTestId(icon).should('exist')
9    cy.findByPlaceholderText(form.email)
10      .as('email')
11      .should('have.prop', 'type', 'email')
12      .and('have.prop', 'required')
13    cy.findByRole('button', { name: form.button }).as('button')
14  })
15
16  it('Tests success flow', () => {
17    cy.intercept(form.url, (req) => {
18      expect(req.body).to.include('email_address')
19      expect(req.body).to.include(user.email)
20      req.reply({
21        statusCode: 200,
22        body: {
23          status: 'success'
24        }
25      })
26    })
27    cy.get('@email').type(user.email, { delay: 50 })
28    cy.get('@button').click()
29    cy.findByText(form.success, { exact: false }).should('be.visible')
30  })
31
32  // More test cases
33})

Design

In this section, I refer to the overall design of the framework. Playwright is headless by design. GUI is an addition. You don't need to configure anything to run Playwright on CI. There is a flag to run Playwright in GUI mode.

🔴  🟡  🟢
npx playwright test --ui

Cypress was designed with the opposite philosophy. GUI is at the center of the experience. You need to adjust the configuration to run Cypress on a CI server.

Interactions with elements

Both Playwright and Cypress offer good APIs to interact with DOM elements. Playwright introduces locators. It's a way to find elements on the page at any moment. If you've ever used the Testing Library, you'll feel at home here.

Interacting with elements in PlaywrightTS
1await page.getByRole('button', { name: 'Submit' }).click()

In Cypress, you can get elements using... the get function. There is also a way to add meaningful names to elements with aliases.

Interacting with elements in CypressJS
1cy.get('button[type=submit]').as('submitBtn')
2
3cy.get('@submitBtn').click()

Auto-wait

Referring to the previous section, both frameworks will automatically wait for those elements (contrary to Selenium). This feature helps test modern, dynamic UIs where components are often attached and detached from the DOM. Because of that, tests are less likely to be flaky.

Shadow DOM

Regarding components, Playwrights' locators will, by default, work with elements in shadow DOM. It's a necessary feature if you have an application with those custom native web components. Cypress can also interact with shadow DOM elements. However, you need to invoke an extra method for that.

API testing

Besides testing UIs, both frameworks allow for API testing. Playwright can access your application REST API. It may help when you want to test your server API or prepare server-side state before visiting the application. Cypress also provides a good developer experience for testing APIs.

Documentation

Both frameworks offer in-depth and broad technical documentation. While working with them, I rarely needed to browse other sources. It's hard to point out why, but I prefer Cypress documentation. Maybe because of the aesthetic? And don't get me wrong - Playwright has good documentation. I quickly picked up and rewrote my E2E tests using it. But it's tough competition - Cypress's documentation is just so good.

Summary

There won't be any plot twists because I wrote about rewriting my tests in Playwright at the beginning. So, you know my choice already. After reading this post, you know the reasons behind this decision, too (I hope). Playwright offers full feature parity with Cypress and extra functionality like parallelization or multi-tab support, which is handy for me. However, saying all that, Cypress is still a great choice. Its popularity is not a coincidence. And how about you - what framework do you choose?