ZeuZ

How to Use Claude Code for QA Automation, Step by Step

Key Takeaways

  • Give Claude Code a green Playwright baseline, a CLAUDE.md file, and a way to see the page before you ask it for tests.
  • Playwright MCP suits exploration, while the Playwright CLI skill uses fewer tokens in large codebases, so choose by job.
  • Review generated tests like any pull request, and break the app on purpose to prove each test can fail.
  • Claude Code writes test code well. It has no test estate: no catalog, run history, schedules, governed data, or evidence trail.
  • In Test Studio the test lands with an ID, reusable steps, variables, a branch, a schedule, and a run history.
  • Our forward deployed engineers work inside your team, each carrying a Test Studio license, and are measured on a baseline agreed in writing in week one.
Blog

Share with your community!

To use Claude Code for QA automation, install it in your test repository, give it a CLAUDE.md file with your testing conventions, connect the Playwright MCP server so it can drive a real browser, then have it write, run, and fix tests that your team reviews like any other pull request.

That workflow is a practical first step toward autonomous software testing, where an agent plans, writes, runs, and repairs tests while people approve the result. This guide gives you the working version: every step has a command, a file, or a prompt you can paste, and the limits come after the steps instead of before them.

We ran the Playwright config, the test file, and the MCP and agent setup commands against a small demo sign-in app, using Playwright 1.56 and Claude Code 2.1. The prompts, the skill file, and the CI workflow follow Anthropic's and Playwright's own documentation, linked at each step.

What do you need before you start?

You need a current Node.js release, a Claude account or API access that includes Claude Code, a repository with an application you can run locally, and a test user the browser can sign in with.

  • Node.js. Playwright lists the latest 22.x, 24.x, or 26.x as supported.

  • A Claude account. Anthropic's quickstart accepts a Claude Pro, Max, Team, or Enterprise subscription, a Claude Console account, or access through a supported cloud provider.

  • A runnable app and a test user. Point the tests at a local or staging environment, never production, and keep the test account's credentials in environment variables.

  • A git repository. Claude's work arrives as a diff you can review, and Anthropic notes that its checkpoints are not a replacement for git.

How do you tell Claude Code your testing conventions?

Put your testing conventions in a CLAUDE.md file at the root of the repository, because Claude Code reads it at the start of every session. Anthropic's guidance is to include the commands Claude cannot guess, the style rules that differ from defaults, and your testing instructions, and to leave out anything Claude can work out by reading the code.

MARKDOWN

# Testing conventions

- Framework: Playwright Test, TypeScript. Tests live in tests/. Config: playwright.config.ts.

- Run one file: npx playwright test tests/<file>.spec.ts. Run all: npx playwright test.

- Locators: prefer getByRole, getByLabel, and getByText. Use CSS or XPath last, with a comment.

- Assertions: web-first only, such as await expect(locator).toBeVisible(). No waitForTimeout.

- Tests must be independent: no shared state, and each test creates its own data.

- Credentials come from E2E_EMAIL and E2E_PASSWORD. Never print or hard-code them.

- Never weaken an assertion to pass a test. First say whether the test or the app is wrong.

Run /init to generate a starter file from your repository, then trim it, and run /context to confirm Claude loaded it. Keep it short: the memory documentation targets under 200 lines per file because longer files reduce how well Claude follows them. It also says CLAUDE.md is context, not enforced configuration, so anything that must never happen belongs in a hook or a permission rule, covered below.

How do you set up Playwright MCP for Claude Code?

Set up Playwright MCP for Claude Code with one command, which registers a server that lets Claude open pages, read them, and act on them in a real browser.

BASH

claude mcp add --scope project playwright -- npx -y @playwright/mcp@latest

claude mcp list

The --scope project flag writes the server into a .mcp.json file at the repository root, so teammates get the same setup. Without it, claude mcp add stores the server at the default local scope instead of sharing it through the repository. This is the file we got:

JSON

{

  "mcpServers": {

    "playwright": {

      "type": "stdio",

      "command": "npx",

      "args": [

        "-y",

        "@playwright/mcp@latest"

      ],

      "env": {}

    }

  }

}

Claude Code asks you to approve a project-scoped server the first time you start a session. Until you do, Claude mcp get playwright to report it as pending approval. Once it is approved, ask Claude to open your login page and describe what it sees.

Playwright MCP works from the page's accessibility tree, not from pixels: each tool returns a structured snapshot of elements, roles, and text, with a reference Claude can act on. That is close to how getByRole finds elements, so what Claude sees maps well onto the locators you want in the test.

There is a cost. Playwright's own comparison says MCP suits specialized agentic loops and exploratory automation, and that the Playwright CLI with skills is the better fit for coding agents such as Claude Code in large codebases, because MCP keeps its tool schemas and page snapshots in context and uses more tokens.

Should you use a Playwright skill for Claude Code instead of MCP?

A Playwright skill for Claude Code is the lighter option: it teaches Claude the Playwright CLI's commands and loads on demand, while MCP keeps its tool definitions in every conversation. Which to choose depends on the job, and you can run more than one.

Option

What it is

Use it when

Trade-off

Playwright MCP

A server that drives a browser through the accessibility tree

Claude should explore a live page, confirm locators, and iterate in one session

Higher token cost, per Playwright's comparison

Playwright CLI skill

playwright-cli plus a skill that teaches Claude its commands

Claude works across a large codebase and you want lower token use

One more tool to install and keep current

Playwright test agents

Planner, generator, and healer agents from npx playwright init-agents --loop=claude

You want separate plan, write, and repair roles

In our run on 1.56 it replaced an existing .mcp.json

A skill of your own

A SKILL.md that encodes your team's test-writing procedure

The same request keeps coming back, such as "add a test for this flow"

You maintain it like code


To install the official skill, add the CLI and then the skill:

BASH

npm install -g @playwright/cli@latest

playwright-cli install --skills

Per Playwright's installation guide, the second command copies a playwright-cli skill into .claude/skills/ in your workspace. Add -g to put it in ~/.claude/skills for every project instead.

A skill of your own is worth writing when you repeat the same request. Anthropic's best-practices guide puts rules that apply everywhere in CLAUDE.md and repeatable workflows in skills. Save this as .claude/skills/new-e2e-test/SKILL.md:

MARKDOWN

---

name: new-e2e-test

description: Write a Playwright test for a user flow, run it, and report the result

disable-model-invocation: true

---

Write a Playwright test for this flow: $ARGUMENTS

 

1. Read tests/login.spec.ts and follow its structure and locator style.

2. Open the page with the Playwright tools and confirm each locator before using it.

3. Write one file in tests/, one behavior per test, with web-first assertions.

4. Run npx playwright test <file>. If it fails, say if the test or the app is wrong.

5. Report the command you ran and its output. Do not edit existing tests.

Run it with /new-e2e-test checkout with a saved card. The disable-model-invocation: true line means Claude runs the skill only when you call it.

Playwright also ships test agents, introduced in version 1.56. One command scaffolds them for Claude Code:

BASH

npx playwright init-agents --loop=claude

In our run it wrote three agent files, playwright-test-planner.md, playwright-test-generator.md, and playwright-test-healer.md, into .claude/agents/, plus a tests/seed.spec.ts and a .mcp.json with a playwright-test server. It replaced the .mcp.json we already had, so run it first or merge the two servers by hand. The documentation describes the planner as producing a test plan, the generator as turning it into tests, and the healer as repairing failing ones.

How do you get Claude Code to write your first test?

Ask for one flow at a time, name the file, state what a pass looks like, and tell Claude to run the test and show the output before it reports back. Anthropic's guidance is that giving Claude a check it can run is the difference between a session you watch and one you walk away from.

TEXT

Using the Playwright tools, open http://localhost:3000/login and write

tests/login.spec.ts with two tests. First, signing in with the E2E_EMAIL and

E2E_PASSWORD test account lands on /dashboard under the heading "Welcome back".

Second, a wrong password shows the alert "Invalid email or password" and stays

on /login. Follow CLAUDE.md. Then run npx playwright test tests/login.spec.ts

and show me the output.

A reviewable result looks like the file below. It passed against our demo app, and it uses role and label locators, web-first assertions, and credentials from the environment:

TYPESCRIPT

import { test, expect } from '@playwright/test';

 

const email = process.env.E2E_EMAIL!;

const password = process.env.E2E_PASSWORD!;

 

test.describe('sign in', () => {

  test.beforeEach(async ({ page }) => {

    await page.goto('/login');

  });

 

  test('signs in with valid credentials', async ({ page }) => {

    await page.getByLabel('Email').fill(email);

    await page.getByLabel('Password').fill(password);

    await page.getByRole('button', { name: 'Sign in' }).click();

 

    await expect(page).toHaveURL(/\/dashboard/);

    await expect(page.getByRole('heading', { name: 'Welcome back' })).toBeVisible();

  });

 

  test('shows an error for a wrong password', async ({ page }) => {

    await page.getByLabel('Email').fill(email);

    await page.getByLabel('Password').fill('not-the-password');

    await page.getByRole('button', { name: 'Sign in' }).click();

 

    await expect(page.getByRole('alert')).toHaveText('Invalid email or password');

    await expect(page).toHaveURL(/\/login/);

  });

});

Do not merge on a green run alone. Review the file against Playwright's best practices:

Check

Why it matters

Red flag

Locators

Role and label locators survive markup changes better than CSS paths

A selector such as .btn-primary > span:nth-child(2)

Assertions

Web-first assertions retry until the condition holds

expect(await locator.isVisible()).toBe(true)

Waits

Fixed sleeps cause flaky tests and slow suites

waitForTimeout(3000)

Independence

Each test should pass alone and in any order

A test that needs data another test created

Credentials and data

Secrets and real customer data stay out of the repository

A literal password in the file

Intent

The test should check the requirement, not just today's behavior

An assertion that copies whatever the page shows now


Then break the app on purpose. Change the expected text or the button label and confirm the test fails. A test that cannot fail is not a test, and it is an easy mistake to miss in generated code.

How do you run the suite and let Claude Code fix failures?

Limited access

Run the suite with npx playwright test, point Claude at the failure, and ask it to decide whether the test or the application is wrong before it edits anything.

To see a real failure, we renamed the button in our demo app from "Sign in" to "Log in" and ran the suite. After the default 30-second timeout, Playwright reported:

TEXT

Error: locator.click: Test timeout of 30000ms exceeded.

Call log:

  - waiting for getByRole('button', { name: 'Sign in' })

It also wrote an error-context.md file with a page snapshot, which includes the line button "Log in". Claude can read both. If the renaming was intentional, the fix is a one-line change to the locator. If it was not, the right output is a bug report, not an edited test. That judgment is why the prompt asks for a decision first:

TEXT

Run npx playwright test. For each failure, read the error and the page snapshot

in test-results/. Tell me whether the test or the app is wrong. Fix only the

tests that are wrong, keep every assertion at least as strict as before, and

show me the diff.

Three habits keep this loop honest. Read the diff for loosened assertions, because the easiest way to turn a test green is to check less. Confirm a fix is not luck by running npx playwright test --repeat-each=10, which repeats every test ten times. And consider a gate that does not depend on Claude's judgment: Anthropic describes a Stop hook that runs your check as a script and blocks the turn from ending until it passes.

How do you limit what Claude Code can read and run?

Limit Claude Code with permission rules in .claude/settings.json: allow the commands the workflow needs, and deny the files and commands it should never touch.

JSON

{

  "permissions": {

    "allow": [

      "Bash(npx playwright test *)",

      "Bash(npm run *)"

    ],

    "deny": [

      "Read(./.env)",

      "Bash(git push *)"

    ]

  }

}

The permissions documentation shows the same pattern: allow npm run commands, deny git push, and deny reads of .env files. Deny rules are checked first. They are still a guard rail, not a wall. The documentation notes that a push written another way, such as git -C . push, is not matched by a git push rule, so use a hook when an action must never happen.

How do you run Claude Code in CI with GitHub Actions?

claude-code-write-run-fix-review-loop

Run your Playwright suite in CI exactly as you would without Claude, then add Claude Code as a separate workflow that responds to @claude comments or runs a headless command.

The initializer's GitHub Actions workflow already does the first part. Playwright's CI guide describes a workflow that runs on pushes and pull requests to main or master, installs dependencies and browsers, runs npx playwright test, and uploads the playwright-report folder as an artifact. Keep that job as your gate.

For the Claude side, Anthropic's GitHub Actions documentation gives this workflow, which reacts when someone mentions @claude in a comment. Add your key as the ANTHROPIC_API_KEY repository secret first:

YAML

name: Claude Code

on:

  issue_comment:

    types: [created]

  pull_request_review_comment:

    types: [created]

jobs:

  claude:

    if: contains(github.event.comment.body, '@claude')

    runs-on: ubuntu-latest

    permissions:

      contents: write

      pull-requests: write

      issues: write

      id-token: write

      actions: read

    steps:

      - uses: actions/checkout@v6

        with:

          fetch-depth: 1

      - uses: anthropics/claude-code-action@v1

        with:

          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}

A comment such as "@claude the checkout test fails on this branch, find out why and open a fix" then starts a run. For scripts, the headless mode runs one prompt and exits. Start read-only, so Claude diagnoses and does not edit:

BASH

claude -p "Run npx playwright test and report any failures with a likely cause" \

  --allowedTools "Bash(npx playwright test *),Read" \

  --permission-mode dontAsk

The --allowedTools flag pre-approves only what the job needs, and --permission-mode dontAsk denies everything else, which matters when no one is there to answer a prompt. Whatever Claude proposes should still arrive as a pull request that a person reviews and that your normal suite checks.

What does Claude Code cover across the six testing steps, and what does it leave with your team?

claude-code-covers-vs-team-owns

Claude Code does its best work in test automation, and it helps with the other five testing steps without owning any of them. This table shows where the line falls.

Step

What Claude Code does

What stays with your team

01 Test design

Reads a spec, ticket, or pull request and proposes scenarios. The Playwright planner agent writes a test plan for it

Deciding which risks matter, what the acceptance criteria are, and what is out of scope

02 Test case creation

Turns a described flow into test steps and spec files, through the generator agent or your own skill

Reviewing intent, and keeping a catalog with IDs, owners, and links to requirements. Claude Code holds none of that

03 Test automation

Writes, runs, and corrects Playwright code in your repository

Conventions, review, test data, environments, and the decision to merge

04 Test execution

Runs npx playwright test locally, or inside a CI workflow

Schedules, parallel capacity, environments, and secrets. CI keeps the logs, and Claude Code has no run history of its own

05 QA reporting and analysis

Summarizes a failure from the error and page snapshot. Playwright's HTML report and trace viewer cover a single run

Trends across runs, release decisions, and an evidence trail that people outside the team can read

06 Test maintenance and optimization

Updates locators and waits when a UI change breaks a test, as the healer agent is built to do

Deciding whether a failure is a product bug, reviewing every diff, and retiring duplicate or low-value tests


In short, Claude Code writes test code well, but it holds no test estate of its own: no catalog, run history, schedules, governed data, or evidence trail. Whatever of those you have comes from your repository and your CI provider, and your team keeps it current.

What goes wrong when an AI agent writes and fixes your tests?

The most common failure is a test that passes without proving much, and review is where it gets caught. In Sonar's survey of more than 1,100 developers, 96% said they do not fully trust that AI-generated code is functionally correct, yet only 48% always check AI-assisted code before committing it. Tests are code, so the same gap applies to them.

  • Weakened assertions. A "fix" that loosens a check turns the run green and hides the defect. Guard against it with the CLAUDE.md rule above and a diff review.

  • Tests that encode a bug. If Claude writes a test from what the page does today, it confirms today's behavior, including mistakes. A person must own the requirement the test checks.

  • Invented locators and steps. Have Claude confirm each locator in a real browser through MCP or the CLI skill, then run the test.

  • Flaky passes. One green run proves little. Repeat the test before you trust it.

  • Exposed secrets and data. Use environment variables, deny rules, and a staging environment with test accounts.

If the product under test is itself built on a language model, assertions need a different approach, which is the subject of how to test generative AI applications.

When is Claude Code enough, and when do you need a test platform?

Claude Code is enough when one team owns a modest set of web tests in one repository and a developer reviews every change. It stops being enough when tests need an owner, a schedule, a history, and an evidence trail that people outside the team can read.

Claude Code is a good fit when:

  • Developers write and review the tests themselves.

  • The suite lives in one repository and targets one or two web applications.

  • A pull request and a CI log are all the evidence anyone asks for.

A platform starts to matter when:

  • QA, product, and compliance need to see what is covered and what ran, without opening a repository.

  • Several teams need shared steps, governed test data, and scheduled runs.

  • Packaged enterprise applications are in scope. If that describes your estate, you are more likely comparing platforms such as Tricentis Tosca than coding assistants.

We build Test Studio, so weigh this section with that in mind. Rebuilding a test estate around a coding assistant is a platform project, not a prompt: you add a catalog, scheduling, governed test data, run history, and reporting, then maintain all of it. An AI test automation platform starts from that estate. In Test Studio the test lands with an ID, reusable steps, variables, a branch, a schedule, and a run history.

See Test Studio run a test against your own build. Book a demo to see it on your application, or download the solution brief for the detail.

If you would rather work with people who have built this before, our forward deployed engineers work inside your team, each carrying a Test Studio license, and are measured on a baseline agreed in writing in week one. Talk to the team.

If the question is wider than testing, an AI SDLC transformation is a bounded engagement: it maps where the hours go across your lifecycle and hands the capability back to your team. It starts with an SDLC assessment.


How do you install Claude Code and start it in your test repository?

Install Claude Code with Anthropic's native installer, confirm it with claude --version, then run claude from the root of your test repository and sign in when it asks.

BASH

curl -fsSL https://claude.ai/install.sh | bash

claude --version

cd /path/to/your/test-repo

claude

That command is for macOS, Linux, and WSL. The quickstart lists the PowerShell, CMD, Homebrew, and WinGet equivalents. On first use Claude Code opens a browser to complete login, and it stores your credentials afterward.

Start with a question, not a task. Ask "what does this project do?" and check that its answer matches your repository, because that shows what it can see. For anything larger than one test, press Shift+Tab until the status bar shows plan mode, or start with claude --permission-mode plan. In plan mode Claude reads files and proposes a plan without editing anything, which Anthropic recommends for work that touches several files. For a single small test, skip it.


How do you set up Claude Code with Playwright?

claude-code-setup-claude-md-playwright-mcp

Set up Claude Code with Playwright by starting from a normal Playwright project, because Claude can then read a working config, run a real test, and copy the patterns already in the repository.

If your repository has no Playwright yet, run the initializer and accept the defaults:

BASH

npm init playwright@latest

npx playwright test

npx playwright show-report

The initializer asks whether to use TypeScript or JavaScript, what to call the tests folder, whether to add a GitHub Actions workflow, and whether to install browsers. Choose TypeScript and the workflow, which we use later. It creates playwright.config.ts and an example test in tests/.

Then edit the config so it starts your app and records evidence when something fails. This version ran against our demo app:

TYPESCRIPT

import { defineConfig, devices } from '@playwright/test';

 

export default defineConfig({

  testDir: './tests',

  fullyParallel: true,

  forbidOnly: !!process.env.CI,

  retries: process.env.CI ? 2 : 0,

  reporter: [['html'], ['list']],

  use: {

    baseURL: 'http://localhost:3000',

    trace: 'on-first-retry',

  },

  projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }],

  webServer: {

    command: 'npm run start',

    url: 'http://localhost:3000',

    reuseExistingServer: !process.env.CI,

  },

});

The webServer block starts your app before the tests run, baseURL lets tests use short paths such as /login, and trace: 'on-first-retry' records a trace the first time a test is retried, so CI failures leave something to inspect.

Run npx playwright test yourself and confirm it passes before Claude touches anything. A green baseline is what lets you tell a mistake Claude introduced from a failure that was already there.

Related Blogs