ZeuZ

Automated UI Testing: Why Tests Break and How to Fix Them

Key Takeaways

  • When a UI change breaks a test, Test Studio writes the fix and your team approves or rejects the diff.
  • Durable locators identify elements through UI change.
  • A virtual cursor shows exactly where the agent clicked, and you can take over the browser at any point.
  • Console, network, and DOM are captured as the test runs.
  • If you want the function run for you, our managed QA runs all six steps while the suite and sign-off stay with you.
Blog

Share with your community!

Automated UI Testing: Why Tests Break and How to Fix Them

Automated UI testing uses software to drive an application's interface the way a user would, clicking buttons, filling in forms, and checking what appears on screen, so a team can confirm that each release still works in every browser without repeating the same checks by hand.

The hard part is not writing the first test. It is keeping hundreds of them passing when the interface changes every sprint. Most broken UI tests fail for a handful of known reasons, and each one has a fix. This guide covers the browser and web app side of gui testing, where those failures are most common.

We build Test Studio, an AI test automation platform, so we see these failures daily. To keep the advice concrete, we ran every example in this guide on Playwright 1.63 against a small checkout page we changed between versions.

What is automated UI testing?

Automated UI testing checks an application through its user interface rather than through its code or APIs. A test opens the app, acts as a user would, and checks the result a user would see.

That makes UI tests the closest thing to a real user in your pipeline. They catch problems that unit and API tests miss: a button hidden behind a banner, a form that submits but shows no confirmation, or a page that works in Chrome but not in Safari.

It also makes them the most fragile tests you own. A UI test depends on the page structure, the timing of every request, the test data, and the browser. When any of those shifts, the test can fail even though the product still works.

Why do automated UI tests break?

Automated UI tests break for five main reasons, and usually only one of them means the product is broken. The table below lists each cause, how it shows up, and where the fix lives.

Cause

How it shows up

Real bug?

Where the fix lives

Brittle locators

"Element not found" after a redesign, even though the button is still on the page

No

The test's locators

Timing

The test passes locally and fails in CI, or fails one run in ten

No

Waits and assertions

Test data and state

The test fails only after another test ran, or only on a shared environment

No

Data setup and test isolation

Environment

The test fails in one browser, screen size, or build agent

Sometimes

Browser and environment setup

Real product change

The test fails because the app now behaves differently

Yes, or an intended change

The product, or the test if the change was intended

Five causes of broken UI tests: brittle locators, timing, test data, environment and real product changes

The first four are usually test problems. They cost time because each one looks like a product failure until someone opens the evidence. The rest of this guide takes them one at a time.

How do you tell a real bug from a broken test?

Tell a real bug from a broken test by reading the evidence from the failed run before you rerun it. A rerun that passes hides the cause, and the same failure comes back next week.

Triaging a failed UI test by reading screenshots, snapshots, console and network evidence before rerunning

What the evidence shows

Likely cause

First thing to check

The element exists on the screenshot, but the test could not find it

Brittle locator

Compare the locator with the current page structure

The element appeared a moment after the test gave up

Timing

Look for fixed waits and checks that run only once

A console error or failed network request at the same moment

Real bug or environment problem

The failing request and its response

The page shows another test's data, or a record already exists

Test data and state

How and where the data was created

The test fails in one browser only

Environment, or a real browser bug

Reproduce by hand in that browser

Playwright's trace viewer records the locator used for each action, DOM snapshots before and after each step, network requests, and console logs. Setting trace: 'retain-on-failure' keeps that record for failed tests only, which is enough to answer most of the questions above without rerunning anything.

How do you fix tests broken by locators?

Fix locator failures by finding elements the way a user finds them, by role and visible name, instead of by page structure. Playwright's own locator guide recommends role locators first and warns that CSS and XPath selectors "can break when the DOM structure changes."

Here is a checkout test written two ways. The first finds the button by its position in the page structure and waits a fixed half second for the confirmation:

TYPESCRIPT

import { test, expect } from '@playwright/test';
test('place order (brittle)', async ({ page }) => {
await page.goto('/');
await page.locator('div.actions > button.btn-primary').click();
await page.waitForTimeout(500);
expect(await page.locator('#msg').textContent()).toBe('Order placed');
});

The second finds the button by its role and name, and uses an assertion that keeps checking until the text appears:

TYPESCRIPT

import { test, expect } from '@playwright/test';
test('place order (durable)', async ({ page }) => {
await page.goto('/');
await page.getByRole('button', { name: 'Place order' }).click();
await expect(page.getByRole('status')).toHaveText('Order placed');
});

We ran both tests against three versions of the same checkout page:

Page version

What changed

Brittle test

Durable test

Version 1, fast confirmation

Nothing (baseline, confirmation after 300 ms)

Passed

Passed

Version 1, slow confirmation

Confirmation took 800 ms instead of 300 ms

Failed: text was still empty after the fixed wait

Passed

Version 2, redesign

New wrapper, new classes, new ID, and a second button added

Failed: timed out waiting for the old selector

Passed

Version 3, renamed button

Button text changed to "Complete purchase"

Not run

Failed: no button named "Place order"

Brittle structure-based locators breaking after a redesign while role and name locators keep working

The last row matters. The durable test still failed when the button's visible name changed, and that is the right result: a user would notice the change too, so someone should confirm it was intended. Good locators do not stop every failure. They stop the failures that tell you nothing.

When a role and name are not unique, add a test ID that developers agree not to change. Playwright calls test IDs "the most resilient way of testing," because they are an explicit contract between the app and the tests.

How do you fix timing failures?

Fix timing failures by removing fixed waits and replacing them with checks that wait for a condition. A fixed wait is a guess about how long the app will take, and in our run, a half-second guess failed the moment the confirmation took 800 ms.

  • Use auto-waiting actions. Before a click, Playwright checks that the element is visible, stable, enabled, and not covered by another element, and waits until those checks pass.

  • Use retrying assertions. Assertions such as toHaveText() and toBeVisible() keep checking until the condition holds or the timeout runs out. A plain expect(value).toBe() checks once.

  • Wait for the result, not the request. Assert on what the user sees, such as a confirmation message, rather than on an internal event the user never notices.

  • Set timeouts on purpose. A longer timeout for a known slow step is fine. A longer timeout across the whole suite only hides problems.

Selenium users can get the same effect with explicit waits. The principle is the same in every tool: wait for a condition, never for a number of seconds.

How do you fix failures caused by test data and environments?

Fix data and environment failures by making each test set up what it needs and clean up after itself, so it passes in any order and on any machine.

  • Create data per test. Create the user, order, or record a test needs through an API call at the start of the test, rather than relying on data left by a previous run.

  • Avoid shared accounts. Two tests that log in as the same user and change the same cart will collide when they run in parallel.

  • Reset state between tests. Start each test in a fresh browser context, so cookies and local storage from one test cannot leak into the next.

  • Pin browser versions in CI. Run the same browser versions locally and in CI, and upgrade them on purpose rather than by surprise.

  • Fix the viewport. Set the screen size in configuration. A layout that collapses into a mobile menu at a smaller width will hide the buttons your tests expect.

Failures that appear in only one browser need a separate look, because some of them are real bugs. Checking the same flows across Chrome, Firefox, Edge, and Safari is the subject of cross-browser testing, and it is worth running as its own scheduled suite.

Where does UI test automation fit in a test suite?

UI test automation fits at the top of the suite, covering the journeys a user takes from start to finish, while unit and API tests cover the logic underneath. Most fragile UI suites are fragile because they test too much through the UI.

  • Test journeys, not rules. Check that a user can sign up, search, and pay. Check every validation rule on the sign-up form with faster API or unit tests.

  • Keep the UI suite small enough to run on every merge. A suite that takes hours gets run nightly, and nightly failures are harder to trace to a change.

  • Move setup below the UI. Log in and create test data through APIs, then use the UI only for the steps under test.

  • Run a smoke subset first. A few critical journeys on every pull request, and the full UI suite before each release.

UI tests are usually the largest part of a regression suite, and our guide on how to automate regression testing covers how to choose and schedule them. The same split applies outside the browser: IoT software testing, for example, checks dashboards through the UI and device data through APIs.

What should you look for in UI testing tools?

The best UI testing tools are the ones that make the five failure causes cheap to diagnose and fix. Compare automated UI testing tools on how they handle failure, not only on how fast they write a first test.

What to check

Why it matters

Question to ask

Locator strategy

Decides how many tests break on a redesign

Does it find elements by role, label, or test ID, or only by CSS and XPath?

Waiting

Decides how many tests fail at random

Does it wait for conditions on its own, or do you add waits by hand?

Evidence on failure

Decides how long triage takes

Do you get screenshots, DOM, console, and network data for every failure?

Repair workflow

Decides what maintenance costs

When a test breaks, who fixes it, and can you review the change first?

Browser coverage

Decides what you can sign off

Does it run Chrome, Firefox, Edge, and Safari, and on which systems?

Who can write tests

Decides who owns the suite

Can testers and product people write cases, or only developers?

Be wary of any tool that claims tests "fix themselves" without a review step. A change that silently rewrites a test can turn a real bug into a passing run. Ask to see exactly what the tool changes and who approves it. Our gui testing checklist covers the screens, states, and controls to include once you have chosen a tool.

What parts of UI testing stay with your team?

UI testing tools take on test automation and test execution. Test design, test case creation, QA reporting and analysis, and test maintenance and optimization stay largely with your team, whichever tool you pick.

Step

What UI testing tools give you

What stays with your team

01 Test design

Little, in most tools

Choosing which journeys, browsers, and screen sizes matter

02 Test case creation

Recorders and code generators

Writing cases a reviewer can check against requirements

03 Test automation

Locators, actions, and assertions

Choosing durable locators and keeping test data independent

04 Test execution

Local runs, parallel runs, and CI

Browser versions, environments, and schedules

05 QA reporting and analysis

Reports, screenshots, and traces

Triage of each failure and the release decision

06 Test maintenance and optimization

Better waiting and locator suggestions

Repairing tests after UI changes and retiring duplicates

UI suites cost the most at step 06. Every interface change lands there, and a suite that nobody repairs is soon a suite nobody trusts.

How does Test Studio handle UI test failures?

Test Studio, our AI test automation platform, is built around the failures this guide describes, with a person approving every change.

  • Repairs you approve. When a UI change breaks a test, Test Studio writes the fix and your team approves or rejects the diff. Nothing repairs itself silently.

  • Durable locators. Durable locators identify elements through UI change, so a new wrapper or class name does not break the test.

  • Evidence from every run. Console, network, and DOM are captured as the test runs, so triage starts from the evidence, not a rerun.

  • A view of what the agent did. A virtual cursor shows exactly where the agent clicked, and you can take over the browser at any point.

  • Coverage. Test Studio covers web and mobile web in Chrome, Firefox, Edge, and Safari, plus Windows and macOS desktop applications and API checks, in one suite.

  • Price. Test Studio is $220 per seat per month, flat, with no metering on test runs.

See Test Studio repair a broken UI test, with the diff ready for your review. Book a demo to try it on your app, or download the solution brief for the detail.

Which option is right for your team?

Which option is right for your team depends on who will keep the UI suite passing:

  • You have engineers who will own the suite. Apply the locator, timing, and data fixes above, and budget time every sprint for test maintenance and optimization.

  • You have tooling and a mandate but no bandwidth, or a hard environment. Our forward deployed engineers work inside your team, each carrying a Test Studio license, and are measured on a baseline agreed in writing in week one. Talk to the team.

  • You want the function run for you. If you want the function run for you, our managed QA runs all six steps while the suite and sign-off stay with you. It is the service behind outsourced QA at ZeuZ.

Frequently asked questions

Why do automated UI tests fail randomly?

Most random failures come from timing. A test that waits a fixed number of seconds, or checks a value only once, passes when the app is fast and fails when it is slow. Shared test data and tests that depend on run order cause the rest. Replace fixed waits with checks that wait for a condition, and give each test its own data.

Talk to the team about stabilizing a flaky UI suite →

Should you use XPath in UI test automation?

Use XPath only when nothing better exists. XPath and CSS selectors often depend on the page structure, so they break when developers add a wrapper or rename a class. Locators based on role and visible name, or on agreed test IDs, survive most redesigns. Keep XPath for rare cases, such as tables with no accessible labels.

Book a demo to see durable locators survive a redesign →

How many UI tests should a project have?

Fewer than most teams write. Cover the journeys users depend on, such as sign-up, search, and checkout, and test detailed rules through faster API and unit tests. A good guide is a UI suite small enough to run on every merge. If the suite takes hours, move checks below the UI before adding more.

Talk to the team about sizing your UI suite →

Can AI fix broken UI tests?

AI can propose a fix, for example by finding the renamed button a locator now misses. The risk is a fix applied without review, which can turn a real bug into a passing test. Use tools that show the change as a diff and let a person approve or reject it, so the suite stays honest.

Book a demo to see a repair proposed as a diff →

Is manual UI testing still needed if you automate?

Yes. Automation is best at repeating known checks on every build. People are still better at exploring new features, judging whether a layout looks right, and spotting confusing wording. Most teams automate the stable journeys and spend manual time on new screens, usability, and anything that changed in the current release.

Download the solution brief to see where Test Studio fits →

How long does it take to fix a broken UI test?

A locator fix takes minutes once you know the cause. The time goes into finding the cause: rerunning the test, reproducing it by hand, and checking whether the product changed. Keeping screenshots, DOM snapshots, console logs, and network data from every failed run cuts most of that time, because the cause is usually visible in the evidence.

Talk to the team about cutting your test maintenance time →

Related Blogs