Kolaboro / Engineering / Visual tests
Visual regression tests
Every day a browser photographs our 20 most important pages on a phone and on a desktop and compares each picture, pixel by pixel, with the one from the day before. If a page moved, broke or lost something, it shows up here.
Loading the latest run...
How it works
- Pages: the same 20 key pages that Lighthouse checks every day (listed in /data/lighthouse.json), at 390 x 844 px (a phone, with touch) and 1440 x 900 px (a desktop): 40 screenshots per run.
- Capture: Chromium driven by Playwright, device pixel ratio 1, reduced motion, animations off, time zone UTC. We capture the top of each page, up to 2400 px tall on the phone and 1800 px on the desktop. The moving "shipping daily" ticker is covered with a grey box and the cookie banner is hidden, so they do not count as changes.
- Compare: each screenshot is compared with the previous run using pixelmatch (colour threshold 0.1, anti-aliasing ignored). A page counts as changed when more than 0.3% of its pixels differ, or when its captured height changed.
- Evidence: for changed pages we publish one small image with three panels: before, after and the differences in red. At most 6 images per run, largest change first, stored inside the results file. The full-size screenshots stay out of the website: they roll forward from run to run in the build cache and each run's set is kept as one archive for 30 days.
- Schedule: every day at 06:41 UTC, and whenever the test code changes.
Limits
- Pages show live numbers and dates. When a count changes, the pixels change too, so some "changed" results are new data, not bugs. A person looks at the image to decide.
- Only the top part of each page is compared. The rest of every page is covered by the daily crawl for broken links, script errors and sideways scrolling (see /status).
- Screenshots come from Chromium on a Linux build machine. Safari and Firefox can draw text slightly differently and are not part of this test.
Related: page weight, real-user speed, accessibility, reliability.