BoxProbe
Open source · MIT

Scout is the runner. We open-sourced it because anything that touches your CI and records your API traffic should be auditable before it runs.

BoxProbe maintains scout as the open-source executor for UI-driven API regression testing. Pixel-anchored locators, no LLM in the hot path, nothing leaves the machine running it.

View on GitHub PyPI · boxprobe-scout Docs
Install
pip install boxprobe-scout
playwright install chromium

Python ≥ 3.11. Tested on 3.11, 3.12, 3.13, and 3.14.

Why open source
01

Deterministic, free to run

Pixel-anchored Locators resolve to viewport coordinates by pure math, not probabilistic selector matching. No LLM in execution. No tokens consumed, no per-request API fees. Cheap enough to run on every PR or nightly without anyone asking about cost.

02

Audit before adopt

Every Python file is reviewable. The recording proxy is a child process on localhost. Nothing leaves the machine running scout — no telemetry, no upload, no account. If something looks off, read the source.

03

Narrow on purpose

scout catches one specific class of bug: API behavior drift that survives your existing tests because it only manifests through real UI interaction. Not a contract tester, not a load test runner, not a unit-test replacement.

Quickstart
Run a scenario, get a diff
# 1. Set up an app config (app.json) + a Python scenario (test.py)
# Format: https://boxprobe.github.io/scout/quickstart/

# 2. Record against baseline
scout run scenarios/ --web-version 1.0.0

# 3. Deploy your app, then record target
scout run scenarios/ --web-version 1.1.0

# 4. Diff the two runs
scout runs                          # list run IDs
scout diff <baseline-id> <target-id>
#   → opens HTML report in your browser
CI / CD integration
Two commands, any pipeline

scout has no special CI plugin to install. It's a CLI — drop two steps into your existing workflow.

# In your CI workflow (any system):
# Replace 1.0.0 / 1.1.0 with whatever your version labels are.

- run: pip install boxprobe-scout
- run: playwright install chromium
- run: scout run scenarios/ --web-version $BASELINE_VERSION
- run: scout run scenarios/ --web-version $TARGET_VERSION
- run: scout diff $(scout runs --json | jq ...)

JUnit XML output is generated alongside the HTML diff for CI integrations that expect it.

Try it on real software
A reproducible Medusa demo

We maintain a worked example against real Medusa: two pinned versions running side by side under docker compose, 15 admin scenarios, one diff report.

github.com/boxprobe/scout-medusa See the case study
Resources