Between versions 2.13.6 and 2.14.0, the Medusa admin UI continued to function normally. All 15 scenarios passed. But browser-side API responses changed in ways no UI test would catch — specifically, additive field changes to two domain resources. We discovered this with scout; the analysis below is BoxProbe's editorial — the raw scout report is linked at the bottom.
categories/categories-crudPOST /admin/product-categories+ $.product_category.external_id: null
Same field also appears on:
GET /admin/product-categories/{id} (×2), POST /admin/product-categories/{id}
Adding `external_id: string | null` follows a Medusa 2.14 pattern of exposing third-party identifier fields on domain objects (also observed on Collection in the same release). The field is null on creation through the admin UI, which suggests the path is server-set via integrations, not user-set via the dashboard. UI continues to render correctly without it — but downstream integrations parsing this response with strict schema validation would encounter an unexpected field.
A small integration test in Medusa's existing Jest suite asserting `external_id` presence in the `POST /admin/product-categories` response body. No new runtime dependencies, no browser runner — the kind of regression fixture that locks the contract without imposing tooling on maintainers.
collections/collections-crud · POST /admin/collections + $.collection.external_id: null
Also on:
GET /admin/collections, GET /admin/collections/{id} (×2)
intentional · same pattern as ProductCategory
Confirms the headline finding is part of a deliberate architectural change in 2.14: external_id is becoming a uniform attribute across multiple domain models, not a one-off addition. Worth lock-down for the same downstream-integration reasons.
Values like `address_1: "test-a9a1e3" → "test-283ab7"` and `handle: "test-cf100e" → "test-93a998"` differ because the scenario uses randomized test input each run, not because Medusa's behavior changed.
In production use of scout, these get suppressed via `diff_ignore.json` value-type rules (`mock_name`, etc.). They surface here intentionally to demonstrate scout's separation of noise filtering from real findings — and the result is that 7 "value diffs" become 0 once filtered, leaving the 2 real structural findings clearly visible.
Test code lives in github.com/boxprobe/scout-medusa.
BoxProbe's default approach for upstream contributions: the artifact submitted upstream is a native test in the project's existing framework — no scout dependency, no new CI workflow, no generated reports in the upstream repo. scout is the discovery layer; the upstream test is what maintainers own.
Every number above is reproducible. scout is the open-source executor, scout-medusa is the demo repo with Docker compose for both Medusa versions plus all 15 scenarios.
# 1. Clone the demo repo
git clone https://github.com/boxprobe/scout-medusa
cd scout-medusa
# 2. Start both Medusa versions (5-10 min first time while images build)
bash compose/start.sh
# 3. Install scout
pip install boxprobe-scout
playwright install chromium
# 4. Record against baseline (v2.13.6 @ localhost:19000)
cd admin
scout run scenarios/ --web-version 2.13.6
# 5. Record against target (v2.14.0 @ localhost:29000)
scout run scenarios/ --web-version 2.14.0 \
--web-base-url http://localhost:29000/app \
--api-base-url http://localhost:29000
# 6. Diff
scout runs
scout diff <baseline-id> <target-id>scout produces a self-contained HTML report alongside this analysis: filterable endpoint table, popup body diffs per row, structural / value / status categorization, known-change suppression. Use it if you want to inspect every paired endpoint, not just the headline findings.