Skip to content

runtime instruments: fixes proven on a live Mendix 11.8 app (#818, #812, page-audit) - #825

Open
MendixMau wants to merge 4 commits into
masterfrom
fix/runtime-instruments-818-812-pageaudit
Open

MendixMau wants to merge 4 commits into
masterfrom
fix/runtime-instruments-818-812-pageaudit

Conversation

@MendixMau

@MendixMau MendixMau commented Nov 20, 2026 •

Copy link
Copy Markdown
Owner

Fixes #818
Fixes #812

Direct lane

What changed and why (one paragraph):
Three runtime-instrument defects, each reproduced on a live Mendix 11.8.0 app running in a container (mxcli run --local --db-type hsqldb) and fixed in its own commit.
(1) #818. journey-runner.js's opt-in ui.no5xx row counted Mendix validation feedback as a server error. A refused save answers HTTP 551 with {"datavalidation":[…]}.

  • The listener now reads only a 551's body, through a new helpers.isValidationFeedback. It has the same shape as fix(monkey): score HTTP 551 datavalidation feedback as info, not crash (#803) #814's monkey classifier.
  • A datavalidation answer becomes a note on the row.
  • A 500 still FAILs. So does a 551 whose body is anything else or cannot be read. An unknown 551 is not proven to be feedback, so FAIL is the safer reading.
  • record5xx now awaits the body reads.
    (2) [bug] fullPageShot crops Atlas pages at the viewport: content scrolls .mx-scrollcontainer-center, not the document #812. helpers.fullPageShot measured the document. On 11.8, Atlas_SideBar and Atlas_TopBar keep the document viewport-tall and scroll .mx-scrollcontainer-center, so the shot was cut at the viewport. Atlas_Default does scroll the document, so the fix has to handle both shapes.
  • fullPageShot picks the outermost scroller that hides content and spans at least half the viewport. It grows the viewport by that scroller's hidden height (width unchanged), shoots, and restores the viewport.
  • The grow is kept only if that same element grew with the window. A fixed-height scroller, such as a 480 px grid body, falls back to plain fullPage; the record names it in skippedScroller.
  • complete is the root's own leftover scroll height.
  • It returns a record: scrollRoot, contentPx, capturedPx, complete, error.
  • It no longer silently swallows a failed shot. It retries a transient Chromium "Unable to capture screenshot" once.
    (3) page-audit P1. DOM_PROBE's [class*="widget-datagrid"] matched a DataGrid2's root plus eight of its parts. live/empty-list-message then reported "9 lists on screen; 4 empty" on a 20-row grid. It now matches the .widget-datagrid class token.

Impact

Row Answer
Process none. No gate, step or artifact is added. One journey verdict can flip FAIL → PASS: a no5xx: true step whose only 5xx was a 551 datavalidation answer. That is the fix. One page-audit verdict can flip FAIL → PASS: live/empty-list-message on a page whose only "empty lists" were DataGrid2 parts.
Dependencies Touches project-tests/e2e/helpers.js (shared by every e2e instrument), journey-runner.js and page-audit.js. New export helpers.isValidationFeedback. fullPageShot now returns a record instead of undefined; its only caller (full-app-walkthrough.js:183) ignores the return value. Order vs #814: no textual conflict. #814 defines its own isValidationFeedback in monkey.js, so whichever merges second can import the helper (a one-line follow-up). No new tools or artifact reads.
Context cost none: no routed or always-loaded file touched (no skills/, no skill-routing.tsv).
Behaviour n/a: scripts only.
Runtime Only fullPageShot costs extra, and only on an inner-scroll page: one viewport resize, a 250 ms settle, and the restore. On the live app the shot took about 0.4 s against the captured page. A 551 adds one body read. Document-scroll pages and steps without a 551 are unchanged.

Time cost: per inner-scroll full-page shot · ~0.3 s measured on a 4-CPU Linux container · can force rework: n · offset: replaces a shot that silently lost everything below 900 px

Fixtures: each new one FAILs on master's file and PASSes on the branch

Fixture (new) master branch
tests/wave2/test-journey-551-validation.sh PASS=3 FAIL=2 (no isValidationFeedback; the captured-551 step FAILs) PASS=7 FAIL=0, 5 runs
tests/wave2/test-fullpageshot-inner-scroll.sh PASS=4 FAIL=6. Plain SideBar shot is 900 px for 1244 px of content. The fixed-grid cases fail on the record only: master returns no record. Previous branch head 734d421d: 6/4, with the shot ballooning PASS=10 FAIL=0, 5 runs
tests/wave2/test-page-audit-datagrid-count.sh PASS=1 FAIL=2 ("9 lists on screen; 4 empty") PASS=3 FAIL=0, 5 runs

Golden input: all captured, none hand-written (tests/wave2/fixtures/runtime-mx118/, provenance in CAPTURE.txt):

  • xas-551-empty-required.json: the live runtime's verbatim /xas/ answer (status, headers, body) to Save with the required Title empty.
  • overview-SideBar-1280x900.mhtml.gz and overview-Default-1280x900.mhtml.gz: CDP MHTML snapshots (DOM plus CSS) of the same 20-row DataGrid2 overview under both layouts. Chromium renders them from file:// with no app.
  • The 551 fixture's stand-in /xas/ replays the captured answer, next to two controls that must still FAIL: a 500 and a non-datavalidation 551.

Existing fixtures that target the changed files. All were run at the branch, scoped:

  • Unchanged result:
    • test-journey-back-no5xx 12/0. Its wiring grep now accepts await record5xx();. That one-line regex change is in commit 1.
    • test-e2e-visible-wait 8/0
    • test-oql-rows 15/0
    • test-design-audit-nav-map 26/0
    • test-journey-control-exit 12/0
    • test-journey-device-net-geo 26/26
    • test-journey-id 12/0
    • test-journey-liveness 36/0
    • test-journey-persona 4/0
    • test-journey-preflight 75/0
    • test-journey-pseudo-text 5/5
    • test-journey-rerun 15/15
    • test-journey-row-click 7/7
    • test-journey-timeout 18/18
    • test-look-ledger 35/0
    • test-monkey-left-app 22/0
    • test-monkey-oracle 14/0
    • test-monkey-template-nav 10/0
    • test-named-controls 37/0
    • test-navto-missing-item rc 0
    • test-otel-capture-until 4/0
    • test-scope-ensure 6/0
    • test-affected-tests 160/0
    • test-harness-scan 8/8
  • Same failure on a pristine origin/master worktree, so not caused by this PR:
    • test-stack-up-local: another agent's app held :8080 on the shared box.
    • test-sync-past-stock
    • test-verify-module-446 21/22 (V3: playwright rows=1)

Field evidence. A live Mendix 11.8.0 app (blank app plus a Tasks module) scaffolded with bin/init-project.sh. It was booted with mxcli run --local --watch --db-type hsqldb under the runtime lock and stopped afterwards. Master's and the branch's project-tests/e2e/*.js were installed into the same app in turn:

Instrument on the live app master branch (734d421; fullPageShot rows re-run at 7070fa4)
journey-runner.js journeys/TasksNeg.journey.json (empty-title save, no5xx: true, delta: 0) PASS 5 FAIL 1: FAIL … no HTTP 5xx during the step — 1 answer(s) >= 500: 551 POST http://localhost:8080/xas/ PASS 6 FAIL 0: PASS … no answer >= 500 (1 validation answer(s), HTTP 551 datavalidation, not counted: 551 POST http://localhost:8080/xas/)
journey-runner.js journeys/Tasks.journey.json (happy path) PASS 10 FAIL 0 PASS 10 FAIL 0
full-app-walkthrough.js shot heights (artifacts/app/*.png) sidebar 1440×900, topbar 1440×900, raise (Atlas_Default) 1440×1278 sidebar 1440×1244, topbar 1440×1239, raise 1440×1278. The SideBar shot now shows rows 16–20 and the "1 to 20 of 35" paging bar. Walkthrough 18 pass / 3 fail on both: these are pre-existing, unrelated failures (act checks run before the act's nav; the single-module rule).
H.fullPageShot direct at 1280×900 / 390×844 Default 1278 / 2658, SideBar 900 / 844, TopBar 900 / 844 Default 1278 / 2658, SideBar 1244 / 1329, TopBar 1239 / 1243 (each equal to the measured .mx-scrollcontainer-center scrollHeight). At 7070fa4, 1280×900: Default 1278, SideBar 1244, TopBar 1239
H.fullPageShot with a 480 px fixed-height grid body injected (the review's case) 900 / 900 734d421: Default 4196, SideBar 2148, complete: true. 7070fa4: Default 900, SideBar 900 (the page content), skippedScroller: div.widget-datagrid-grid-body, viewport restored
page-audit.js --page Tasks.Task_Overview_SideBar live/empty-list-message FAIL "9 lists on screen; 4 empty"; verdicts 20 pass / 6 fail PASS "1 lists on screen; 0 empty"; 21 / 5. The five left are real static findings on this probe page: no empty-state class, no H1, no card, page-name suffix, no wireframe.

Not changed, flagged for follow-up (out of this PR's scope):

  • page-audit.js:590 takes its own page.screenshot({ fullPage: true }) and has the same [bug] fullPageShot crops Atlas pages at the viewport: content scrolls .mx-scrollcontainer-center, not the document #812 crop on inner-scroll layouts.

  • [class*="widget-gallery"] in the same DOM_PROBE selector is the same substring shape. It probably over-counts Gallery parts and items. It is left unchanged because no Gallery DOM was captured, and rule 1 forbids a hand-written fixture.

  • No client data anywhere in the diff. The fixtures are a demo app on localhost. The leak guard is clean. It warned on two new capitalised words, "Grown" and "TasksNeg"; both are generic.

  • Size cap: 3 code files, about 100 lines of instrument code; the rest is fixtures, captures and CHANGELOG

  • Test tier reached: T2–T3. Scoped fixtures ran on captured input, and every instrument ran against a live Mendix 11.8 runtime.

  • For a new/changed instrument: the golden input is captured. Path resolution is untouched, so both layouts behave as before. The fixtures use url.pathToFileURL and plain gzip/sed, so they are Git-Bash-safe. They have not been run on macOS or Windows.

  • For a new skill: n/a

  • CHANGELOG.md line appended in each commit, credited (bin/check-pr-discipline.sh clean)

  • New bug entries: none

Review fix (7070fa4). The independent review's verdict was MERGE-AFTER-FIX: the scroll-root pick ballooned on fixed-height scrollers. Fixed as listed above. The fixture now also re-runs both captures with one appended CSS rule that makes .widget-datagrid-grid-body a fixed 480 or 800 px scroller. In the 800 px SideBar case, the outermost root (the centre) must win over the grid body. CAPTURE.txt and the fixture header now give the capture's real row count: 30 rows, 20 per page. Scoped re-runs:

  • the 3 new fixtures, 5 runs each, all passing;
  • back-no5xx 12/0, e2e-visible-wait 8/0, oql-rows 15/0;
  • check-scripts, the leak guard and check-pr-discipline are clean.

Review: pending independent review (re-review of 7070fa4)

https://claude.ai/code/session_01HXpQSjXefSNicGGqP4aiSC


Generated by Claude Code

…r for ui.no5xx (#818)

Mendix answers a refused save with HTTP 551 and a {"datavalidation":[...]} body. The opt-in
no5xx row counted every answer >= 500, so a negative-path step whose save was refused by design
read FAIL. The listener now reads a 551's body (helpers.isValidationFeedback, same shape as
#814's monkey classifier) and records a datavalidation answer as a note on the row; record5xx
awaits those body reads. A 500, and a 551 with any other or unreadable body, stay hits.

Golden input is the verbatim 551 captured from a live Mendix 11.8.0 runtime. The existing
back/no5xx fixture's wiring grep accepts `await record5xx();`.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HXpQSjXefSNicGGqP4aiSC
…ntainer (#812)

page.screenshot({ fullPage: true }) measures the document. Atlas_SideBar and Atlas_TopBar on
Mendix 11.8 scroll .mx-scrollcontainer-center inside a viewport-tall document, so the shot was
cut at the viewport. fullPageShot now finds the largest scroller spanning at least half the
viewport, grows the viewport by its hidden height (up to three times, capped at 16000 px, stopping
when growth no longer shrinks it), shoots, and restores the viewport. It returns a record with
scrollRoot, documentPx, contentPx, capturedPx and complete. Atlas_Default keeps plain fullPage.

The wait after a resize is page.waitForTimeout, not requestAnimationFrame: a saved MHTML page
never runs rAF, which hung the first version against the captured fixture.

A failed shot is no longer swallowed: the record carries `error`, and a transient
"Unable to capture screenshot" (1 in 10 fixture runs on a loaded 4-CPU box) is retried once.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HXpQSjXefSNicGGqP4aiSC
DOM_PROBE selected lists with [class*="widget-datagrid"], which matched a DataGrid2's root and
eight of its parts. Four parts hold no row, so live/empty-list-message (P1) reported
"9 lists on screen; 4 empty" on a 20-row grid. The selector now uses the .widget-datagrid class
token. The fixture lifts DOM_PROBE from page-audit.js and runs it over a captured 11.8 page.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HXpQSjXefSNicGGqP4aiSC
@MendixMau

Copy link
Copy Markdown
Owner Author

Review: MERGE-AFTER-FIX — fullPageShot grows the viewport for any fixed-height scroller of at least 50% of the viewport. The shot then gains thousands of blank pixels, and the record still says complete: true. This is a regression on Atlas_Default pages that master shot correctly.
Value: Fixes #818 as reported. The live 11.8 TasksNeg journey went from FAIL to PASS, with the 551 kept as a note (journey-runner.js:1088-1105). Fixes #812 for the SideBar and TopBar shells: live shots were 1244 and 1239 px, which matches the centre's scrollHeight. The .widget-datagrid token fix is right (page-audit.js:402). The older captured DOM design-audit-a11y/page-datagrid.html has root "mx-name-dataGrid21 widget-datagrid", and .mx-datagrid still covers the legacy grid. Nothing is overbuilt.
Regression risk: helpers.js:646-655. The scroll root is chosen by the largest extra among scrollers with clientHeight >= innerHeight/2, not by being the outermost or viewport-tied scroller. I injected a 480 px fixed-height .widget-datagrid-grid-body into the live 11.8 app. The Default page then shot at 4196 px (master: the 900 px document), mostly blank, root div.widget-datagrid-grid-body, complete: true. On SideBar it shot 2148 px against 900 px of content. A synthetic SideBar shell plus a 500 px grid chose the grid over the centre and shot 6400 px against 1800 px. Virtual-scroll or fixed-height grids and list views are real Mendix shapes. Restore is in finally on every path. I checked a screenshot error (EISDIR) and a page closed mid-wait: the viewport came back at 900 and the error was recorded. iPhone emulation was unchanged (maxTouchPoints, pointer:coarse, DPR, innerWidth). No exit codes, gates or parsed output change. #818: Mendix 560 errors are unaffected, and only a 551 with a datavalidation array is exempt. Responses still pending at page.off are missed exactly as on master. Bodies are awaited before the row is recorded, so row order is unchanged.
Correctness: complete = !m.best (:687) reads true when the scroller simply dropped below the 50% threshold after the grow, so the record's purpose fails exactly in this case. isValidationFeedback accepts {"datavalidation":[]} (minor). screenshotRetry retries non-transient errors too (minor). The golden input is captured, not hand-written. MHTML text is demo rows only ("Filler task NN", demo users) on 127.0.0.1, with no client data. CAPTURE.txt says 35 rows and the fixture header says 29, but the capture shows "1 to 20 of 30" (cosmetic). Fixtures: 551 7/0, fullpageshot 6/0 (5 reruns, no flakes), datagrid 3/0, back-no5xx 12/0.
Context cost: 65568 → 65568 (no routed files).
Time cost: about 0.3–0.5 s per inner-scroll shot (measured live), and 0.1–0.2 s for the DOM scan on document pages. The degenerate case also writes 4–16k-px PNGs. No conflicts with master. Repo size: 230 KB of gzipped MHTML (≈0.8 MB unpacked each), which is acceptable.
Exact fixes required:

  1. helpers.js innerScrollExtra/fullPageShot: only treat a scroller as the page root if it is viewport-tied. After the first setViewportSize, re-measure that same element. If its clientHeight did not grow by roughly the added height, restore the viewport and fall back to plain fullPage with complete: true. Prefer the outermost qualifying scroller over the one with the largest extra.
  2. Set complete from the chosen element's own residual scrollHeight - clientHeight, not from !m.best against a moved 50% threshold.
  3. Add a fixture case: the captured Default and SideBar MHTML with an injected fixed-height (≥450 px) scrolling grid body. Assert that the shot is no taller than the document or centre content (+60 px) and that the viewport is restored.

Generated by Claude Code

…root (review of #825)

The independent review found that the first version picked the scroller with the largest hidden
height among those at least half the viewport tall. A fixed-height DataGrid2 body (480 px) then
became the "root". The shot grew to 4196 px, mostly blank, on Atlas_Default, where master shot
the 900 px document. The record still said complete.

- The root is now the OUTERMOST qualifying scroller, tagged so the same element is re-measured.
- After each grow, the root's own clientHeight must have grown by at least half the added height.
  If it did not, the scroller is not tied to the viewport: restore, then take the plain fullPage.
  The record names it in skippedScroller.
- complete is the root's own leftover scroll (<= 4 px), not a re-pick against a moved threshold.

Fixture: the captured Default and SideBar MHTML, re-run with one appended CSS rule. The rule makes
.widget-datagrid-grid-body a fixed 480 or 800 px scroller. Results:
- previous branch helpers: 6/4;
- this commit: 10/0.
Live 11.8 run, injected 480 px grid body: Default 900 (was 4196), SideBar 900 (was 2148). Plain
SideBar and TopBar are still 1244 and 1239; Default is still 1278.
CAPTURE.txt and the fixture header now give the capture's real row count: 30 rows, 20 per page.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HXpQSjXefSNicGGqP4aiSC
@MendixMau

Copy link
Copy Markdown
Owner Author

Review: MERGE — re-review at 7070fa4. The fixed-height-scroller regression is fixed: on a live 11.8 app, a fixed-height grid body no longer grows the viewport, and the outermost scroller is now the shot root.
Value: #818: the live TasksNeg journey reads PASS 6 / FAIL 0, with the 551 datavalidation answer kept as a note. A 500, and a 551 with any other body, still FAIL (journey-runner.js:1088-1105). #812: live shots are SideBar 1244 px, TopBar 1239 px and Default 1278 px, each equal to the measured content. page-audit: the .widget-datagrid class token (page-audit.js:402) also matches the older captured DOM in design-audit-a11y/page-datagrid.html. Nothing is overbuilt.
Regression risk: helpers.js:646-725. The root is the outermost qualifying scroller, tagged and re-measured after the first grow. If that box did not grow by at least half the added height, the viewport is restored and plain fullPage is used. Live results with a grid body injected at 480 px: Default 900 px, SideBar 900 px, TopBar 900 px, each root: document with skippedScroller named. That is master's behaviour. My first review saw 4196 and 2148 px here. With the grid at 800 px, SideBar and TopBar chose the centre and shot 977 / 972 px, equal to the content. The synthetic shell with a 500 px grid chose div.center (1800 px) on desktop and on iPhone emulation. Emulation state was the same before and after the shot: width, height, touch points, coarse pointer, pixel ratio. On the error path (EISDIR) the viewport was restored to 900, the error was recorded and no data-mxtk-shot-root tag was left. No gates, exit codes or parsed outputs change. Both layouts are unaffected (no path code).
Correctness: No blocking bugs. Nit: on the plain path, if the page closes after a successful shot, the record reads complete: true with capturedPx: null (:722). Earlier nits stay optional: {"datavalidation":[]} counts as feedback, and the retry also covers errors that are not transient. The golden input is captured. The new g480/g800 cases inject CSS into the captured MHTML; they are not hand-written DOM. Fixtures, each run 5×: fullpageshot 10/0, 551-validation 7/0, datagrid-count 3/0; back-no5xx 12/0. No flakes. The MHTML holds only demo data on 127.0.0.1.
Context cost: 65568 → 65568 words (no routed files touched).
Time cost: Live: 0.3–0.8 s per shot when a scroller qualifies (one grow plus restore, two 250 ms settles), about 0.1 s on document-scroll pages. Gzipped fixtures add 230 KB. No conflicts with master.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants