From e488e71f1db1207e6cd7b414fd872d5fa3352993 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 20 Nov 2026 19:10:09 +0000 Subject: [PATCH 1/4] fix(e2e harness): a fresh scaffold's project.config.js ships empty, not another app's values The installed tests/e2e/project.config.js is the template, and the template carried one sample app's CONFIGURED values: NAV, PROBE_PAGE, two users in ROLES, a 3-act WALKTHROUGH, 14 PAGE_WIREFRAME rows, project widget names in READY_SELECTORS. On a fresh scaffold of a real Mendix 11.8 app, monkey reached 0 of 8 targets and full-app-walkthrough died at its first seed with an OQL error for an entity the app never had. Every CONFIGURED block now ships empty with a FILL ME marker and its shape in a comment. `node project.config.js` marks each empty block and names the instrument it stops. The walkthrough's NOT CONFIGURED message names the blocks to fill; page-audit advice prints a "no exemplar set" placeholder instead of a page; install-tests.sh lists the blocks. Fixture: tests/wave2/test-e2e-config-neutral.sh (master 1/16, branch 16/16). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HXpQSjXefSNicGGqP4aiSC --- CHANGELOG.md | 1 + bin/install-tests.sh | 8 +- project-tests/e2e/full-app-walkthrough.js | 7 +- project-tests/e2e/page-audit-rules.js | 4 +- project-tests/e2e/project.config.template.js | 338 +++++-------------- tests/wave2/test-e2e-config-neutral.sh | 94 ++++++ 6 files changed, 190 insertions(+), 262 deletions(-) create mode 100755 tests/wave2/test-e2e-config-neutral.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index e9e148df..00a6ecd7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m `docs/RELEASE-PROCESS.md` §8. ## Unreleased +- fix(e2e harness, install-tests): **a fresh scaffold's `tests/e2e/project.config.js` no longer carries another app's menu, users and process**: the installed file is the template, and the template shipped one sample app's CONFIGURED values (8 menu captions, a probe page, two users, a 3-act walkthrough, 14 wireframe rows, project widget names in `READY_SELECTORS`), so a fresh scaffold's monkey reached 0 of 8 targets and `full-app-walkthrough.js` died at its first seed with an OQL error for an entity the app never had. Every CONFIGURED block now ships empty with a `FILL ME` marker and the shape in a comment; `node tests/e2e/project.config.js` marks each empty block and names the instrument it stops; the walkthrough's NOT CONFIGURED message points at the blocks to fill; page-audit advice says `` instead of naming a page; `install-tests.sh` lists the blocks. A project's own `project.config.js` is never rewritten. Fixture `tests/wave2/test-e2e-config-neutral.sh` (master 1/16, branch 16/16) — field run on a blank Mendix 11.8 app under `mxcli run --local` (2026-11-20) - new(doctor, bug ledger): **a guard for mxcli's silent MxBuild fallback**: when the project's Mendix version is not in `~/.mxcli/mxbuild/` (and no Studio Pro of that version exists), `mxcli docker build` / `docker check` fall back to the newest cached MxBuild and print its path but no version warning (`run --local` is exact-or-refuse); `bin/doctor.sh` now warns, naming the model version, the cached ones and `./mxcli setup mxbuild -p .mpr` (`mxtk_mxbuild_cache_gap`, warning only, never changes a gate). Ledger entry `BUG-DRAFT-mxbuild-silent-version-fallback`; upstream issue not filed. — a four-app Mendix build (field run), independent review - fix(gate-check): **a fresh clone no longer FAILs Stage 3 as "stale render"**: `blueprint.html` / `process-map.html` freshness used `[ render -ot source ]`, a nanosecond mtime compare, and a checkout stamps files in directory order (`process-map.html` lands before `process-model.json`), so an unedited project read stale on every clone (reproduced: 200-wireframe checkout, 10 of 10 clones). New `bin/lib/render-fresh.sh` keeps `-ot` as the first test (a pass never flips, no git call) and, only where `-ot` would FAIL, asks git once per render (one `log`, one `status`) whether the source was committed after the render's commit; measured 0.19 s for 200 sources all `-ot`-stale on a 300-commit history (a per-source git version cost 12.9 s). Limit: a depth-1 clone cannot see a genuinely stale render. Field run: a four-app Mendix build checked out on a second machine. — a four-app Mendix build (field run), independent review - fix(test-stack-up, e2e-harness-base): **the app port follows the app's configured port, and a multi-app start is no longer pinned to 8080**: with no `APP_PORTS`, `test-stack-up.sh` puts `RUN_LOCAL_APP_PORT`, then `APP_PORT` from the `.docker/.env` beside the `.mpr` (`apps//.docker/.env`; the project-root one as fallback) first, ahead of the old fixed `8080 8081 8084` tail. A fresh `run --local` start for a model under `apps/` with no handshake and no configured port used to fall back to 8080 for every app (`--app-port 8080`); it now exits 4 naming `RUN_LOCAL_APP_PORT`. `e2e-harness-base.md` states that a PAD/Docker runtime listens on 8080 *inside* the container only. Fixtures T28–T32 added. Field run: a four-app Mendix build on 8081–8084 whose fresh start failed with every app pinned to 8080. — a four-app Mendix build (field run), independent review diff --git a/bin/install-tests.sh b/bin/install-tests.sh index 15c8f709..b7bf9c87 100755 --- a/bin/install-tests.sh +++ b/bin/install-tests.sh @@ -66,10 +66,10 @@ echo echo "engine -> $DST ($copied copied, $skipped kept)" echo echo "Next, in this order:" -echo " 1. node $DST/project.config.js # print what it derived; check id and .mpr" -echo " 2. edit NAV / TARGETS / PAGE_WIREFRAME / exemplars / walkthroughs in project.config.js" +echo " 1. node $DST/project.config.js # what it derived (check id and .mpr), and each block still empty" +echo " 2. fill the FILL ME blocks in project.config.js: NAV, ROLES, WALKTHROUGH, PAGE_WIREFRAME, exemplars" echo " 3. node $DST/report-normalize.js --selftest # must print 'all ok'" echo " 4. node $DST/journey-runner.selftest.js" echo -echo "Step 2 is not optional. Unedited, the audits walk another app's page names and report" -echo "fault on every page — which looks like a broken harness, not an unedited config." +echo "Step 2 is not optional. The blocks ship empty: until they are filled, monkey.js and" +echo "full-app-walkthrough.js stop with NOT CONFIGURED (exit 2) instead of measuring anything." diff --git a/project-tests/e2e/full-app-walkthrough.js b/project-tests/e2e/full-app-walkthrough.js index a576b147..7ee388a4 100644 --- a/project-tests/e2e/full-app-walkthrough.js +++ b/project-tests/e2e/full-app-walkthrough.js @@ -51,8 +51,8 @@ // ENGINE / CONFIG LINE // Nothing in this file knows a module, an entity, a page, a widget, a menu // caption, a user or a URL. All of that is `WALKTHROUGH` and `ROLES` in -// project.config.js — see project.config.template.js, which ships a worked -// example and the mxcli command that produces the right answer for each field. +// project.config.js — see project.config.template.js, which ships the shape of +// each block (empty) and the mxcli command that produces the right answer for each field. // ============================================================================ const H = require(__dirname + '/helpers.js'); @@ -75,7 +75,8 @@ if (!WT || !Array.isArray(WT.acts) || WT.acts.length === 0) { '\n NOT CONFIGURED — project.config.js exports no `walkthrough` with acts.\n' + ' This instrument walks ONE business process across modules and roles; the process\n' + ' is a statement about your application, so it lives in project.config.js.\n' + - ' Copy the WALKTHROUGH / ROLES blocks from project.config.template.js and fill them in.\n' + + ' Fill the WALKTHROUGH and ROLES blocks in tests/e2e/project.config.js (the comment above\n' + + ' WALKTHROUGH shows the shape; `node tests/e2e/project.config.js` lists what is still empty).\n' + ' Writing an empty findings.json here would report a clean walkthrough that never ran.\n'); process.exit(2); } diff --git a/project-tests/e2e/page-audit-rules.js b/project-tests/e2e/page-audit-rules.js index d97861b8..5a8553df 100644 --- a/project-tests/e2e/page-audit-rules.js +++ b/project-tests/e2e/page-audit-rules.js @@ -37,7 +37,9 @@ const PROJ = require('./project.config'); const MPR = PROJ.mprName; -const EX = PROJ.exemplars; +// A fresh scaffold leaves them null: the advice then says what to set, never "null's ...". +const EX = Object.fromEntries(['listPage', 'overviewPage', 'detailPage'].map((k) => + [k, (PROJ.exemplars || {})[k] || ``])); // ── MDL parsing ────────────────────────────────────────────────────────────── // A widget declaration is ` ( props ) { body }`, where props may span diff --git a/project-tests/e2e/project.config.template.js b/project-tests/e2e/project.config.template.js index 0bb35334..b59d2942 100644 --- a/project-tests/e2e/project.config.template.js +++ b/project-tests/e2e/project.config.template.js @@ -40,7 +40,7 @@ const path = require('path'); // exactly two levels below the root", which is a project layout assumption like // any other. The two-levels-up answer is kept only as the last resort, so a // project with no .mpr on disk yet still gets a usable root instead of a crash. -// F-042 (MarkUseCase, 2026-08-28): a two-tree layout — tests/e2e at the repo root, the +// F-042 (field run, 2026-08-28): a two-tree layout — tests/e2e at the repo root, the // model under a sibling app/ — defeats a pure ancestor walk, because app/ is never an // ancestor of tests/e2e. The walk therefore probes each ancestor's app/ child too, and // two explicit overrides win over any walking: PROJECT_ROOT names the root outright, @@ -130,8 +130,8 @@ function mprPath() { // NAME_OVERRIDE to null in a project whose directory name is correct. // // Two-tree layout: ROOT is app/, and "app" names nothing — `id` came out as `app`, so every -// trace assertion queried a service called `app` (card-disbursement requirements-driven -// build, 2026-09-25). There the model's own name is the identity, and it is also what +// trace assertion queried a service called `app` (a requirements-driven field build, +// 2026-09-25). There the model's own name is the identity, and it is also what // `mxcli run --local --trace` defaults OTEL_SERVICE_NAME to. Read inside a try: this file // must not throw on require, and a missing .mpr still throws where `mpr` is accessed. const NAME_OVERRIDE = null; // TEMPLATE: set only if your checkout dir != the app name @@ -145,7 +145,13 @@ const DISPLAY_NAME = process.env.PROJECT_DISPLAY_NAME || ID.replace(/[-_]+/g, ' // ============================================================================ // CONFIGURED — everything below this line is a statement about THIS model. -// A new project MUST edit every block below. None of it can be derived. +// A new project MUST fill every block below. None of it can be derived. +// +// THIS FILE SHIPS EMPTY ON PURPOSE. It used to ship one sample app's menu, pages, +// users and process, which a fresh scaffold then ran as if they were its own: monkey +// reached "0 of 8 targets" and the walkthrough died on an OQL error for an entity the +// app never had. An empty block makes each instrument stop with "NOT CONFIGURED: fill +// " instead. `node tests/e2e/project.config.js` lists what is still empty. // ============================================================================ // ── CONFIGURED: credentials ────────────────────────────────────────────────── @@ -191,16 +197,12 @@ function credentials() { const DEFAULT_PORT = '8081'; // ── CONFIGURED: post-login readiness selectors ─────────────────────────────── -// Proof that the SPA finished routing into the app shell. The first two are -// generic to every Mendix app and are the engine's own default; the rest are -// THIS project's widget names, needed because the phone layout has no nav tree -// (it auto-routes straight to the FieldScan tiles). +// Proof that the SPA finished routing into the app shell. The two below are +// generic to every Mendix app. Add THIS project's widget names only when a +// layout has neither (a phone layout with no nav tree, say). // Read the real names with: ./mxcli -p -c "DESCRIBE PAGE ." const READY_SELECTORS = [ '.mx-navigationtree', - '.mx-name-btnSendMessage', - '.mx-name-tileMove', - '.mx-name-bM', '.mx-page', ]; @@ -220,123 +222,19 @@ const DS_CSS = designPath('design', 'ds.css'); // ── CONFIGURED: navigation captions ────────────────────────────────────────── // navClick() clicks the real in the Mendix nav tree, so these -// must match the menu captions EXACTLY. +// must match the menu captions EXACTLY. One entry per menu item monkey.js fuzzes: +// key: ['Item'] for a top-level item, key: ['Group', 'Item'] for a nested one. // Produce the right answer with: ./mxcli -p -c "DESCRIBE NAVIGATION" +// FILL ME — empty, monkey.js exits 2 "NOT CONFIGURED: NAV" without a browser. const NAV = { - dashboard: ['Dashboard'], - equipment: ['Assets & Barcodes', 'Equipment'], - locations: ['Assets & Barcodes', 'Locations'], - fieldScan: ['Assets & Barcodes', 'Field Scan'], - barcodeLabel: ['Assets & Barcodes', 'Barcode Labels'], - aiAssistant: ['AI', 'AI Assistant'], - aiAdmin: ['AI', 'AI Admin'], - designSystem: ['Configuration', 'Design System'], }; -// ── CONFIGURED: full-app walkthrough steps ─────────────────────────────────── -// One entry per stop on the golden path. Every `.mx-name-*` selector and every -// `checks` entry is a widget name; every `oql` statement names domain entities. -// None of it is guessable. -// Widget names: ./mxcli -p -c "DESCRIBE PAGE ." -// Entity names: ./mxcli -p -c "SHOW ENTITIES IN " +// ── CONFIGURED: page stops (legacy) ────────────────────────────────────────── +// A map of named page-stop lists, read only through config.js resolveTarget(). +// No shipped instrument calls it; kept so a project's own script that does keeps +// working. Shape: { '': { label, ready, pages: [{ key, label, nav, ready, +// checks: [widget names], oql: [{ label, sql }] }] } }. const TARGETS = { - 'full-app': { - label: `${ID} — Full App Walkthrough`, - ready: '.mx-page', - pages: [ - { - key: 'dashboard', label: 'Dashboard', nav: NAV.dashboard, - ready: '.mx-name-dvKPI', - checks: ['dvKPI', 'dtTotal', 'dtActive', 'dtMfr', 'dtUnclass', 'btnNewItem', 'btnItemMgmt', - 'cChartEquip', 'cChartMfr', 'cChartTrend', 'lvActivityFeed'], - oql: [ - { label: 'Total equipment', sql: 'SELECT COUNT(*) AS n FROM Equipment.Item' }, - { label: 'Active equipment', sql: 'SELECT COUNT(*) AS n FROM Equipment.Item WHERE Active = true' }, - { label: 'Total locations', sql: 'SELECT COUNT(*) AS n FROM Location.Location' }, - { label: 'Placements', sql: 'SELECT COUNT(*) AS n FROM FieldScan.ItemPlacement' }, - ], - }, - { - key: 'equipment-list', label: 'Equipment — Overview', nav: NAV.equipment, - ready: '.mx-name-dgItems', - checks: ['dgItems', 'btnRegister', 'btnExportItems', 'btnResetItems', 'btnEditItem1'], - oql: [{ label: 'Equipment rows', sql: 'SELECT COUNT(*) AS n FROM Equipment.Item' }], - }, - { - key: 'equipment-detail', label: 'Equipment — Detail (read + cancel)', nav: null, - ready: '.mx-name-dvItem', - checks: ['dvItem', 'dynItemCode', 'dynModelName', 'txtDesc1', 'btnCancel'], - oql: [], - }, - { - key: 'equipment-create', label: 'Equipment — Create + Cancel (no DB write)', nav: NAV.equipment, - ready: '.mx-name-dgItems', - checks: [], - oql: [], - }, - { - key: 'locations', label: 'Locations — Overview', nav: NAV.locations, - ready: '.mx-name-dgLocations', - checks: ['dgLocations', 'btnRegisterLoc', 'btnExportLoc', 'btnResetLoc', 'btnDetailLoc1'], - oql: [{ label: 'Location rows', sql: 'SELECT COUNT(*) AS n FROM Location.Location' }], - }, - { - key: 'location-detail', label: 'Locations — Detail (edit remark + cancel)', nav: null, - ready: '.mx-name-dvLocation', - checks: ['dvLocation', 'dynLocId', 'dynLocName', 'txtRemark', 'btnBack'], - oql: [], - }, - { - key: 'barcode-label', label: 'Barcode Labels', nav: NAV.barcodeLabel, - ready: '.mx-name-lvItems', - checks: ['lvItems', 'btnPrint', 'ctnLabelPreview', 'txtItemInfo'], - oql: [{ label: 'Print-enabled equipment', - sql: 'SELECT COUNT(*) AS n FROM Equipment.Item WHERE PrintEnabled = true' }], - }, - { - key: 'fieldscan-home', label: 'Field Scan — Home', nav: NAV.fieldScan, - ready: '.mx-name-bR', - checks: ['bR', 'bM', 'bC', 'tileReceive', 'tileMove', 'tileCount'], - oql: [], - }, - { - key: 'fieldscan-receive', label: 'Field Scan — Receive (resolve a real barcode)', nav: null, - ready: '.mx-name-dvScanCtx', - checks: ['dvScanCtx', 'tbScannedCode', 'btnResolve', 'tbQuantity', 'btnReceive', 'lvHistory'], - oql: [{ label: 'ItemPlacement rows', sql: 'SELECT COUNT(*) AS n FROM FieldScan.ItemPlacement' }], - }, - { - key: 'fieldscan-move', label: 'Field Scan — Move (resolve item + location)', nav: NAV.fieldScan, - ready: '.mx-name-dvScanCtx', - checks: ['dvScanCtx', 'tbScannedCode', 'btnResolve', 'btnMove'], - oql: [{ label: 'MovementLog rows', sql: 'SELECT COUNT(*) AS n FROM FieldScan.MovementLog' }], - }, - { - key: 'fieldscan-count', label: 'Field Scan — Stock Count', nav: NAV.fieldScan, - ready: '.mx-name-dvCtx', - checks: ['dvCtx', 'tbLocCode', 'btnResolveLoc', 'btnStart', 'tbAssetCode', 'btnScanAsset'], - oql: [{ label: 'CountSession rows', sql: 'SELECT COUNT(*) AS n FROM FieldScan.CountSession' }], - }, - { - key: 'ai-chat', label: 'AI Assistant — Chat', nav: NAV.aiAssistant, - ready: '.mx-name-dvChat', - checks: ['heading', 'cHead', 'dvChat', 'cCopilot'], - oql: [], - }, - { - key: 'ai-admin', label: 'AI Assistant — Admin (buttons present, not clicked)', nav: NAV.aiAdmin, - ready: '.mx-name-btnLoadItemKB', - checks: ['btnLoadItemKB', 'btnLoadLocKB', 'cItemKB', 'cLocKB'], - oql: [], - }, - { - key: 'design-system', label: 'Design System — Component Gallery', nav: NAV.designSystem, - ready: '.mx-page', - checks: [], - oql: [], - }, - ], - }, }; // ── CONFIGURED: the page -> wireframe map ──────────────────────────────────── @@ -345,34 +243,16 @@ const TARGETS = { // drift and no way to notice. // // Explicit and hand-verified against each wireframe's own , NOT -// fuzzy-matched. A token heuristic once paired AIAssistant.AIAdmin_Home with -// home.html at 0.67 ("home" is in both names) and then reported 31 findings from -// a wireframe that has nothing to do with that page. A wrong mapping does not -// produce a weak finding, it produces a confident wrong one — so an unmapped -// page is `skipped` in design-audit (covered by rung 7) and graded as a coverage -// gap in page-audit (requirement R4). +// fuzzy-matched. A token heuristic once paired an admin page with home.html at +// 0.67 ("home" is in both names) and then reported 31 findings from a wireframe +// that has nothing to do with that page. A wrong mapping does not produce a weak +// finding, it produces a confident wrong one — so an unmapped page is `skipped` +// in design-audit (covered by rung 7) and graded as a coverage gap in page-audit +// (requirement R4). // Produce the candidate list with: ls design/wireframes // Confirm each pairing by reading that wireframe's <title>. +// FILL ME — shape: '<Module>.<Page>': '<file>.html', // "<the wireframe's <title>>" const PAGE_WIREFRAME = { - 'Analytics.Dashboard_Home': 'home.html', // "Home Dashboard" - 'Equipment.Item_Overview': 'item-management.html', // "Equipment" - 'Equipment.Item_Detail': 'item-edit.html', // "Item Edit modal" - 'Equipment.BarcodeLabel_Overview': 'barcode-labels.html', // "Barcode Label Renderer" - 'Location.Location_Overview': 'location-management.html', - 'Location.Location_Detail': 'location-detail.html', - 'FieldScan.MobileScan_Count': 'mobile-scan.html', // "Mobile Field-Scan" - 'FieldScan.Receive_Scan': 'movement-scan.html', // "Movement Scan (Receive · Move)" - 'FieldScan.Move_Scan': 'movement-scan.html', - 'RoutingManagement.Route_List': 'route-list.html', - 'RoutingManagement.Route_Detail': 'route-detail.html', - 'OperationManagement.Operation_List': 'operation-list.html', - 'ProcessVariant.ProcessVariant_List': 'processvariant-list.html', - 'ReworkDeviationRouting.ReworkRoute_List': 'reworkroute-list.html', - // Deliberately unmapped — no wireframe was ever drawn for these five. Not a - // mapping bug, a coverage gap, and reported as a finding on each page rather - // than silently omitted: - // AIAssistant.Chat_Home, AIAssistant.AIAdmin_Home, StyleGallery.Gallery_Home, - // Equipment.Item_Overview_Lab, FieldScan.FieldScan_Home }; // ── CONFIGURED: the live positive-control probe page ───────────────────────── @@ -384,7 +264,7 @@ const PAGE_WIREFRAME = { // choice because it may happen to be fully readable. // Find a candidate with: ./mxcli -p <mpr> describe --format elk page <Module>.<Page> // and compare the tree node count against the page's own mdlSource. -const PROBE_PAGE = process.env.PROBE_PAGE || 'RoutingManagement.Route_List'; +const PROBE_PAGE = process.env.PROBE_PAGE || null; // FILL ME, or leave null for the page-scope fallback function probePage(scopeFile) { if (PROBE_PAGE) return PROBE_PAGE; @@ -399,12 +279,13 @@ function probePage(scopeFile) { // ── CONFIGURED: exemplar pages quoted in page-audit advice strings ─────────── // page-audit-rules.js's rule LOGIC is generic; its `exemplar` and `remediate` // text points a reader at a page in THIS project that already does the thing -// right. Repoint these at the equivalent pages of a new project — an exemplar -// naming a page that does not exist is worse than no exemplar. +// right. Name the equivalent pages of this project — an exemplar naming a page +// that does not exist is worse than no exemplar, so null prints "no exemplar set". +// FILL ME — '<Module>.<Page>' each, or leave null. const EXEMPLARS = { - listPage: 'RoutingManagement.Route_List', - overviewPage: 'Equipment.Item_Overview', - detailPage: 'Equipment.Item_Detail', + listPage: null, + overviewPage: null, + detailPage: null, }; // ── CONFIGURED: the roles the business process passes through ──────────────── @@ -413,19 +294,14 @@ const EXEMPLARS = { // exactly one business role: an account carrying two of them cannot fail a role // boundary and every negative check below it is vacuous. // Find the roles: ./mxcli -p <mpr> -c "SHOW SECURITY ROLES" -// Find the accounts: the app's own Administration → Accounts page +// Find the accounts: the app's own Administration → Accounts page, or the +// demo users (`create or modify demo user` sets a known password) // Env overrides exist so CI can inject credentials without editing this file. +// FILL ME — shape: +// requester: { label: '<role as the business says it>', +// user: process.env.ROLE_REQUESTER_USER || '<user>', +// pass: process.env.ROLE_REQUESTER_PASS || '<password>' }, const ROLES = { - requester: { - label: 'Manufacturing Engineer', - user: process.env.ROLE_REQUESTER_USER || 'erika.engineer', - pass: process.env.ROLE_REQUESTER_PASS || 'Engineer2025!', - }, - approver: { - label: 'Production Planner', - user: process.env.ROLE_APPROVER_USER || 'paul.planner', - pass: process.env.ROLE_APPROVER_PASS || 'Planner2025!', - }, }; // ── CONFIGURED: the one cross-module, cross-role business process ──────────── @@ -457,94 +333,44 @@ const ROLES = { // Any step may carry `label` (what the report calls it) and `fatal: true` (nothing // after this is worth attempting — later steps are then reported NOT REACHED // rather than left silently absent). `{{name}}` interpolates a seed or a capture. +// FILL ME — `acts` is empty, so full-app-walkthrough.js exits 2 "NOT CONFIGURED". +// The shape of one process (illustrative names; replace every one of them): +// +// seeds: [{ name: 'orderNo', sql: "SELECT OrderNo AS o FROM Sales.Order WHERE Status = 'Open' LIMIT 1" }], +// acts: [ +// { key: 'raise', role: 'requester', module: 'Sales', label: 'raise an order', +// // `checks` are read on the act's landing page: after its handoffAssert, and +// // after its LEADING `nav` steps — so a first act may simply start with a nav. +// checks: ['dgOrders'], +// steps: [ +// { do: 'nav', group: 'Sales', item: 'Orders', ready: 'dgOrders', fatal: true }, +// { do: 'baseline', name: 'orders', sql: 'SELECT COUNT(*) AS n FROM Sales.Order' }, +// { do: 'click', widget: 'btnNew', fatal: true }, +// { do: 'fill', widget: 'txtReference', value: 'E2E walkthrough' }, +// { do: 'click', widget: 'btnSave', settle: 3000 }, +// { do: 'oql', label: 'the order was written', deltaFrom: 'orders', by: 1, +// sql: 'SELECT COUNT(*) AS n FROM Sales.Order', fatal: true }, +// { do: 'expectAbsent', widget: 'btnApprove', label: 'the requester cannot approve' }, +// ] }, +// { key: 'approve', role: 'approver', module: 'Approvals', label: 'approve it', +// handoffAssert: [ // runs right after the sign-in: can role B SEE role A's work? +// { do: 'nav', group: 'Approvals', item: 'My Queue', ready: 'dgQueue', fatal: true }, +// { do: 'expectText', widget: 'dgQueue', text: '{{orderNo}}' } ], +// checks: ['dgQueue', 'btnApprove'], +// steps: [ { do: 'click', widget: 'btnApprove', settle: 3000 } ] }, +// ], +// cleanup: { role: 'approver', steps: [ /* oql checks that put the data back */ ] }, const WALKTHROUGH = { label: `${ID} — cross-role business process`, // One sentence, in the language the business uses. It is printed in the report. - process: 'Engineer raises a routing change → planner approves it → the change is visible on the shop floor', - - // Live values, never literals: a walk pinned to 'RT-PCBA-008' reports "the + process: '', + // Live values, never literals: a walk pinned to one record number reports "the // feature is broken" the first time somebody reseeds the database. - seeds: [ - { name: 'routeCode', sql: "SELECT RoutingCode AS c FROM RoutingManagement.Route WHERE LifecycleState = 'Released' LIMIT 1" }, - ], - - acts: [ - { - key: 'raise', role: 'requester', module: 'RoutingManagement', - label: 'raise a change request against a released route', - checks: ['dgRoutes', 'btnSearch'], - steps: [ - { do: 'nav', group: 'Routing', item: 'Route List', ready: 'dgRoutes', label: 'Route List opens for the engineer', fatal: true }, - { do: 'fill', widget: 'txtRoutingCode', value: '{{routeCode}}', debounceMs: 2500, label: 'search for the seeded route' }, - { do: 'click', widget: 'btnSearch', label: 'run the search' }, - { do: 'expectRows', widget: 'dgRoutes', min: 1, label: 'the route is found' }, - { do: 'baseline', name: 'requests', sql: 'SELECT COUNT(*) AS n FROM RoutingManagement.ChangeRequest' }, - { do: 'click', widget: 'btnRequestChange', label: 'open the change request form', fatal: true }, - { do: 'fill', widget: 'txtReason', value: 'E2E cross-role walkthrough', label: 'state a reason' }, - { do: 'click', widget: 'btnSubmit', settle: 3000, label: 'submit the request' }, - { do: 'oql', label: 'the request was written', deltaFrom: 'requests', by: 1, - sql: 'SELECT COUNT(*) AS n FROM RoutingManagement.ChangeRequest', fatal: true }, - { do: 'capture', name: 'requestNo', - sql: 'SELECT RequestNo AS r FROM RoutingManagement.ChangeRequest ORDER BY CreatedDate DESC LIMIT 1' }, - { do: 'oql', label: 'it is queued for approval, not auto-approved', expect: 'Submitted', - sql: "SELECT Status AS s FROM RoutingManagement.ChangeRequest WHERE RequestNo = '{{requestNo}}'" }, - { do: 'expectAbsent', widget: 'btnApprove', label: 'the engineer cannot approve their own request' }, - { do: 'shot', name: 'act-raise' }, - ], - }, - { - key: 'approve', role: 'approver', module: 'ApprovalManagement', - label: 'approve the request the engineer raised', - // handoffAssert runs immediately after the sign-in, on the HANDOFF page of the - // report, and it is the point of this whole instrument: proof that the second - // role can SEE the first role's work. Nothing in a per-module journey can make - // that claim, because no per-module journey ever changes identity. - handoffAssert: [ - { do: 'nav', group: 'Approvals', item: 'My Queue', ready: 'dgApprovals', label: 'the planner has an approval queue', fatal: true }, - { do: 'expectText', widget: 'dgApprovals', text: '{{requestNo}}', - label: "the engineer's request appears in the planner's queue" }, - ], - checks: ['dgApprovals', 'btnApprove'], - steps: [ - { do: 'click', widget: 'btnOpenRequest', label: 'open the request', fatal: true }, - { do: 'expectText', widget: 'dvRequest', text: '{{routeCode}}', label: 'it is the route the engineer picked' }, - { do: 'click', widget: 'btnApprove', settle: 3000, label: 'approve it' }, - { do: 'oql', label: 'approval is recorded against the request', expect: 'Approved', - sql: "SELECT Status AS s FROM RoutingManagement.ChangeRequest WHERE RequestNo = '{{requestNo}}'" }, - { do: 'oql', label: 'the approver on the record is the planner, not the engineer', - expect: ROLES.approver.user, - sql: "SELECT ApprovedBy AS u FROM RoutingManagement.ChangeRequest WHERE RequestNo = '{{requestNo}}'" }, - { do: 'shot', name: 'act-approve' }, - ], - }, - { - key: 'consume', role: 'requester', module: 'ProcessVariant', - label: 'the engineer sees the approved change downstream', - handoffAssert: [ - { do: 'nav', group: 'Routing', item: 'Route List', ready: 'dgRoutes', label: 'the engineer is back in Route List' }, - { do: 'expectText', widget: 'dgRoutes', text: '{{routeCode}}', label: "the planner's approval is visible to the engineer" }, - ], - checks: ['dgRoutes'], - steps: [ - { do: 'expectMatchesDb', widget: 'dtOpenRequests', - sql: 'SELECT COUNT(*) AS n FROM RoutingManagement.ChangeRequest WHERE Status = \'Submitted\'', - label: 'the open-requests tile agrees with the database' }, - { do: 'shot', name: 'act-consume' }, - ], - }, - ], - + seeds: [], + acts: [], // Put the data back. A walk that writes and does not restore makes the second - // run measure the first run's leftovers — and the cleanup itself is reported, - // because a cleanup that silently failed is exactly what the next reader needs - // to know before believing anything. - cleanup: { - role: 'approver', - steps: [ - { do: 'oql', label: 'no walkthrough request left in Submitted state', expect: '0', - sql: "SELECT COUNT(*) AS n FROM RoutingManagement.ChangeRequest WHERE Reason = 'E2E cross-role walkthrough' AND Status = 'Submitted'" }, - ], - }, + // run measure the first run's leftovers — and the cleanup itself is reported. + cleanup: null, }; // ── CONFIGURED: the weak walkthrough instruments ───────────────────────────── @@ -560,7 +386,7 @@ const WALKTHROUGH = { // run, or ran without writing", which reads as a broken app and sends the reader // to debug configuration that was never wrong. That is not hypothetical: this // template declared `full-app-walkthrough` for months while no such file had ever -// existed in the toolkit, and PROJECT-A's 6-module pass reported exactly that +// existed in the toolkit, and a 6-module field pass reported exactly that // (2026-08-20, process/improvement-plan-e2e-reporting.md Finding 5). const WALKTHROUGHS = [ { instrument: 'full-app-walkthrough', input: 'findings', script: 'full-app-walkthrough.js', tags: [] }, @@ -627,17 +453,21 @@ if (require.main === module) { // "undefined", which reads as "not configured" when it only means "not an array". const n = (v) => (Array.isArray(v) ? v.length : v && typeof v === 'object' ? Object.keys(v).length : 0); const e = module.exports; - console.log(` NAV ${n(e.NAV)}`); + // An empty block says which instrument it stops, so the first run is a to-do list, not a guess. + const fill = (k, why) => (n(e[k]) ? '' : ` ← FILL ME: ${why}`); + const setEx = Object.values(e.exemplars || {}).filter(Boolean).length; + console.log(` NAV ${n(e.NAV)}${fill('NAV', 'monkey.js exits 2 NOT CONFIGURED')}`); console.log(` TARGETS ${n(e.TARGETS)}`); - console.log(` PAGE_WIREFRAME ${n(e.PAGE_WIREFRAME)}`); - console.log(` exemplars ${n(e.exemplars)}`); + console.log(` PAGE_WIREFRAME ${n(e.PAGE_WIREFRAME)}${fill('PAGE_WIREFRAME', 'every page reads as unmapped')}`); + console.log(` exemplars ${setEx} of ${n(e.exemplars)} set`); console.log(` walkthroughs ${n(e.walkthroughs)}`); - console.log(` roles ${n(e.roles)}`); + console.log(` roles ${n(e.roles)}${fill('roles', 'full-app-walkthrough.js cannot sign in as anyone')}`); // Roles and modules, not act count: a walk with ten acts and one role is the // single-persona blind spot this instrument exists to close, and it should be // visible as such the moment somebody prints this file. const acts = (e.walkthrough && e.walkthrough.acts) || []; const uniq = (k) => [...new Set(acts.map((a) => a[k]).filter(Boolean))]; console.log(` walkthrough ${acts.length} act(s), ${uniq('role').length} role(s), ${uniq('module').length} module(s)` - + `${uniq('role').length < 2 ? ' ← SINGLE PERSONA: this is a module journey, not a process' : ''}`); + + `${!acts.length ? ' ← FILL ME: full-app-walkthrough.js exits 2 NOT CONFIGURED' + : uniq('role').length < 2 ? ' ← SINGLE PERSONA: this is a module journey, not a process' : ''}`); } diff --git a/tests/wave2/test-e2e-config-neutral.sh b/tests/wave2/test-e2e-config-neutral.sh new file mode 100755 index 00000000..b189b2d5 --- /dev/null +++ b/tests/wave2/test-e2e-config-neutral.sh @@ -0,0 +1,94 @@ +#!/usr/bin/env bash +# Usage: bash test-e2e-config-neutral.sh [path-to-project.config.template.js] +# +# Fixture: a fresh scaffold's tests/e2e/project.config.js carries no other app's values. +# +# The defect (field run 2026-11-20, a blank Mendix 11.8 app scaffolded with init-project.sh and +# run for real under `mxcli run --local`): the installed project.config.js is byte-identical to +# project.config.template.js, and the template carried one sample app's CONFIGURED values — a +# menu, a probe page, two users, a walkthrough, 14 wireframe rows. Out of the box monkey reached +# "0 of 8 target(s)" and full-app-walkthrough died at its first seed with an OQL error for an +# entity the app never had. The fix ships every CONFIGURED block empty, so each instrument stops +# with NOT CONFIGURED and `node project.config.js` lists what is still to fill. +# +# Golden input: the template itself, installed by the real bin/install-tests.sh into a temp +# project, exactly as init-project.sh does it. + +set -uo pipefail + +TOOLKIT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +SUT="${1:-$TOOLKIT/project-tests/e2e/project.config.template.js}" +case "$SUT" in /*) ;; *) SUT="$PWD/$SUT" ;; esac +[ -f "$SUT" ] || { echo "SKIP: subject not found at $SUT"; exit 0; } +command -v node >/dev/null 2>&1 || { echo "SKIP: node not installed"; exit 0; } +E2E="$(cd "$(dirname "$SUT")" && pwd)" + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } + +WORK="$(mktemp -d "${TMPDIR:-/tmp}/cfgneutral.XXXXXX")" +trap 'rm -rf "$WORK"' EXIT +mkdir -p "$WORK/proj"; : > "$WORK/proj/Fresh.mpr" + +# Install the way a scaffold does: the real install-tests.sh, pointed at the SUT's directory. +if [ -x "$TOOLKIT/bin/install-tests.sh" ] && [ "$E2E" = "$TOOLKIT/project-tests/e2e" ]; then + "$TOOLKIT/bin/install-tests.sh" "$WORK/proj" >"$WORK/install.log" 2>&1 || true +fi +D="$WORK/proj/tests/e2e" +if [ ! -f "$D/project.config.js" ]; then # a SUT outside the toolkit: copy beside its engine + mkdir -p "$D"; cp "$E2E"/*.js "$D/" 2>/dev/null; cp "$SUT" "$D/project.config.js" +fi +cmp -s "$SUT" "$D/project.config.js" && ok "the installed project.config.js is the template" \ + || bad "installed project.config.js differs from the subject" + +echo "== the CONFIGURED blocks ship empty ==" +IFS= read -r -d '' JS <<'EOF' || true +const c = require(process.argv[1]); +const n = (v) => (Array.isArray(v) ? v.length : v && typeof v === 'object' ? Object.keys(v).length : -1); +console.log('NAV ' + n(c.NAV)); +console.log('ROLES ' + n(c.roles)); +console.log('WIRE ' + n(c.PAGE_WIREFRAME)); +console.log('ACTS ' + n((c.walkthrough || {}).acts)); +console.log('SEEDS ' + n((c.walkthrough || {}).seeds || [])); +console.log('TPAGES ' + Object.values(c.TARGETS || {}).reduce((a, t) => a + ((t && t.pages) || []).length, 0)); +console.log('EX ' + Object.values(c.exemplars || {}).filter(Boolean).length); +console.log('PROBE ' + c.probePage('/nonexistent/page-scope.json')); +console.log('READY ' + c.readySelectors.filter((s) => /mx-name-/.test(s)).length); +EOF +OUT="$(env -u PROBE_PAGE node -e "$JS" "$D/project.config.js" 2>&1)" +line() { printf '%s\n' "$OUT" | sed -n "s/^$1 //p"; } +[ "$(line NAV)" = 0 ] && ok "NAV is empty" || bad "NAV ships entries" "$(line NAV)" +[ "$(line ROLES)" = 0 ] && ok "ROLES is empty (no users)" || bad "ROLES ships users" "$(line ROLES)" +[ "$(line WIRE)" = 0 ] && ok "PAGE_WIREFRAME is empty" || bad "PAGE_WIREFRAME ships rows" "$(line WIRE)" +[ "$(line ACTS)" = 0 ] && [ "$(line SEEDS)" = 0 ] && ok "WALKTHROUGH has no acts and no seeds" \ + || bad "WALKTHROUGH ships a process" "acts=$(line ACTS) seeds=$(line SEEDS)" +[ "$(line TPAGES)" = 0 ] && ok "TARGETS carry no page stops" || bad "TARGETS ship page stops" "$(line TPAGES)" +[ "$(line EX)" = 0 ] && ok "exemplars are unset" || bad "exemplars name pages" "$(line EX)" +[ "$(line PROBE)" = null ] && ok "PROBE_PAGE defaults to none (page-scope fallback)" || bad "PROBE_PAGE ships a page" "$(line PROBE)" +[ "$(line READY)" = 0 ] && ok "readySelectors name no project widget" || bad "readySelectors carry widget names" "$(line READY)" + +# Generic, so it needs no list of the old values: outside comments, no quoted <Module>.<Name>. +QN="$(sed -e 's#^[[:space:]]*//.*##' -e 's#[[:space:]]//[[:space:]].*##' "$SUT" | grep -noE "'[A-Z][A-Za-z0-9_]*\.[A-Z][A-Za-z0-9_]*'" | head -3)" +[ -z "$QN" ] && ok "no quoted <Module>.<Name> literal outside comments" || bad "qualified names in code" "$QN" + +echo "== a first run says what to fill ==" +P="$(cd "$D" && node project.config.js 2>&1)" +for k in NAV roles walkthrough; do + printf '%s\n' "$P" | grep -qE "^ $k .*FILL ME" && ok "node project.config.js marks $k FILL ME" \ + || bad "no FILL ME marker for $k" "$(printf '%s\n' "$P" | grep -E "^ $k " | head -1)" +done + +if [ -f "$D/full-app-walkthrough.js" ] && [ -f "$D/helpers.js" ]; then + W="$(cd "$D" && APP_PORT=1 timeout 60 node full-app-walkthrough.js 2>&1)"; RC=$? + [ "$RC" -eq 2 ] && ok "full-app-walkthrough exits 2 on the fresh config" || bad "walkthrough exit code" "rc=$RC $(printf '%s' "$W" | tail -2)" + printf '%s\n' "$W" | grep -q 'NOT CONFIGURED' && printf '%s\n' "$W" | grep -q 'WALKTHROUGH and ROLES' \ + && ok "it names the blocks to fill" || bad "no NOT CONFIGURED line naming WALKTHROUGH/ROLES" "$(printf '%s' "$W" | head -4)" + [ ! -f "$D/artifacts/findings.json" ] && ok "and writes no findings.json" || bad "a findings.json was written" +else + bad "full-app-walkthrough.js / helpers.js not installed beside the config" +fi + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] From 887dee1b6271aa1d84877a9b814aca523696ce02 Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Fri, 20 Nov 2026 19:11:21 +0000 Subject: [PATCH 2/4] fix(full-app-walkthrough): read an act's checks after its leading nav steps The act loop evaluated act.checks before act.steps. The first act has no handoffAssert to navigate for it, so a first act whose step 1 is the nav to its page read every check on the home page and failed them, one line before its own nav passed (field run on a real Mendix 11.8 app: "x raise:widget:dgTasks not visible", then "v raise:Tasks opens"). An act's leading `nav` steps are now its arrival: they run first, then the widget checks, then the remaining steps. A fatal arrival leaves the checks NOT REACHED. Acts that arrive through a handoffAssert are unchanged. Fixture: tests/wave2/test-walkthrough-act-order.sh (master 2/4, branch 4/4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXpQSjXefSNicGGqP4aiSC --- CHANGELOG.md | 1 + project-tests/e2e/full-app-walkthrough.js | 49 ++++++--- tests/wave2/test-walkthrough-act-order.sh | 125 ++++++++++++++++++++++ 3 files changed, 159 insertions(+), 16 deletions(-) create mode 100755 tests/wave2/test-walkthrough-act-order.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 00a6ecd7..dc333617 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m `docs/RELEASE-PROCESS.md` §8. ## Unreleased +- fix(full-app-walkthrough): **an act's widget `checks` are read on its landing page, after its leading `nav` steps**: the act loop read `act.checks` before the act's own `steps`, and the first act has no handoffAssert to navigate for it, so a first act whose step 1 is the nav to its page (the template example's own shape) FAILed every check against the home page (`✗ raise:widget:dgTasks … not visible`, then `✓ raise:Tasks opens`). Leading `nav` steps now run first, then `checks`, then the rest; a fatal arrival leaves the checks NOT REACHED instead of reading them on the previous page; handoff acts are unchanged. Fixture `tests/wave2/test-walkthrough-act-order.sh` (master 2/4, branch 4/4) — field run on a real Mendix 11.8 app under `mxcli run --local` (2026-11-20) - fix(e2e harness, install-tests): **a fresh scaffold's `tests/e2e/project.config.js` no longer carries another app's menu, users and process**: the installed file is the template, and the template shipped one sample app's CONFIGURED values (8 menu captions, a probe page, two users, a 3-act walkthrough, 14 wireframe rows, project widget names in `READY_SELECTORS`), so a fresh scaffold's monkey reached 0 of 8 targets and `full-app-walkthrough.js` died at its first seed with an OQL error for an entity the app never had. Every CONFIGURED block now ships empty with a `FILL ME` marker and the shape in a comment; `node tests/e2e/project.config.js` marks each empty block and names the instrument it stops; the walkthrough's NOT CONFIGURED message points at the blocks to fill; page-audit advice says `<no exemplar: set exemplars.listPage …>` instead of naming a page; `install-tests.sh` lists the blocks. A project's own `project.config.js` is never rewritten. Fixture `tests/wave2/test-e2e-config-neutral.sh` (master 1/16, branch 16/16) — field run on a blank Mendix 11.8 app under `mxcli run --local` (2026-11-20) - new(doctor, bug ledger): **a guard for mxcli's silent MxBuild fallback**: when the project's Mendix version is not in `~/.mxcli/mxbuild/` (and no Studio Pro of that version exists), `mxcli docker build` / `docker check` fall back to the newest cached MxBuild and print its path but no version warning (`run --local` is exact-or-refuse); `bin/doctor.sh` now warns, naming the model version, the cached ones and `./mxcli setup mxbuild -p <app>.mpr` (`mxtk_mxbuild_cache_gap`, warning only, never changes a gate). Ledger entry `BUG-DRAFT-mxbuild-silent-version-fallback`; upstream issue not filed. — a four-app Mendix build (field run), independent review - fix(gate-check): **a fresh clone no longer FAILs Stage 3 as "stale render"**: `blueprint.html` / `process-map.html` freshness used `[ render -ot source ]`, a nanosecond mtime compare, and a checkout stamps files in directory order (`process-map.html` lands before `process-model.json`), so an unedited project read stale on every clone (reproduced: 200-wireframe checkout, 10 of 10 clones). New `bin/lib/render-fresh.sh` keeps `-ot` as the first test (a pass never flips, no git call) and, only where `-ot` would FAIL, asks git once per render (one `log`, one `status`) whether the source was committed after the render's commit; measured 0.19 s for 200 sources all `-ot`-stale on a 300-commit history (a per-source git version cost 12.9 s). Limit: a depth-1 clone cannot see a genuinely stale render. Field run: a four-app Mendix build checked out on a second machine. — a four-app Mendix build (field run), independent review diff --git a/project-tests/e2e/full-app-walkthrough.js b/project-tests/e2e/full-app-walkthrough.js index 7ee388a4..1a4b0919 100644 --- a/project-tests/e2e/full-app-walkthrough.js +++ b/project-tests/e2e/full-app-walkthrough.js @@ -351,6 +351,15 @@ async function runStep(page, meta, s) { } } +// An act's arrival: its leading `nav` steps. `checks` describe the page the act lands on, so they +// are read after these (and after a handoffAssert), never on the page the previous act left. +function splitArrival(act) { + const steps = act.steps || []; + let i = 0; + while (i < steps.length && steps[i] && steps[i].do === 'nav') i++; + return { arrive: steps.slice(0, i), rest: steps.slice(i) }; +} + function describe(s) { switch (s.do) { case 'nav': return `navigate to ${[s.group, s.item].filter(Boolean).join(' › ')}`; @@ -533,22 +542,30 @@ async function switchTo(page, roleKey, handoffMeta) { console.log(`\n── ${meta.label} ──`); - // Widget presence for the act's landing page, recorded as widgets[] so the - // report groups them the way it groups every other walkthrough's. - for (const name of act.checks || []) { - const visible = await H.visibleWithin(page.locator(`.mx-name-${name}`).first(), 2500); - meta.widgets.push({ name, status: visible ? 'ok' : 'missing', - detail: visible ? '' : `.mx-name-${name} not visible` }); - if (visible) R.pass(`${meta.key}:widget:${name}`, `Widget ${name} present`); - else R.fail(`${meta.key}:widget:${name}`, `Widget ${name} present`, `.mx-name-${name} not visible`); - } - - for (const s of act.steps || []) { - const ok = await runStep(page, meta, s); - // `fatal` is the config's way of saying "there is no point continuing" — - // a form that never opened, a record that was never created. Everything - // after it is reported NOT REACHED rather than left absent. - if (!ok && s.fatal) { stoppedAt = `${meta.label}: ${s.label || describe(s)}`; break; } + const runSteps = async (list) => { + for (const s of list) { + const ok = await runStep(page, meta, s); + // `fatal` is the config's way of saying "there is no point continuing" — + // a form that never opened, a record that was never created. Everything + // after it is reported NOT REACHED rather than left absent. + if (!ok && s.fatal) { stoppedAt = `${meta.label}: ${s.label || describe(s)}`; return false; } + } + return true; + }; + // The act's LEADING nav steps are how it arrives on its landing page (a first act has + // no handoffAssert to navigate for it), so they run before its widget checks are read. + const { arrive, rest } = splitArrival(act); + if (await runSteps(arrive)) { + // Widget presence for the act's landing page, recorded as widgets[] so the + // report groups them the way it groups every other walkthrough's. + for (const name of act.checks || []) { + const visible = await H.visibleWithin(page.locator(`.mx-name-${name}`).first(), 2500); + meta.widgets.push({ name, status: visible ? 'ok' : 'missing', + detail: visible ? '' : `.mx-name-${name} not visible` }); + if (visible) R.pass(`${meta.key}:widget:${name}`, `Widget ${name} present`); + else R.fail(`${meta.key}:widget:${name}`, `Widget ${name} present`, `.mx-name-${name} not visible`); + } + await runSteps(rest); } if (!meta.screenshot) await shot(page, meta, `act-${String(i + 1).padStart(2, '0')}-${meta.key}`); if (stoppedAt) break; diff --git a/tests/wave2/test-walkthrough-act-order.sh b/tests/wave2/test-walkthrough-act-order.sh new file mode 100755 index 00000000..184d58cb --- /dev/null +++ b/tests/wave2/test-walkthrough-act-order.sh @@ -0,0 +1,125 @@ +#!/usr/bin/env bash +# Usage: bash test-walkthrough-act-order.sh [path-to-full-app-walkthrough.js] +# +# Fixture: full-app-walkthrough.js reads an act's widget `checks` on the act's LANDING page. +# +# The defect (field run 2026-11-20, a real Mendix 11.8 app under `mxcli run --local`): the act +# loop evaluated `act.checks` before the act's own `steps`. The first act has no handoffAssert to +# navigate for it, so when its first step is the `nav` that opens its page — the shape the +# template's own example act had — every check was read on the home page and FAILed: +# ✗ raise:widget:dgTasks … not visible +# ✓ raise:Tasks opens (the nav, one line later) +# Now an act's LEADING nav steps run first, then its checks, then the rest of its steps. +# +# No browser and no app: helpers.js is replaced by a stub whose page knows which menu item it +# is on and which widgets each page shows; config.js and the walkthrough are the real files. + +set -uo pipefail + +SUT="${1:-}" +[ -z "$SUT" ] && SUT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-tests/e2e/full-app-walkthrough.js" +[ -f "$SUT" ] || { echo "SKIP: subject not found at $SUT"; exit 0; } +command -v node >/dev/null 2>&1 || { echo "SKIP: node not installed"; exit 0; } +E2E="$(cd "$(dirname "$SUT")" && pwd)" +[ -f "$E2E/config.js" ] || { echo "FAIL — config.js not beside $SUT"; exit 1; } + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } + +WORK="$(mktemp -d "${TMPDIR:-/tmp}/walkorder.XXXXXX")" +trap 'rm -rf "$WORK"' EXIT +D="$WORK/tests/e2e"; mkdir -p "$D"; : > "$WORK/Fixture.mpr" +cp "$SUT" "$D/full-app-walkthrough.js"; cp "$E2E/config.js" "$D/config.js" + +cat > "$D/helpers.js" <<'EOF' +'use strict'; +const fs = require('fs'), path = require('path'); +// Which widgets each "page" shows. The app starts on Home. +const PAGES = { Home: ['homeTile'], Orders: ['dgOrders', 'btnNew'], Queue: ['dgQueue', 'btnApprove'] }; +const page = { + cur: 'Home', + on() {}, url() { return 'http://stub/' + this.cur; }, video() { return null; }, + locator(sel) { const pg = this; return { first: () => ({ sel, pg }) }; }, +}; +const results = []; +module.exports = { + fs, path, process, + ensureDirs: (...d) => d.forEach((x) => fs.mkdirSync(x, { recursive: true })), + makeReporter: () => ({ + pass: (id, l) => { results.push({ id, ok: true }); console.log(' ✓ ' + id); }, + fail: (id, l, d) => { results.push({ id, ok: false }); console.log(' ✗ ' + id + ' — ' + d); }, + summary: () => ({ results, passed: results.filter((r) => r.ok).length, failed: results.filter((r) => !r.ok).length }), + }), + launchBrowser: async () => ({ browser: { close: async () => {} }, + context: { close: async () => {}, tracing: { stop: async () => {} } }, page }), + login: async () => ({ ok: true, user: 'u1' }), + logout: async () => true, + navTo: async (pg, group, item) => { if (!PAGES[item]) throw new Error(`navigation item "${item}" not found`); pg.cur = item; }, + visibleWithin: async (loc) => { + const m = /mx-name-([A-Za-z0-9_]+)/.exec(loc.sel); + if (!m) return true; // the app-shell selector + return (PAGES[loc.pg.cur] || []).includes(m[1]); + }, + fullPageShot: async () => {}, oqlScalar: () => '1', dismissModal: async () => {}, + humanClick: async () => {}, humanType: async () => {}, humanTypeKeys: async () => {}, + injectCursor: async () => {}, injectOqlOverlay: async () => {}, +}; +EOF + +cat > "$D/project.config.js" <<'EOF' +'use strict'; +const path = require('path'); +module.exports = { + root: path.resolve(__dirname, '..', '..'), rootSource: 'fixture', mpr: path.resolve(__dirname, '..', '..', 'Fixture.mpr'), + mprName: 'Fixture.mpr', id: 'fixture', displayName: 'fixture', otelService: 'fixture', defaultPort: '1', + mxcli: './mxcli', credentials: () => ({ user: 'u1', pass: 'p', fallbackUser: '', fallbackPass: '' }), + readySelectors: ['.mx-page'], signOutPattern: null, NAV: {}, TARGETS: {}, PAGE_WIREFRAME: {}, + probePage: () => null, exemplars: {}, walkthroughs: [], + roles: { r1: { label: 'Requester', user: 'u1', pass: 'p' }, r2: { label: 'Approver', user: 'u2', pass: 'p' } }, + walkthrough: { + label: 'fixture', process: 'raise then approve', seeds: [], + acts: [ + // The first act: arrives by its own leading nav step (no handoffAssert). + { key: 'raise', role: 'r1', module: 'M1', checks: ['dgOrders', 'btnNew'], + steps: [{ do: 'nav', item: 'Orders', ready: 'dgOrders', label: 'Orders opens', fatal: true }, + { do: 'expect', widget: 'btnNew', label: 'New is offered' }] }, + // A handoff act: arrives by handoffAssert; its first step is NOT a nav, so it is unchanged. + { key: 'approve', role: 'r2', module: 'M2', checks: ['dgQueue'], + handoffAssert: [{ do: 'nav', item: 'Queue', ready: 'dgQueue', fatal: true }], + steps: [{ do: 'expect', widget: 'btnApprove', label: 'Approve is offered' }] }, + // A fatal arrival that fails: checks must be NOT REACHED, not read on the wrong page. + { key: 'gone', role: 'r2', module: 'M2', checks: ['dgGone'], + steps: [{ do: 'nav', item: 'Gone', ready: 'dgGone', label: 'Gone opens', fatal: true }] }, + ], + cleanup: null, + }, +}; +EOF + +OUT="$(cd "$D" && APP_PORT=1 SPEED=1000 DEMO_SPEED=1000 timeout 120 node full-app-walkthrough.js 2>&1)"; RC=$? +F="$D/artifacts/findings.json" +if [ ! -f "$F" ]; then bad "the walk wrote no findings.json" "rc=$RC $(printf '%s' "$OUT" | tail -5)"; else + IFS= read -r -d '' JS <<'EOF' || true +const d = JSON.parse(require('fs').readFileSync(process.argv[1], 'utf8')); +const pg = (k) => d.pages.find((p) => p.key === k) || { widgets: [], steps: [] }; +const w = (k, n) => (pg(k).widgets.find((x) => x.name === n) || {}); +console.log('RAISE ' + w('raise', 'dgOrders').status + ',' + w('raise', 'btnNew').status); +console.log('APPROVE ' + w('approve', 'dgQueue').status); +console.log('GONE ' + w('gone', 'dgGone').status + '|' + /NOT REACHED/.test(w('gone', 'dgGone').detail || '')); +console.log('STEPS ' + pg('raise').steps.map((s) => s.label + ':' + s.status).join(',')); +EOF + R="$(node -e "$JS" "$F" 2>&1)" + line() { printf '%s\n' "$R" | sed -n "s/^$1 //p"; } + [ "$(line RAISE)" = "ok,ok" ] && ok "first act: checks read AFTER its leading nav (dgOrders, btnNew present)" \ + || bad "first act's checks were read before it navigated" "$(line RAISE)" + [ "$(line APPROVE)" = "ok" ] && ok "handoff act: checks read after handoffAssert (unchanged)" || bad "handoff act checks" "$(line APPROVE)" + [ "$(line GONE)" = "missing|true" ] && ok "a fatal arrival leaves its checks NOT REACHED, not read on the previous page" \ + || bad "fatal-arrival checks" "$(line GONE)" + printf '%s\n' "$(line STEPS)" | grep -q 'Orders opens:pass' && printf '%s\n' "$(line STEPS)" | grep -q 'New is offered:pass' \ + && ok "the first act's steps still all run, once each" || bad "first act steps" "$(line STEPS)" +fi + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] From 0ecdc7e4e690b916d5f1f690dcdc8bea08ccf847 Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Fri, 20 Nov 2026 19:12:44 +0000 Subject: [PATCH 3/4] fix(monkey): an empty NAV is NOT CONFIGURED, and 0 of 0 targets is not measured With the template's NAV now empty, a fresh scaffold's monkey signed in, reported "reached 0 of 0 target(s)" and exited 0: exitCode() treated zero targets as a clean run. (0 of N > 0 already exits 2 since #679.) An empty NAV now stops before the browser with a SETUP NOT CONFIGURED line that says how to fill it, and exitCode() returns 2 for zero targets. The old sample NAV is still recognised (config.js TEMPLATE_NAV) for configs installed before the template went empty. Fixture: tests/wave2/test-monkey-template-nav.sh extended (master monkey.js 9/13, branch 13/13). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXpQSjXefSNicGGqP4aiSC --- CHANGELOG.md | 1 + project-tests/e2e/config.js | 8 ++--- project-tests/e2e/monkey.js | 17 +++++++--- tests/wave2/test-monkey-template-nav.sh | 45 ++++++++++++++++++------- 4 files changed, 50 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index dc333617..a1f89eac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m `docs/RELEASE-PROCESS.md` §8. ## Unreleased +- fix(monkey): **an empty NAV is NOT CONFIGURED (exit 2), and "reached 0 of 0 target(s)" is not measured**: with the template's NAV now empty, a fresh scaffold's monkey signed in, reached 0 of 0 targets and exited 0 — a run that fuzzed nothing reading green to anything that checks the exit code (`exitCode()` treated 0 targets as clean; the 0-of-8 case was already exit 2 since #679). An empty NAV now stops before the browser with `SETUP NOT CONFIGURED: NAV … is empty` and how to fill it, `exitCode()` returns 2 for 0 targets, and the old sample NAV is still recognised for configs installed earlier. Fixture `tests/wave2/test-monkey-template-nav.sh` extended (master monkey.js 9/13, branch 13/13) — field run on a real Mendix 11.8 app under `mxcli run --local` (2026-11-20) - fix(full-app-walkthrough): **an act's widget `checks` are read on its landing page, after its leading `nav` steps**: the act loop read `act.checks` before the act's own `steps`, and the first act has no handoffAssert to navigate for it, so a first act whose step 1 is the nav to its page (the template example's own shape) FAILed every check against the home page (`✗ raise:widget:dgTasks … not visible`, then `✓ raise:Tasks opens`). Leading `nav` steps now run first, then `checks`, then the rest; a fatal arrival leaves the checks NOT REACHED instead of reading them on the previous page; handoff acts are unchanged. Fixture `tests/wave2/test-walkthrough-act-order.sh` (master 2/4, branch 4/4) — field run on a real Mendix 11.8 app under `mxcli run --local` (2026-11-20) - fix(e2e harness, install-tests): **a fresh scaffold's `tests/e2e/project.config.js` no longer carries another app's menu, users and process**: the installed file is the template, and the template shipped one sample app's CONFIGURED values (8 menu captions, a probe page, two users, a 3-act walkthrough, 14 wireframe rows, project widget names in `READY_SELECTORS`), so a fresh scaffold's monkey reached 0 of 8 targets and `full-app-walkthrough.js` died at its first seed with an OQL error for an entity the app never had. Every CONFIGURED block now ships empty with a `FILL ME` marker and the shape in a comment; `node tests/e2e/project.config.js` marks each empty block and names the instrument it stops; the walkthrough's NOT CONFIGURED message points at the blocks to fill; page-audit advice says `<no exemplar: set exemplars.listPage …>` instead of naming a page; `install-tests.sh` lists the blocks. A project's own `project.config.js` is never rewritten. Fixture `tests/wave2/test-e2e-config-neutral.sh` (master 1/16, branch 16/16) — field run on a blank Mendix 11.8 app under `mxcli run --local` (2026-11-20) - new(doctor, bug ledger): **a guard for mxcli's silent MxBuild fallback**: when the project's Mendix version is not in `~/.mxcli/mxbuild/` (and no Studio Pro of that version exists), `mxcli docker build` / `docker check` fall back to the newest cached MxBuild and print its path but no version warning (`run --local` is exact-or-refuse); `bin/doctor.sh` now warns, naming the model version, the cached ones and `./mxcli setup mxbuild -p <app>.mpr` (`mxtk_mxbuild_cache_gap`, warning only, never changes a gate). Ledger entry `BUG-DRAFT-mxbuild-silent-version-fallback`; upstream issue not filed. — a four-app Mendix build (field run), independent review diff --git a/project-tests/e2e/config.js b/project-tests/e2e/config.js index 524d8e61..85387b36 100644 --- a/project-tests/e2e/config.js +++ b/project-tests/e2e/config.js @@ -233,10 +233,10 @@ function resolveTarget() { return t; } -// The NAV project.config.template.js ships: the toolkit's sample app, not this project (#679). -// A project.config.js still carrying it navigates menu captions the app does not have, so a -// monkey run reaches nothing and measures nothing. Compared like TEMPLATE_PASSWORDS above; -// tests/wave2/test-monkey-template-nav.sh asserts this copy equals the template's. +// The NAV project.config.template.js used to ship: a sample app's, not this project's (#679). +// The template now ships NAV empty, but a project.config.js installed before that still carries +// this one and navigates menu captions the app does not have, so a monkey run reaches nothing +// and measures nothing. Kept as the signature that recognises it; compared like TEMPLATE_PASSWORDS. const TEMPLATE_NAV = { dashboard: ['Dashboard'], equipment: ['Assets & Barcodes', 'Equipment'], diff --git a/project-tests/e2e/monkey.js b/project-tests/e2e/monkey.js index 112f2c8d..7cff7f3d 100644 --- a/project-tests/e2e/monkey.js +++ b/project-tests/e2e/monkey.js @@ -290,12 +290,13 @@ async function round(page, target, bucket) { return traceRan; } -// Exit codes (#679): 1 = a crash-class finding; 2 = instrument fault, NOT measured (the NAV is -// the template's sample, or 0 of N targets were reached); 3 = unknown --target; 0 otherwise. +// Exit codes (#679): 1 = a crash-class finding; 2 = instrument fault, NOT measured (NAV is empty +// or the old template sample, or 0 of N targets were reached — N = 0 included: a fresh scaffold's +// NAV is empty, and "reached 0 of 0" fuzzed nothing); 3 = unknown --target; 0 otherwise. // "0 of 8 reached" and "8 of 8 reached, no crash" used to both exit 0. function exitCode({ crashes, targets, unreached }) { if (crashes > 0) return 1; - if (targets > 0 && unreached >= targets) return 2; + if (targets === 0 || unreached >= targets) return 2; return 0; } @@ -306,8 +307,14 @@ module.exports = { leftApp, SIGN_OUT, SIGN_OUT_DEFAULT, signOutPattern, isSignOu if (require.main === module) (async () => { console.log(`MONKEY_SEED=${SEED} rounds=${ROUNDS} (re-run exactly: --seed ${SEED})`); - // Before the browser (#679): a NAV still equal to the template's sample would land on menu - // captions this app does not have — a run that measures nothing, and costs a login. + // Before the browser (#679): an empty NAV (a fresh scaffold's) or one still equal to the old + // template sample lands on nothing this app has — a run that measures nothing, and costs a login. + if (!NAV || !Object.keys(NAV).length) { + console.log(' SETUP NOT CONFIGURED: NAV in tests/e2e/project.config.js is empty — monkey has no menu item ' + + 'to fuzz. Fill it from `mxcli -p <mpr> -c "describe navigation"` (key: [\'Group\', \'Item\'] or [\'Item\']). ' + + 'Monkey NOT run: not measured.'); + process.exit(2); + } if (navIsTemplate()) { console.log(' SETUP NAV is the unedited sample from project.config.template.js — none of its menu ' + 'captions are this app\'s. Fill NAV in tests/e2e/project.config.js from ' + diff --git a/tests/wave2/test-monkey-template-nav.sh b/tests/wave2/test-monkey-template-nav.sh index 4a1b3a87..660c5ec0 100755 --- a/tests/wave2/test-monkey-template-nav.sh +++ b/tests/wave2/test-monkey-template-nav.sh @@ -12,6 +12,11 @@ # # Golden input: the template NAV is project.config.template.js itself, copied in as the # project's project.config.js exactly as bin/install-tests.sh seeds a new project. +# +# Follow-up (field run 2026-11-20, a real Mendix 11.8 app): the template now ships NAV EMPTY, and +# an empty NAV reached "0 of 0 target(s)" and exited 0 after a login. An empty NAV now exits 2 +# before the browser, and exitCode() treats 0 targets as not measured. The old sample NAV is still +# recognised (config.js TEMPLATE_NAV) for project.config.js files installed before the change. set -uo pipefail @@ -41,23 +46,24 @@ mkdir -p "$WORK/tests/e2e"; : > "$WORK/Fixture.mpr" cp "$SUT" "$E2E/helpers.js" "$E2E/otel.js" "$E2E/config.js" "$WORK/tests/e2e/" cp "$CFG" "$WORK/tests/e2e/project.config.js" -echo "== config.js: navIsTemplate() against the shipped template ==" +echo "== config.js: navIsTemplate() against the old sample and the shipped (empty) template ==" IFS= read -r -d '' JS <<'EOF' || true const C = require('./config.js'); if (typeof C.navIsTemplate !== 'function') { console.log('NOFN'); process.exit(0); } -console.log('TPL ' + C.navIsTemplate()); -console.log('SAME ' + (JSON.stringify(C.TEMPLATE_NAV) === JSON.stringify(require('./project.config.js').NAV))); -const edited = Object.assign({}, C.NAV, { dashboard: ['Home'] }); +const S = C.TEMPLATE_NAV; +console.log('TPL ' + C.navIsTemplate({ nav: S })); +console.log('EMPTY ' + Object.keys(require('./project.config.js').NAV || {}).length + ',' + C.navIsTemplate()); +const edited = Object.assign({}, S, { dashboard: ['Home'] }); console.log('EDITED ' + C.navIsTemplate({ nav: edited })); -const reordered = {}; for (const k of Object.keys(C.NAV).reverse()) reordered[k] = C.NAV[k]; +const reordered = {}; for (const k of Object.keys(S).reverse()) reordered[k] = S[k]; console.log('REORDER ' + C.navIsTemplate({ nav: reordered })); -console.log('ALLOW ' + C.navIsTemplate({ env: { ALLOW_TEMPLATE_NAV: '1' } })); +console.log('ALLOW ' + C.navIsTemplate({ nav: S, env: { ALLOW_TEMPLATE_NAV: '1' } })); EOF OUT="$(cd "$WORK/tests/e2e" && printf '%s' "$JS" | APP_PORT=1 node - 2>&1)" line() { printf '%s\n' "$OUT" | sed -n "s/^$1 //p"; } if printf '%s\n' "$OUT" | grep -qx NOFN; then bad "#679 config.js exports no navIsTemplate()"; else - [ "$(line TPL)" = true ] && ok "#679 the template's NAV, unedited, reads as the sample" || bad "#679 template NAV not detected" "$OUT" - [ "$(line SAME)" = true ] && ok "config.js TEMPLATE_NAV equals project.config.template.js's NAV (no drift)" || bad "TEMPLATE_NAV drifted from the template" "$(line SAME)" + [ "$(line TPL)" = true ] && ok "#679 the old sample NAV, unedited, reads as the sample" || bad "#679 sample NAV not detected" "$OUT" + [ "$(line EMPTY)" = "0,false" ] && ok "the shipped template's NAV is empty (and is not 'the sample')" || bad "template NAV not empty" "$(line EMPTY)" [ "$(line EDITED)" = false ] && ok "one edited caption → not the sample" || bad "edited NAV flagged as template" "$(line EDITED)" [ "$(line REORDER)" = true ] && ok "key order is ignored" || bad "reordered template NAV missed" "$(line REORDER)" [ "$(line ALLOW)" = false ] && ok "ALLOW_TEMPLATE_NAV=1 overrides" || bad "ALLOW_TEMPLATE_NAV ignored" "$(line ALLOW)" @@ -68,19 +74,34 @@ IFS= read -r -d '' JS2 <<'EOF' || true const M = require('./monkey.js'); if (typeof M.exitCode !== 'function') { console.log('NOFN'); process.exit(0); } const e = (c, t, u) => M.exitCode({ crashes: c, targets: t, unreached: u }); -console.log('CODES ' + [e(0, 8, 8), e(0, 8, 0), e(1, 8, 8), e(0, 8, 7), e(1, 8, 0), e(0, 0, 0)].join(',')); +console.log('CODES ' + [e(0, 8, 8), e(0, 8, 0), e(1, 8, 8), e(0, 8, 7), e(1, 8, 0), e(0, 0, 0), e(1, 0, 0)].join(',')); EOF OUT="$(cd "$WORK/tests/e2e" && printf '%s' "$JS2" | APP_PORT=1 timeout 60 node - 2>&1)" if printf '%s\n' "$OUT" | grep -qx NOFN; then bad "#679 monkey.js exports no exitCode()"; else - [ "$(line CODES)" = "2,0,1,0,1,0" ] \ - && ok "#679 0 of N reached → 2 (not measured); some reached → 0; a crash → 1; no targets → 0" \ + [ "$(line CODES)" = "2,0,1,0,1,2,1" ] \ + && ok "#679 0 of N reached → 2 (not measured); some reached → 0; a crash → 1; 0 of 0 → 2" \ || bad "#679 exit codes" "$(line CODES)" fi grep -qF 'process.exit(code);' "$SUT" && grep -qF 'const code = exitCode({ crashes: crashes.length, targets: targets.length, unreached: unreached.length });' "$SUT" \ && ok "#679 main exits with exitCode() (the tested function is the one that runs)" \ || bad "#679 main does not use exitCode()" -echo "== monkey.js main: the sample NAV exits 2 BEFORE the browser ==" +echo "== monkey.js main: the shipped (empty) NAV exits 2 BEFORE the browser ==" +RUNOUT="$(cd "$WORK/tests/e2e" && env -u ALLOW_TEMPLATE_NAV APP_PORT=1 timeout 120 node monkey.js --rounds 1 2>&1)"; RC=$? +[ "$RC" -eq 2 ] && ok "empty NAV → exit 2 (not measured)" || bad "empty NAV exit code" "rc=$RC $(printf '%s' "$RUNOUT" | tail -3)" +printf '%s\n' "$RUNOUT" | grep -q 'SETUP NOT CONFIGURED: NAV' && ok "a SETUP line says NAV is not configured and how to fill it" \ + || bad "no NOT CONFIGURED line for the empty NAV" "$(printf '%s' "$RUNOUT" | tail -3)" +printf '%s\n' "$RUNOUT" | grep -qE 'INSTRUMENT FAULT|\(login\)|MONKEY SUMMARY|page\.goto|loginOnce' \ + && bad "the empty-NAV run went past the check (browser/login reached)" "$(printf '%s' "$RUNOUT" | tail -3)" \ + || ok "nothing after the NAV check ran: no browser launch, no login attempt" + +echo "== monkey.js main: the old sample NAV exits 2 BEFORE the browser ==" +# A project.config.js installed before the template went empty: its NAV is the old sample. +( cd "$WORK/tests/e2e" && APP_PORT=1 node -e ' + const fs = require("fs"), f = "project.config.js"; + const nav = JSON.stringify(require("./config.js").TEMPLATE_NAV || {}); + fs.writeFileSync(f, fs.readFileSync(f, "utf8").replace(/const NAV = \{[\s\S]*?\n\};/, "const NAV = " + nav + ";")); +' ) >/dev/null 2>&1 # APP_PORT=1: nothing listens. Without the check the run launches a browser and fails at login # (exit 1), or exits 2 on a missing playwright — neither names the NAV. RUNOUT="$(cd "$WORK/tests/e2e" && env -u ALLOW_TEMPLATE_NAV APP_PORT=1 timeout 120 node monkey.js --rounds 1 2>&1)"; RC=$? From 340afd7537f96bcc822b92941d8a8283bf264575 Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Fri, 20 Nov 2026 20:18:48 +0000 Subject: [PATCH 4/4] =?UTF-8?q?fix(full-app-walkthrough,=20monkey):=20revi?= =?UTF-8?q?ew=20of=20#826=20=E2=80=94=20handoff=20acts=20read=20checks=20a?= =?UTF-8?q?fter=20the=20assert?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Independent review: the nav-first arrival applied to every act, so a handoff act that arrives on page X through its handoffAssert and opens its steps with a nav to Y had its X checks read on Y (reproduced live on Mendix 11.8: see:widget:dgSideBar passed on master, failed on the previous head). After a non-empty handoffAssert the checks are read straight away again; leading-nav arrival applies only to acts without one. The template comment says so. Monkey's NOT CONFIGURED line names --skip-monkey for an app with no menu to fuzz. Fixtures: test-walkthrough-act-order.sh gains that case (master 3/5, previous head 4/5, branch 5/5); test-monkey-template-nav.sh asserts --skip-monkey (previous head 12/13, 13/13). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXpQSjXefSNicGGqP4aiSC --- CHANGELOG.md | 4 ++-- project-tests/e2e/full-app-walkthrough.js | 14 +++++++++----- project-tests/e2e/monkey.js | 4 ++-- project-tests/e2e/project.config.template.js | 5 +++-- tests/wave2/test-monkey-template-nav.sh | 3 ++- tests/wave2/test-walkthrough-act-order.sh | 9 +++++++++ 6 files changed, 27 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a1f89eac..4fd67aef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,8 +18,8 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m `docs/RELEASE-PROCESS.md` §8. ## Unreleased -- fix(monkey): **an empty NAV is NOT CONFIGURED (exit 2), and "reached 0 of 0 target(s)" is not measured**: with the template's NAV now empty, a fresh scaffold's monkey signed in, reached 0 of 0 targets and exited 0 — a run that fuzzed nothing reading green to anything that checks the exit code (`exitCode()` treated 0 targets as clean; the 0-of-8 case was already exit 2 since #679). An empty NAV now stops before the browser with `SETUP NOT CONFIGURED: NAV … is empty` and how to fill it, `exitCode()` returns 2 for 0 targets, and the old sample NAV is still recognised for configs installed earlier. Fixture `tests/wave2/test-monkey-template-nav.sh` extended (master monkey.js 9/13, branch 13/13) — field run on a real Mendix 11.8 app under `mxcli run --local` (2026-11-20) -- fix(full-app-walkthrough): **an act's widget `checks` are read on its landing page, after its leading `nav` steps**: the act loop read `act.checks` before the act's own `steps`, and the first act has no handoffAssert to navigate for it, so a first act whose step 1 is the nav to its page (the template example's own shape) FAILed every check against the home page (`✗ raise:widget:dgTasks … not visible`, then `✓ raise:Tasks opens`). Leading `nav` steps now run first, then `checks`, then the rest; a fatal arrival leaves the checks NOT REACHED instead of reading them on the previous page; handoff acts are unchanged. Fixture `tests/wave2/test-walkthrough-act-order.sh` (master 2/4, branch 4/4) — field run on a real Mendix 11.8 app under `mxcli run --local` (2026-11-20) +- fix(monkey): **an empty NAV is NOT CONFIGURED (exit 2), and "reached 0 of 0 target(s)" is not measured**: with the template's NAV now empty, a fresh scaffold's monkey signed in, reached 0 of 0 targets and exited 0 — a run that fuzzed nothing reading green to anything that checks the exit code (`exitCode()` treated 0 targets as clean; the 0-of-8 case was already exit 2 since #679). An empty NAV now stops before the browser with `SETUP NOT CONFIGURED: NAV … is empty`, how to fill it, and `--skip-monkey` for an app with no menu to fuzz, `exitCode()` returns 2 for 0 targets, and the old sample NAV is still recognised for configs installed earlier. Fixture `tests/wave2/test-monkey-template-nav.sh` extended (master monkey.js 9/13, branch 13/13) — field run on a real Mendix 11.8 app under `mxcli run --local` (2026-11-20) +- fix(full-app-walkthrough): **an act's widget `checks` are read on its landing page, after its leading `nav` steps**: the act loop read `act.checks` before the act's own `steps`, and the first act has no handoffAssert to navigate for it, so a first act whose step 1 is the nav to its page (the template example's own shape) FAILed every check against the home page (`✗ raise:widget:dgTasks … not visible`, then `✓ raise:Tasks opens`). For an act with no handoffAssert, leading `nav` steps now run first, then `checks`, then the rest; a fatal arrival leaves the checks NOT REACHED instead of reading them on the previous page. After a non-empty handoffAssert the checks are read straight away, as before, even when the act's steps open with a nav elsewhere (independent review: an earlier cut read them after that nav). Fixture `tests/wave2/test-walkthrough-act-order.sh` (master 3/5, branch 5/5) — field run on a real Mendix 11.8 app under `mxcli run --local` (2026-11-20) - fix(e2e harness, install-tests): **a fresh scaffold's `tests/e2e/project.config.js` no longer carries another app's menu, users and process**: the installed file is the template, and the template shipped one sample app's CONFIGURED values (8 menu captions, a probe page, two users, a 3-act walkthrough, 14 wireframe rows, project widget names in `READY_SELECTORS`), so a fresh scaffold's monkey reached 0 of 8 targets and `full-app-walkthrough.js` died at its first seed with an OQL error for an entity the app never had. Every CONFIGURED block now ships empty with a `FILL ME` marker and the shape in a comment; `node tests/e2e/project.config.js` marks each empty block and names the instrument it stops; the walkthrough's NOT CONFIGURED message points at the blocks to fill; page-audit advice says `<no exemplar: set exemplars.listPage …>` instead of naming a page; `install-tests.sh` lists the blocks. A project's own `project.config.js` is never rewritten. Fixture `tests/wave2/test-e2e-config-neutral.sh` (master 1/16, branch 16/16) — field run on a blank Mendix 11.8 app under `mxcli run --local` (2026-11-20) - new(doctor, bug ledger): **a guard for mxcli's silent MxBuild fallback**: when the project's Mendix version is not in `~/.mxcli/mxbuild/` (and no Studio Pro of that version exists), `mxcli docker build` / `docker check` fall back to the newest cached MxBuild and print its path but no version warning (`run --local` is exact-or-refuse); `bin/doctor.sh` now warns, naming the model version, the cached ones and `./mxcli setup mxbuild -p <app>.mpr` (`mxtk_mxbuild_cache_gap`, warning only, never changes a gate). Ledger entry `BUG-DRAFT-mxbuild-silent-version-fallback`; upstream issue not filed. — a four-app Mendix build (field run), independent review - fix(gate-check): **a fresh clone no longer FAILs Stage 3 as "stale render"**: `blueprint.html` / `process-map.html` freshness used `[ render -ot source ]`, a nanosecond mtime compare, and a checkout stamps files in directory order (`process-map.html` lands before `process-model.json`), so an unedited project read stale on every clone (reproduced: 200-wireframe checkout, 10 of 10 clones). New `bin/lib/render-fresh.sh` keeps `-ot` as the first test (a pass never flips, no git call) and, only where `-ot` would FAIL, asks git once per render (one `log`, one `status`) whether the source was committed after the render's commit; measured 0.19 s for 200 sources all `-ot`-stale on a 300-commit history (a per-source git version cost 12.9 s). Limit: a depth-1 clone cannot see a genuinely stale render. Field run: a four-app Mendix build checked out on a second machine. — a four-app Mendix build (field run), independent review diff --git a/project-tests/e2e/full-app-walkthrough.js b/project-tests/e2e/full-app-walkthrough.js index 1a4b0919..ddfc0402 100644 --- a/project-tests/e2e/full-app-walkthrough.js +++ b/project-tests/e2e/full-app-walkthrough.js @@ -351,8 +351,9 @@ async function runStep(page, meta, s) { } } -// An act's arrival: its leading `nav` steps. `checks` describe the page the act lands on, so they -// are read after these (and after a handoffAssert), never on the page the previous act left. +// An act's arrival when no handoffAssert brought it there: its leading `nav` steps. `checks` +// describe the page the act lands on, so they are read after these, never on the page the +// previous act left. After a non-empty handoffAssert the checks are read straight away. function splitArrival(act) { const steps = act.steps || []; let i = 0; @@ -521,6 +522,8 @@ async function switchTo(page, roleKey, handoffMeta) { // A role change is a HANDOFF, and it gets its own page in the report so the // seam is visible as a thing that was tested rather than buried inside an act. + // An act that arrives through a handoffAssert is ON its landing page once that assert ran. + const arrivedByHandoff = act.role !== currentRole && (act.handoffAssert || []).length > 0; if (act.role !== currentRole) { const from = currentRole ? (ROLES[currentRole].label || currentRole) : '(nobody)'; const to = (ROLES[act.role] && ROLES[act.role].label) || act.role; @@ -552,9 +555,10 @@ async function switchTo(page, roleKey, handoffMeta) { } return true; }; - // The act's LEADING nav steps are how it arrives on its landing page (a first act has - // no handoffAssert to navigate for it), so they run before its widget checks are read. - const { arrive, rest } = splitArrival(act); + // An act's landing page is where its handoffAssert left it. An act WITHOUT one (the first + // act, or one with no role change) arrives by its LEADING nav steps, so those run before its + // widget checks are read. A handoff act's steps, a leading nav included, run after them. + const { arrive, rest } = arrivedByHandoff ? { arrive: [], rest: act.steps || [] } : splitArrival(act); if (await runSteps(arrive)) { // Widget presence for the act's landing page, recorded as widgets[] so the // report groups them the way it groups every other walkthrough's. diff --git a/project-tests/e2e/monkey.js b/project-tests/e2e/monkey.js index 7cff7f3d..8d4d699a 100644 --- a/project-tests/e2e/monkey.js +++ b/project-tests/e2e/monkey.js @@ -311,8 +311,8 @@ if (require.main === module) (async () => { // template sample lands on nothing this app has — a run that measures nothing, and costs a login. if (!NAV || !Object.keys(NAV).length) { console.log(' SETUP NOT CONFIGURED: NAV in tests/e2e/project.config.js is empty — monkey has no menu item ' + - 'to fuzz. Fill it from `mxcli -p <mpr> -c "describe navigation"` (key: [\'Group\', \'Item\'] or [\'Item\']). ' + - 'Monkey NOT run: not measured.'); + 'to fuzz. Fill it from `mxcli -p <mpr> -c "describe navigation"` (key: [\'Group\', \'Item\'] or [\'Item\']); ' + + 'an app with no menu to fuzz runs verify-module.sh with --skip-monkey instead. Monkey NOT run: not measured.'); process.exit(2); } if (navIsTemplate()) { diff --git a/project-tests/e2e/project.config.template.js b/project-tests/e2e/project.config.template.js index b59d2942..5db037e9 100644 --- a/project-tests/e2e/project.config.template.js +++ b/project-tests/e2e/project.config.template.js @@ -339,8 +339,9 @@ const ROLES = { // seeds: [{ name: 'orderNo', sql: "SELECT OrderNo AS o FROM Sales.Order WHERE Status = 'Open' LIMIT 1" }], // acts: [ // { key: 'raise', role: 'requester', module: 'Sales', label: 'raise an order', -// // `checks` are read on the act's landing page: after its handoffAssert, and -// // after its LEADING `nav` steps — so a first act may simply start with a nav. +// // `checks` are read on the act's landing page: straight after its handoffAssert +// // when it has one; otherwise after its LEADING `nav` steps — so a first act may +// // simply start with a nav. // checks: ['dgOrders'], // steps: [ // { do: 'nav', group: 'Sales', item: 'Orders', ready: 'dgOrders', fatal: true }, diff --git a/tests/wave2/test-monkey-template-nav.sh b/tests/wave2/test-monkey-template-nav.sh index 660c5ec0..5a64b1a8 100755 --- a/tests/wave2/test-monkey-template-nav.sh +++ b/tests/wave2/test-monkey-template-nav.sh @@ -89,7 +89,8 @@ grep -qF 'process.exit(code);' "$SUT" && grep -qF 'const code = exitCode({ crash echo "== monkey.js main: the shipped (empty) NAV exits 2 BEFORE the browser ==" RUNOUT="$(cd "$WORK/tests/e2e" && env -u ALLOW_TEMPLATE_NAV APP_PORT=1 timeout 120 node monkey.js --rounds 1 2>&1)"; RC=$? [ "$RC" -eq 2 ] && ok "empty NAV → exit 2 (not measured)" || bad "empty NAV exit code" "rc=$RC $(printf '%s' "$RUNOUT" | tail -3)" -printf '%s\n' "$RUNOUT" | grep -q 'SETUP NOT CONFIGURED: NAV' && ok "a SETUP line says NAV is not configured and how to fill it" \ +printf '%s\n' "$RUNOUT" | grep -q 'SETUP NOT CONFIGURED: NAV' && printf '%s\n' "$RUNOUT" | grep -q -- '--skip-monkey' \ + && ok "a SETUP line says NAV is not configured, how to fill it, and names --skip-monkey" \ || bad "no NOT CONFIGURED line for the empty NAV" "$(printf '%s' "$RUNOUT" | tail -3)" printf '%s\n' "$RUNOUT" | grep -qE 'INSTRUMENT FAULT|\(login\)|MONKEY SUMMARY|page\.goto|loginOnce' \ && bad "the empty-NAV run went past the check (browser/login reached)" "$(printf '%s' "$RUNOUT" | tail -3)" \ diff --git a/tests/wave2/test-walkthrough-act-order.sh b/tests/wave2/test-walkthrough-act-order.sh index 184d58cb..26786c78 100755 --- a/tests/wave2/test-walkthrough-act-order.sh +++ b/tests/wave2/test-walkthrough-act-order.sh @@ -88,6 +88,12 @@ module.exports = { { key: 'approve', role: 'r2', module: 'M2', checks: ['dgQueue'], handoffAssert: [{ do: 'nav', item: 'Queue', ready: 'dgQueue', fatal: true }], steps: [{ do: 'expect', widget: 'btnApprove', label: 'Approve is offered' }] }, + // A handoff act whose steps OPEN with a nav elsewhere (independent review of #826): its + // checks describe the handoffAssert's page, so they are read there, before that nav. + { key: 'back', role: 'r1', module: 'M1', checks: ['dgOrders'], + handoffAssert: [{ do: 'nav', item: 'Orders', ready: 'dgOrders', fatal: true }], + steps: [{ do: 'nav', item: 'Queue', ready: 'dgQueue', label: 'Queue opens' }, + { do: 'expect', widget: 'dgQueue', label: 'the queue is listed' }] }, // A fatal arrival that fails: checks must be NOT REACHED, not read on the wrong page. { key: 'gone', role: 'r2', module: 'M2', checks: ['dgGone'], steps: [{ do: 'nav', item: 'Gone', ready: 'dgGone', label: 'Gone opens', fatal: true }] }, @@ -106,6 +112,7 @@ const pg = (k) => d.pages.find((p) => p.key === k) || { widgets: [], steps: [] } const w = (k, n) => (pg(k).widgets.find((x) => x.name === n) || {}); console.log('RAISE ' + w('raise', 'dgOrders').status + ',' + w('raise', 'btnNew').status); console.log('APPROVE ' + w('approve', 'dgQueue').status); +console.log('BACK ' + w('back', 'dgOrders').status + '|' + pg('back').steps.map((s) => s.label + ':' + s.status).join(',')); console.log('GONE ' + w('gone', 'dgGone').status + '|' + /NOT REACHED/.test(w('gone', 'dgGone').detail || '')); console.log('STEPS ' + pg('raise').steps.map((s) => s.label + ':' + s.status).join(',')); EOF @@ -114,6 +121,8 @@ EOF [ "$(line RAISE)" = "ok,ok" ] && ok "first act: checks read AFTER its leading nav (dgOrders, btnNew present)" \ || bad "first act's checks were read before it navigated" "$(line RAISE)" [ "$(line APPROVE)" = "ok" ] && ok "handoff act: checks read after handoffAssert (unchanged)" || bad "handoff act checks" "$(line APPROVE)" + case "$(line BACK)" in ok\|*"Queue opens:pass"*) ok "handoff act opening with a nav: checks read on the handoffAssert page, then its nav runs" ;; + *) bad "handoff act's checks were read after its own leading nav (on the wrong page)" "$(line BACK)" ;; esac [ "$(line GONE)" = "missing|true" ] && ok "a fatal arrival leaves its checks NOT REACHED, not read on the previous page" \ || bad "fatal-arrival checks" "$(line GONE)" printf '%s\n' "$(line STEPS)" | grep -q 'Orders opens:pass' && printf '%s\n' "$(line STEPS)" | grep -q 'New is offered:pass' \