Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m
`docs/RELEASE-PROCESS.md` §8.

## Unreleased
- fix(monkey): **an empty NAV is NOT CONFIGURED (exit 2), and "reached 0 of 0 target(s)" is not measured**: with the template's NAV now empty, a fresh scaffold's monkey signed in, reached 0 of 0 targets and exited 0 — a run that fuzzed nothing reading green to anything that checks the exit code (`exitCode()` treated 0 targets as clean; the 0-of-8 case was already exit 2 since #679). An empty NAV now stops before the browser with `SETUP NOT CONFIGURED: NAV … is empty`, how to fill it, and `--skip-monkey` for an app with no menu to fuzz, `exitCode()` returns 2 for 0 targets, and the old sample NAV is still recognised for configs installed earlier. Fixture `tests/wave2/test-monkey-template-nav.sh` extended (master monkey.js 9/13, branch 13/13) — field run on a real Mendix 11.8 app under `mxcli run --local` (2026-11-20)
- fix(full-app-walkthrough): **an act's widget `checks` are read on its landing page, after its leading `nav` steps**: the act loop read `act.checks` before the act's own `steps`, and the first act has no handoffAssert to navigate for it, so a first act whose step 1 is the nav to its page (the template example's own shape) FAILed every check against the home page (`✗ raise:widget:dgTasks … not visible`, then `✓ raise:Tasks opens`). For an act with no handoffAssert, leading `nav` steps now run first, then `checks`, then the rest; a fatal arrival leaves the checks NOT REACHED instead of reading them on the previous page. After a non-empty handoffAssert the checks are read straight away, as before, even when the act's steps open with a nav elsewhere (independent review: an earlier cut read them after that nav). Fixture `tests/wave2/test-walkthrough-act-order.sh` (master 3/5, branch 5/5) — field run on a real Mendix 11.8 app under `mxcli run --local` (2026-11-20)
- fix(e2e harness, install-tests): **a fresh scaffold's `tests/e2e/project.config.js` no longer carries another app's menu, users and process**: the installed file is the template, and the template shipped one sample app's CONFIGURED values (8 menu captions, a probe page, two users, a 3-act walkthrough, 14 wireframe rows, project widget names in `READY_SELECTORS`), so a fresh scaffold's monkey reached 0 of 8 targets and `full-app-walkthrough.js` died at its first seed with an OQL error for an entity the app never had. Every CONFIGURED block now ships empty with a `FILL ME` marker and the shape in a comment; `node tests/e2e/project.config.js` marks each empty block and names the instrument it stops; the walkthrough's NOT CONFIGURED message points at the blocks to fill; page-audit advice says `<no exemplar: set exemplars.listPage …>` instead of naming a page; `install-tests.sh` lists the blocks. A project's own `project.config.js` is never rewritten. Fixture `tests/wave2/test-e2e-config-neutral.sh` (master 1/16, branch 16/16) — field run on a blank Mendix 11.8 app under `mxcli run --local` (2026-11-20)
- new(doctor, bug ledger): **a guard for mxcli's silent MxBuild fallback**: when the project's Mendix version is not in `~/.mxcli/mxbuild/` (and no Studio Pro of that version exists), `mxcli docker build` / `docker check` fall back to the newest cached MxBuild and print its path but no version warning (`run --local` is exact-or-refuse); `bin/doctor.sh` now warns, naming the model version, the cached ones and `./mxcli setup mxbuild -p <app>.mpr` (`mxtk_mxbuild_cache_gap`, warning only, never changes a gate). Ledger entry `BUG-DRAFT-mxbuild-silent-version-fallback`; upstream issue not filed. — a four-app Mendix build (field run), independent review
- fix(gate-check): **a fresh clone no longer FAILs Stage 3 as "stale render"**: `blueprint.html` / `process-map.html` freshness used `[ render -ot source ]`, a nanosecond mtime compare, and a checkout stamps files in directory order (`process-map.html` lands before `process-model.json`), so an unedited project read stale on every clone (reproduced: 200-wireframe checkout, 10 of 10 clones). New `bin/lib/render-fresh.sh` keeps `-ot` as the first test (a pass never flips, no git call) and, only where `-ot` would FAIL, asks git once per render (one `log`, one `status`) whether the source was committed after the render's commit; measured 0.19 s for 200 sources all `-ot`-stale on a 300-commit history (a per-source git version cost 12.9 s). Limit: a depth-1 clone cannot see a genuinely stale render. Field run: a four-app Mendix build checked out on a second machine. — a four-app Mendix build (field run), independent review
- fix(test-stack-up, e2e-harness-base): **the app port follows the app's configured port, and a multi-app start is no longer pinned to 8080**: with no `APP_PORTS`, `test-stack-up.sh` puts `RUN_LOCAL_APP_PORT`, then `APP_PORT` from the `.docker/.env` beside the `.mpr` (`apps/<App>/.docker/.env`; the project-root one as fallback) first, ahead of the old fixed `8080 8081 8084` tail. A fresh `run --local` start for a model under `apps/` with no handshake and no configured port used to fall back to 8080 for every app (`--app-port 8080`); it now exits 4 naming `RUN_LOCAL_APP_PORT`. `e2e-harness-base.md` states that a PAD/Docker runtime listens on 8080 *inside* the container only. Fixtures T28–T32 added. Field run: a four-app Mendix build on 8081–8084 whose fresh start failed with every app pinned to 8080. — a four-app Mendix build (field run), independent review
Expand Down
8 changes: 4 additions & 4 deletions bin/install-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,10 @@ echo
echo "engine -> $DST ($copied copied, $skipped kept)"
echo
echo "Next, in this order:"
echo " 1. node $DST/project.config.js # print what it derived; check id and .mpr"
echo " 2. edit NAV / TARGETS / PAGE_WIREFRAME / exemplars / walkthroughs in project.config.js"
echo " 1. node $DST/project.config.js # what it derived (check id and .mpr), and each block still empty"
echo " 2. fill the FILL ME blocks in project.config.js: NAV, ROLES, WALKTHROUGH, PAGE_WIREFRAME, exemplars"
echo " 3. node $DST/report-normalize.js --selftest # must print 'all ok'"
echo " 4. node $DST/journey-runner.selftest.js"
echo
echo "Step 2 is not optional. Unedited, the audits walk another app's page names and report"
echo "fault on every page — which looks like a broken harness, not an unedited config."
echo "Step 2 is not optional. The blocks ship empty: until they are filled, monkey.js and"
echo "full-app-walkthrough.js stop with NOT CONFIGURED (exit 2) instead of measuring anything."
8 changes: 4 additions & 4 deletions project-tests/e2e/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -233,10 +233,10 @@ function resolveTarget() {
return t;
}

// The NAV project.config.template.js ships: the toolkit's sample app, not this project (#679).
// A project.config.js still carrying it navigates menu captions the app does not have, so a
// monkey run reaches nothing and measures nothing. Compared like TEMPLATE_PASSWORDS above;
// tests/wave2/test-monkey-template-nav.sh asserts this copy equals the template's.
// The NAV project.config.template.js used to ship: a sample app's, not this project's (#679).
// The template now ships NAV empty, but a project.config.js installed before that still carries
// this one and navigates menu captions the app does not have, so a monkey run reaches nothing
// and measures nothing. Kept as the signature that recognises it; compared like TEMPLATE_PASSWORDS.
const TEMPLATE_NAV = {
dashboard: ['Dashboard'],
equipment: ['Assets & Barcodes', 'Equipment'],
Expand Down
60 changes: 41 additions & 19 deletions project-tests/e2e/full-app-walkthrough.js
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,8 @@
// ENGINE / CONFIG LINE
// Nothing in this file knows a module, an entity, a page, a widget, a menu
// caption, a user or a URL. All of that is `WALKTHROUGH` and `ROLES` in
// project.config.js — see project.config.template.js, which ships a worked
// example and the mxcli command that produces the right answer for each field.
// project.config.js — see project.config.template.js, which ships the shape of
// each block (empty) and the mxcli command that produces the right answer for each field.
// ============================================================================

const H = require(__dirname + '/helpers.js');
Expand All @@ -75,7 +75,8 @@ if (!WT || !Array.isArray(WT.acts) || WT.acts.length === 0) {
'\n NOT CONFIGURED — project.config.js exports no `walkthrough` with acts.\n' +
' This instrument walks ONE business process across modules and roles; the process\n' +
' is a statement about your application, so it lives in project.config.js.\n' +
' Copy the WALKTHROUGH / ROLES blocks from project.config.template.js and fill them in.\n' +
' Fill the WALKTHROUGH and ROLES blocks in tests/e2e/project.config.js (the comment above\n' +
' WALKTHROUGH shows the shape; `node tests/e2e/project.config.js` lists what is still empty).\n' +
' Writing an empty findings.json here would report a clean walkthrough that never ran.\n');
process.exit(2);
}
Expand Down Expand Up @@ -350,6 +351,16 @@ async function runStep(page, meta, s) {
}
}

// An act's arrival when no handoffAssert brought it there: its leading `nav` steps. `checks`
// describe the page the act lands on, so they are read after these, never on the page the
// previous act left. After a non-empty handoffAssert the checks are read straight away.
function splitArrival(act) {
const steps = act.steps || [];
let i = 0;
while (i < steps.length && steps[i] && steps[i].do === 'nav') i++;
return { arrive: steps.slice(0, i), rest: steps.slice(i) };
}

function describe(s) {
switch (s.do) {
case 'nav': return `navigate to ${[s.group, s.item].filter(Boolean).join(' › ')}`;
Expand Down Expand Up @@ -511,6 +522,8 @@ async function switchTo(page, roleKey, handoffMeta) {

// A role change is a HANDOFF, and it gets its own page in the report so the
// seam is visible as a thing that was tested rather than buried inside an act.
// An act that arrives through a handoffAssert is ON its landing page once that assert ran.
const arrivedByHandoff = act.role !== currentRole && (act.handoffAssert || []).length > 0;
if (act.role !== currentRole) {
const from = currentRole ? (ROLES[currentRole].label || currentRole) : '(nobody)';
const to = (ROLES[act.role] && ROLES[act.role].label) || act.role;
Expand All @@ -532,22 +545,31 @@ async function switchTo(page, roleKey, handoffMeta) {

console.log(`\n── ${meta.label} ──`);

// Widget presence for the act's landing page, recorded as widgets[] so the
// report groups them the way it groups every other walkthrough's.
for (const name of act.checks || []) {
const visible = await H.visibleWithin(page.locator(`.mx-name-${name}`).first(), 2500);
meta.widgets.push({ name, status: visible ? 'ok' : 'missing',
detail: visible ? '' : `.mx-name-${name} not visible` });
if (visible) R.pass(`${meta.key}:widget:${name}`, `Widget ${name} present`);
else R.fail(`${meta.key}:widget:${name}`, `Widget ${name} present`, `.mx-name-${name} not visible`);
}

for (const s of act.steps || []) {
const ok = await runStep(page, meta, s);
// `fatal` is the config's way of saying "there is no point continuing" —
// a form that never opened, a record that was never created. Everything
// after it is reported NOT REACHED rather than left absent.
if (!ok && s.fatal) { stoppedAt = `${meta.label}: ${s.label || describe(s)}`; break; }
const runSteps = async (list) => {
for (const s of list) {
const ok = await runStep(page, meta, s);
// `fatal` is the config's way of saying "there is no point continuing" —
// a form that never opened, a record that was never created. Everything
// after it is reported NOT REACHED rather than left absent.
if (!ok && s.fatal) { stoppedAt = `${meta.label}: ${s.label || describe(s)}`; return false; }
}
return true;
};
// An act's landing page is where its handoffAssert left it. An act WITHOUT one (the first
// act, or one with no role change) arrives by its LEADING nav steps, so those run before its
// widget checks are read. A handoff act's steps, a leading nav included, run after them.
const { arrive, rest } = arrivedByHandoff ? { arrive: [], rest: act.steps || [] } : splitArrival(act);
if (await runSteps(arrive)) {
// Widget presence for the act's landing page, recorded as widgets[] so the
// report groups them the way it groups every other walkthrough's.
for (const name of act.checks || []) {
const visible = await H.visibleWithin(page.locator(`.mx-name-${name}`).first(), 2500);
meta.widgets.push({ name, status: visible ? 'ok' : 'missing',
detail: visible ? '' : `.mx-name-${name} not visible` });
if (visible) R.pass(`${meta.key}:widget:${name}`, `Widget ${name} present`);
else R.fail(`${meta.key}:widget:${name}`, `Widget ${name} present`, `.mx-name-${name} not visible`);
}
await runSteps(rest);
}
if (!meta.screenshot) await shot(page, meta, `act-${String(i + 1).padStart(2, '0')}-${meta.key}`);
if (stoppedAt) break;
Expand Down
17 changes: 12 additions & 5 deletions project-tests/e2e/monkey.js
Original file line number Diff line number Diff line change
Expand Up @@ -290,12 +290,13 @@ async function round(page, target, bucket) {
return traceRan;
}

// Exit codes (#679): 1 = a crash-class finding; 2 = instrument fault, NOT measured (the NAV is
// the template's sample, or 0 of N targets were reached); 3 = unknown --target; 0 otherwise.
// Exit codes (#679): 1 = a crash-class finding; 2 = instrument fault, NOT measured (NAV is empty
// or the old template sample, or 0 of N targets were reached — N = 0 included: a fresh scaffold's
// NAV is empty, and "reached 0 of 0" fuzzed nothing); 3 = unknown --target; 0 otherwise.
// "0 of 8 reached" and "8 of 8 reached, no crash" used to both exit 0.
function exitCode({ crashes, targets, unreached }) {
if (crashes > 0) return 1;
if (targets > 0 && unreached >= targets) return 2;
if (targets === 0 || unreached >= targets) return 2;
return 0;
}

Expand All @@ -306,8 +307,14 @@ module.exports = { leftApp, SIGN_OUT, SIGN_OUT_DEFAULT, signOutPattern, isSignOu
if (require.main === module) (async () => {
console.log(`MONKEY_SEED=${SEED} rounds=${ROUNDS} (re-run exactly: --seed ${SEED})`);

// Before the browser (#679): a NAV still equal to the template's sample would land on menu
// captions this app does not have — a run that measures nothing, and costs a login.
// Before the browser (#679): an empty NAV (a fresh scaffold's) or one still equal to the old
// template sample lands on nothing this app has — a run that measures nothing, and costs a login.
if (!NAV || !Object.keys(NAV).length) {
console.log(' SETUP NOT CONFIGURED: NAV in tests/e2e/project.config.js is empty — monkey has no menu item ' +
'to fuzz. Fill it from `mxcli -p <mpr> -c "describe navigation"` (key: [\'Group\', \'Item\'] or [\'Item\']); ' +
'an app with no menu to fuzz runs verify-module.sh with --skip-monkey instead. Monkey NOT run: not measured.');
process.exit(2);
}
if (navIsTemplate()) {
console.log(' SETUP NAV is the unedited sample from project.config.template.js — none of its menu ' +
'captions are this app\'s. Fill NAV in tests/e2e/project.config.js from ' +
Expand Down
4 changes: 3 additions & 1 deletion project-tests/e2e/page-audit-rules.js
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,9 @@

const PROJ = require('./project.config');
const MPR = PROJ.mprName;
const EX = PROJ.exemplars;
// A fresh scaffold leaves them null: the advice then says what to set, never "null's ...".
const EX = Object.fromEntries(['listPage', 'overviewPage', 'detailPage'].map((k) =>
[k, (PROJ.exemplars || {})[k] || `<no exemplar: set exemplars.${k} in tests/e2e/project.config.js>`]));

// ── MDL parsing ──────────────────────────────────────────────────────────────
// A widget declaration is `<type> <name> ( props ) { body }`, where props may span
Expand Down
Loading
Loading