Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,12 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m
`docs/RELEASE-PROCESS.md` Β§8.

## Unreleased
- process(tests): **golden mxcli v0.32 module tables for the three module parsers**: v0.32 adds Version and Type columns after Source and a "(N modules)" / "(N pages)" trailer, and every capture in `fixtures/show-modules/` and `fixtures/rehost-list-modules/` predated both. New verbatim `show modules`, `list modules` and `SHOW PAGES IN` captures, and `test-show-modules-v032.sh` asserts `own-modules.awk` (own and marketplace), `page-scope.sh` and `review-module.sh`'s module-exists grep on them. All three already read v0.32 correctly; this pins it β€” real-mxcli field run (mxcli v0.32.0-374, blank Mendix 11.8.0 app + a Sales module)
- fix(describe-baseline): **an untagged mxcli build still stamps its version**: `mxcli_version` kept only a `vX.Y.Z`, and a `make build` on a shallow clone prints `mxcli version 591bcd7 (…)`, so no `-- baseline-mxcli:` line was written and a later mxcli upgrade read as DRIFT instead of RESTAMP. The commit word is now recorded when there is no tag. Fixture `test-describe-baseline.sh` on verbatim untagged and v0.32.0-374 `--version` captures β€” real-mxcli field run (mxcli source build v0.32.0-374)
- fix(init): **the closing leak-guard line names the denylist init actually wrote**: with `MXTK_LEAKGUARD_DENYFILE` set, init registered the project name in that file and then said "no `<toolkit>/.leakguard-deny` found on this machine. Create it", because the nudge tested the default path. Fixture `test-init-leakguard-nudge.sh` β€” real-mxcli field run (scaffold of a blank Mendix 11.8.0 app)
- fix(exec): **a rolled-back exec no longer leaves the new module's source folder behind**: the snapshot covers the `.mpr` and `mprcontents/` only, and the gate's own `mx check` generates `themesource/<module>/` (five files plus an empty `public/` on MxBuild 11.8.0) for a module the script created, so after a failed gate restored the model the folder stayed, untracked, for a module the model no longer had β€” one `git add -A` from being committed. After a successful auto-restore, `exec.sh` now moves the source folders of the modules this run created, read from mxcli's own exec output (`Created module: X`; an existing module prints `Module 'X' already exists` and is never touched) β€” `themesource/<x>`, `javasource/<x>`, `javascriptsource/<X>` or `<x>`, mxcli's own naming β€” into the snapshot (`.mpr-snapshots/<snap>/stray-module-dirs/`) and prints where, when they did not exist at snapshot time and hold no tracked file. Any other folder, including one another session creates during the exec window, is left alone (single tree and `app/`). Fixture `test-exec-rollback-module-dirs.sh` on the verbatim generated files β€” real-mxcli field run (mxcli v0.32.0-374, MxBuild 11.8.0, blank Mendix 11.8.0 app), independent review of #823
- fix(exec, _common): **the mxbuild auto-download works when `MXCLI_HOME` points elsewhere**: the toolkit looks for the cache at `${MXCLI_HOME:-~/.mxcli}`, but mxcli itself always caches under `~/.mxcli` (`os.UserHomeDir`), so `exec.sh` downloaded, printed `MxBuild ready: …`, then reported "no runnable mxbuild was found afterwards" and refused the exec. `mxtk_ensure_mxbuild` now takes the path mxcli reports when re-discovery finds nothing (same version only) and says that `MXCLI_HOME` is not mxcli's. Fixture `test-mxbuild-setup-home.sh` on a verbatim `setup mxbuild` capture β€” real-mxcli field run (mxcli v0.32.0-374, blank Mendix 11.8.0 app)
- fix(exec): **a refused pre-exec check keeps the cause of a v0.32 reference error in the BUILD-LOG row**: mxcli v0.32 prints a reference error as two lines under `Reference errors:` (`statement 3: microflow 'X' has reference errors:` then ` - entity not found: …`) and the 🚫 row kept only the header, which names no cause. The cause line is now appended. Fixture `test-exec-check-refusal.sh` F, on a verbatim v0.32.0 capture β€” real-mxcli field run (mxcli v0.32.0-374, blank Mendix 11.8.0 app)
- new(doctor, bug ledger): **a guard for mxcli's silent MxBuild fallback**: when the project's Mendix version is not in `~/.mxcli/mxbuild/` (and no Studio Pro of that version exists), `mxcli docker build` / `docker check` fall back to the newest cached MxBuild and print its path but no version warning (`run --local` is exact-or-refuse); `bin/doctor.sh` now warns, naming the model version, the cached ones and `./mxcli setup mxbuild -p <app>.mpr` (`mxtk_mxbuild_cache_gap`, warning only, never changes a gate). Ledger entry `BUG-DRAFT-mxbuild-silent-version-fallback`; upstream issue not filed. β€” a four-app Mendix build (field run), independent review
- fix(gate-check): **a fresh clone no longer FAILs Stage 3 as "stale render"**: `blueprint.html` / `process-map.html` freshness used `[ render -ot source ]`, a nanosecond mtime compare, and a checkout stamps files in directory order (`process-map.html` lands before `process-model.json`), so an unedited project read stale on every clone (reproduced: 200-wireframe checkout, 10 of 10 clones). New `bin/lib/render-fresh.sh` keeps `-ot` as the first test (a pass never flips, no git call) and, only where `-ot` would FAIL, asks git once per render (one `log`, one `status`) whether the source was committed after the render's commit; measured 0.19 s for 200 sources all `-ot`-stale on a 300-commit history (a per-source git version cost 12.9 s). Limit: a depth-1 clone cannot see a genuinely stale render. Field run: a four-app Mendix build checked out on a second machine. β€” a four-app Mendix build (field run), independent review
- fix(test-stack-up, e2e-harness-base): **the app port follows the app's configured port, and a multi-app start is no longer pinned to 8080**: with no `APP_PORTS`, `test-stack-up.sh` puts `RUN_LOCAL_APP_PORT`, then `APP_PORT` from the `.docker/.env` beside the `.mpr` (`apps/<App>/.docker/.env`; the project-root one as fallback) first, ahead of the old fixed `8080 8081 8084` tail. A fresh `run --local` start for a model under `apps/` with no handshake and no configured port used to fall back to 8080 for every app (`--app-port 8080`); it now exits 4 naming `RUN_LOCAL_APP_PORT`. `e2e-harness-base.md` states that a PAD/Docker runtime listens on 8080 *inside* the container only. Fixtures T28–T32 added. Field run: a four-app Mendix build on 8081–8084 whose fresh start failed with every app pinned to 8080. β€” a four-app Mendix build (field run), independent review
Expand Down
8 changes: 5 additions & 3 deletions bin/init-project.sh
Original file line number Diff line number Diff line change
Expand Up @@ -791,12 +791,14 @@ echo "Machine preflight (bin/doctor.sh):"
# three lines, never a prompt, never a failure: the denylist protects the TOOLKIT repo, and
# a project scaffold must not block on it (kickoff is just the cheapest moment to remember).
echo ""
if [ -f "$TOOLKIT_ROOT/.leakguard-deny" ]; then
# $DENYFILE, not $TOOLKIT_ROOT/.leakguard-deny: with MXTK_LEAKGUARD_DENYFILE set, the registration above
# wrote that file and this line then said "no …/.leakguard-deny found" (real-mxcli scaffold, 2026-11-20).
if [ -f "$DENYFILE" ]; then
echo "Leak guard: if this project is for a new client, add their name (and codename) to"
echo " $TOOLKIT_ROOT/.leakguard-deny β€” one regex per line β€” and update the toolkit repo's"
echo " $DENYFILE β€” one regex per line β€” and update the toolkit repo's"
echo " LEAKGUARD_DENY GitHub Actions secret to match. 30 seconds now beats a scrub later."
else
echo "Leak guard: no $TOOLKIT_ROOT/.leakguard-deny found on this machine. Create it (one"
echo "Leak guard: no $DENYFILE found on this machine. Create it (one"
echo " client-name regex per line, gitignored) so check-no-client-data.sh can catch client"
echo " names before they reach the public toolkit repo β€” see that script's header."
fi
Expand Down
26 changes: 21 additions & 5 deletions project-bin/_common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1771,15 +1771,31 @@ mxtk_ensure_mxbuild() {
fi
echo "β†’ mxbuild not found β€” downloading the toolchain for this model's Mendix version" >&2
echo " ($mxcli setup mxbuild β€” the same download bin/doctor.sh --install runs; cached under ~/.mxcli/mxbuild/)" >&2
if (cd "$root" && "$mxcli" setup mxbuild -p "$(basename "$mpr")" >&2); then
# mxcli always caches under its own home (~/.mxcli, os.UserHomeDir) and ignores MXCLI_HOME, which only
# the toolkit reads. So the path mxcli prints last ("MxBuild ready: <path>", mxcli v0.32 setup.go; capture
# in tests/wave2/fixtures/mxbuild-setup/) is taken as the answer when re-discovery under MXCLI_HOME finds
# nothing: before, a "successful" download was followed by "no runnable mxbuild found" and a refused exec.
local _sout _src _ready _vdir
_sout="$(mktemp "${TMPDIR:-/tmp}/mxbuild-setup.XXXXXX")"
(cd "$root" && "$mxcli" setup mxbuild -p "$(basename "$mpr")") 2>&1 | tee "$_sout" >&2; _src=${PIPESTATUS[0]}
_ready="$(tr -d '\r' < "$_sout" | sed -n 's/^MxBuild ready: //p' | tail -1)"; rm -f "$_sout"
if [ "$_src" -eq 0 ]; then
found="$(find_mxbuild 2>/dev/null || true)"
if [ -n "$found" ] && [ -x "$found" ]; then
if [ -n "$ver" ] && ! mxtk_mxbuild_for_version "$ver" >/dev/null 2>&1; then
echo " ⚠ download finished but there is still no mxbuild for $ver β€” using $found; the gate will report the refusal as unverified." >&2
if [ -n "$found" ] && [ -x "$found" ] && { [ -z "$ver" ] || mxtk_mxbuild_for_version "$ver" >/dev/null 2>&1; }; then
printf '%s\n' "$found"; return 0
fi
if [ -n "$_ready" ] && [ -x "$_ready" ]; then
_vdir="${_ready%/modeler/mxbuild*}"
if [ -z "$ver" ] || mxtk_version_in_name "$_vdir" "$ver"; then
[ -n "${MXCLI_HOME:-}" ] && echo " note: mxcli caches under ~/.mxcli whatever MXCLI_HOME says (MXCLI_HOME=$MXCLI_HOME); using the mxbuild it reported: $_ready" >&2
printf '%s\n' "$_ready"; return 0
fi
fi
if [ -n "$found" ] && [ -x "$found" ]; then
echo " ⚠ download finished but there is still no mxbuild for $ver β€” using $found; the gate will report the refusal as unverified." >&2
printf '%s\n' "$found"; return 0
fi
echo " download reported success but no runnable mxbuild was found afterwards (looked under ${MXCLI_HOME:-$HOME/.mxcli}/mxbuild/)." >&2
echo " download reported success but no runnable mxbuild was found afterwards (looked under ${MXCLI_HOME:-$HOME/.mxcli}/mxbuild/${_ready:+ and at $_ready, the path mxcli reported})." >&2
else
echo " '$mxcli setup mxbuild' failed β€” a blocked network or proxy is the usual cause (the download comes from the Mendix CDN)." >&2
fi
Expand Down
13 changes: 11 additions & 2 deletions project-bin/describe-baseline.sh
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,16 @@ describe_hash() { # $1 = KIND, $2 = Doc
echo "sha256=$h"
}
# "v0.26.0" from `mxcli --version` (captured: "mxcli version v0.26.0 (2026-10-11T18:37:07Z)").
mxcli_version() { "$MXCLI" --version 2>/dev/null | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1; }
# A build with no release tag in reach (a `make build` on a shallow clone) prints only a commit:
# "mxcli version 591bcd7 (2026-11-20T18:49:18Z)" (captured, tests/wave2/fixtures/restatement-guard/).
# That word is recorded as it is; before, nothing was stamped and an upgrade read as DRIFT.
mxcli_version() {
local line v
line="$("$MXCLI" --version 2>/dev/null | grep -m1 '^mxcli version ')"
v="$(printf '%s\n' "$line" | grep -oE 'v[0-9]+\.[0-9]+\.[0-9]+' | head -1)"
[ -n "$v" ] || v="$(printf '%s\n' "$line" | awk '{ print $3 }')"
printf '%s\n' "$v"
}

if [ "$CMD" = live ]; then
need_mxcli || exit 3
Expand Down Expand Up @@ -165,7 +174,7 @@ fi
need_mxcli || exit 3
KINDS="$(restated)"
KEY="$(script_key "$SCRIPT")"
WAS_V="$(sed -nE 's/^[[:space:]]*--[[:space:]]*baseline-mxcli:[[:space:]]*(v[0-9.]+).*$/\1/p' "$SCRIPT" | head -1)"
WAS_V="$(sed -nE 's/^[[:space:]]*--[[:space:]]*baseline-mxcli:[[:space:]]*([^[:space:]]+).*$/\1/p' "$SCRIPT" | head -1)"
NOW_V=""; [ -n "$WAS_V" ] && NOW_V="$(mxcli_version)"
RC=0
while read -r doc want; do
Expand Down
74 changes: 71 additions & 3 deletions project-bin/exec.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1066,7 +1066,8 @@ if [ "$LOCK_TAKEN" != 1 ]; then
echo " β†’ Wait for the other exec to finish, then re-run. Nothing was written."
exit 1
fi
trap 'mxtk_resume_watchers; exec_lock_release; [ "$_PROFILING" != 1 ] || _prof_report' EXIT
EXEC_OUT="" # never an inherited value: the EXIT trap removes it
trap 'mxtk_resume_watchers; exec_lock_release; [ -z "${EXEC_OUT:-}" ] || rm -f -- "$EXEC_OUT"; [ "$_PROFILING" != 1 ] || _prof_report' EXIT
mxtk_resume_leftover "$PROJECT_ROOT" # a watcher an earlier, SIGKILLed exec.sh left stopped

# 3. A raw `mxcli exec` elsewhere on the machine is NOT this project's business. The old guard
Expand Down Expand Up @@ -1589,11 +1590,15 @@ step_done guards
# Shapes are real mxcli v0.26.0 captures (tests/wave2/fixtures/exec-check-refusal/): an error
# violation is "βœ— <message> [RULE]" after any warnings, in ANSI colour even when piped; a parse
# failure is "- line N:M ..." under "Syntax errors found:". Rule id first, so truncation keeps it.
# v0.32 reference errors take two lines under "Reference errors:", a header ending in ":" and the
# cause as " - ..." (fixtures/exec-check-refusal/reference-v0.32.0.out); both go in the row,
# because the header alone ("microflow 'X' has reference errors:") names no cause.
check_refusal_reason() { # $1 = captured check output
_esc="$(printf '\033')"
tr -d '\r' < "$1" 2>/dev/null | sed "s/${_esc}\[[0-9;]*m//g" | awk '
/^(Syntax|Reference) errors( found)?:$/ { hdr = 1; next }
hdr && h == "" && NF { h = $0 }
hd == 1 { hd = 2; if ($0 ~ /^ *- /) { d = $0; sub(/^ *- */, "", d); h = h " " d } }
hdr && h == "" && NF { h = $0; if (h ~ /:$/) hd = 1 }
e == "" && /^ *βœ— / && !/ found$/ { e = $0 }
f == "" && /[Ee]rror/ && !/^WARNING: / { f = $0 }
END { r = (e != "") ? e : ((h != "") ? h : f); print r }' \
Expand Down Expand Up @@ -1737,10 +1742,56 @@ if [ ! -x "$MXBUILD" ] || [ ! -x "$JAVA_EXE" ]; then
fi
fi

# The snapshot holds the .mpr and mprcontents/ only. A module the script creates gets source folders
# beside the .mpr that the snapshot does not cover: the gate's own `mx check` generates
# themesource/<module>/ (settings.json, web/*, native/*, public/) for every module that lacks one
# (mxcli v0.32 + MxBuild 11.8.0, captured in fixtures/exec-rollback-module-dirs/). After a rollback
# those folders stayed, untracked, for a module the model no longer has. So after a successful restore,
# the folders of the modules THIS RUN created (`Created module: X`: themesource/<x>, javasource/<x>,
# javascriptsource/<X> or <x>, the names mxcli's removeModuleSourceDirs uses) are moved into the
# snapshot (stray-module-dirs/), when they were not there at snapshot time and hold no tracked file.
# "Creates" is read from mxcli's own exec output (script_created_modules), not from the script text.
# Nothing else is touched: another session's or the user's new folder in the same window stays put.
module_dirs() {
local d
for d in "$MODEL_DIR"/themesource/*/ "$MODEL_DIR"/javasource/*/ "$MODEL_DIR"/javascriptsource/*/; do
[ -d "$d" ] && printf '%s\n' "${d%/}"
done
return 0
}
# Modules mxcli actually created in this run, from its exec stdout (captured verbatim, mxcli v0.32:
# "Created module: X"; an existing module prints "Module 'X' already exists", `if not exists` on an
# existing one "module X already exists, skipped (if not exists)", a module role "Created module role:
# M.R" β€” none of those match). Not the script text: `create or modify module Sales` on an existing
# module named a folder the gate regenerates, and the move then claimed the script created it.
# No captured exec output (a --patch run, or no write) = no names = nothing is moved.
script_created_modules() {
[ -n "${EXEC_OUT:-}" ] && [ -f "$EXEC_OUT" ] || return 0
tr -d '\r' < "$EXEC_OUT" | sed -n 's/^Created module: \([A-Za-z_][A-Za-z0-9_]*\)$/\1/p' || true
}
move_stray_module_dirs() {
local m lm d rel dest
[ -n "${NEWEST_SNAP:-}" ] || return 0
for m in $(script_created_modules); do
lm="$(printf '%s' "$m" | tr '[:upper:]' '[:lower:]')"
for d in "$MODEL_DIR/themesource/$lm" "$MODEL_DIR/javasource/$lm" \
"$MODEL_DIR/javascriptsource/$m" "$MODEL_DIR/javascriptsource/$lm"; do
[ -d "$d" ] && [ ! -L "$d" ] || continue
printf '%s\n' "$MODDIRS_BEFORE" | grep -qxF -- "$d" && continue
rel="${d#"$PROJECT_ROOT"/}"
[ -n "$(git -C "$PROJECT_ROOT" ls-files -- "$rel" 2>/dev/null | head -1)" ] && continue
dest="${NEWEST_SNAP%/}/stray-module-dirs/$rel"
mkdir -p "$(dirname "$dest")" && mv -- "$d" "$dest" \
&& echo " β†’ Moved $rel/ (source folder of module $m, which this run created: mxcli printed \"Created module: $m\") to $dest"
done
done
}
step_done check
echo "β†’ Snapshotting model..."
./bin/snapshot-mpr.sh
step_done snapshot
# Module source folders as they stand before the write (see module_dirs / move_stray_module_dirs above).
MODDIRS_BEFORE="$(module_dirs)"
# The snapshot just taken. A --patch run hands it to the script as MXPATCH_BACKUP_DIR, so the
# .bak that bin/mxpatch.py keeps of each unit it rewrites lands beside the snapshot, OUTSIDE
# the model tree, and is pruned with it. Until 2026-10-15 it was <unit>.mxunit.bak inside
Expand Down Expand Up @@ -1772,6 +1823,7 @@ restore_snapshot() {
elif R_OUT=$(MXTK_RESTORE_SAME_LEDGER=1 bash "$RESTORER" "$NEWEST_SNAP" 2>&1); then
RESTORED=1
printf '%s\n' "$R_OUT" | sed 's/^/ β†’ /'
[ -n "${MODDIRS_BEFORE+x}" ] && move_stray_module_dirs
echo " β†’ Auto-restored from: $NEWEST_SNAP"
else
printf '%s\n' "$R_OUT" | sed 's/^/ /'
Expand Down Expand Up @@ -2324,7 +2376,23 @@ EOF_HV
rm -f "$HV_BASE" "$HV_POST"
fi
else
"${PMXCLI:-./mxcli}" exec "$SCRIPT" -p "$MPR" || EXEC_STATUS=$?
# stdout is kept (EXEC_OUT) for move_stray_module_dirs, which reads its "Created module:" lines,
# but only when the script can create a module: mktemp + tee are two process starts, a per-row cost
# every other exec must not pay (test-exec-process-budget.sh). The test reads the script with the
# `read` builtin and a bash pattern (case-insensitive by bracket classes; bash 3.2-safe, no fork).
# It may over-match (a comment, `create module role`): that only captures, the moves still need
# mxcli's own "Created module:" line. Otherwise exec runs exactly as before the capture existed.
# Neither the capture nor tee may stop exec.sh under set -e once mxcli has written: a failed
# mktemp leaves EXEC_OUT empty (tee then only passes stdout through, and nothing is moved later),
# and a failed tee is swallowed. EXEC_STATUS is mxcli's own exit code either way.
_SCRIPT_TXT=""; IFS= read -r -d '' _SCRIPT_TXT < "$SCRIPT" || true
if [[ "$_SCRIPT_TXT" == *[Cc][Rr][Ee][Aa][Tt][Ee][[:space:]]*[Mm][Oo][Dd][Uu][Ll][Ee]* ]]; then
EXEC_OUT="$(mktemp "${TMPDIR:-/tmp}/exec-out.XXXXXX" 2>/dev/null)" || EXEC_OUT=""
{ "${PMXCLI:-./mxcli}" exec "$SCRIPT" -p "$MPR" | tee ${EXEC_OUT:+"$EXEC_OUT"}; EXEC_STATUS=${PIPESTATUS[0]}; } || true
else
"${PMXCLI:-./mxcli}" exec "$SCRIPT" -p "$MPR" || EXEC_STATUS=$?
fi
unset _SCRIPT_TXT
fi

if [ "$EXEC_STATUS" -ne 0 ]; then
Expand Down
27 changes: 27 additions & 0 deletions tests/wave2/fixtures/exec-check-refusal/reference-v0.32.0.mdl
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
mdl 1;
create module Broken2;

create persistent entity Broken2.Thing (
Name: string(100)
);

create microflow Broken2.ACT_Thing ()
returns boolean as $ok
begin
$T = create Broken2.Thing (Nmae = 'x');
$L = create Broken2.Missing (Name = 'y');
declare $ok boolean = true;
return $ok;
end;

create page Broken2.Thing_Edit
(
title: 'Edit',
layout: Atlas_Core.NoSuchLayout,
params: { $Thing: Broken2.Thing }
)
{
dataview dv1 (datasource: $Thing) {
textbox tb1 (label: 'Name', attribute: Nmae)
}
};
Loading
Loading