Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m
`docs/RELEASE-PROCESS.md` §8.

## Unreleased
- fix(file-bugs, init-project): **bug-report redaction no longer garbles ordinary English**: `init-project.sh .` took the name `.` and wrote `\b.\b` to the toolkit clone's denylist, which file-bugs.sh reads, so every space and hyphen became `<redacted>` and the scan refused the report as a deny-list match; separately, an attribute named `Round` turned "Workaround" into `WorkaAttr390` and one named `Used` redacted the word "used". `init-project.sh` now resolves the folder before taking its name, writes it regex-escaped (`\bmy\.app\b`), dedupes on that exact line and refuses a name under 3 characters. `file-bugs.py` ignores, with a warning, any deny pattern that matches empty text, a space or `a-b c` (this repairs clones that already hold the line), and matches attribute names only where an identifier segment starts (a short or common-word one in its model spelling only); `ACT_Round_Save`, `$w/Used`, `Kundennummer2` are still redacted; known limit: a distinctive attribute inside a lowercase compound (`prekundennummer`) and an all-caps weak one (`USED`) are no longer redacted. A probe-matching deny line that still holds a name (`Acme|Initrode|`, `(Acme)?`, `Öko|` — any 3-letter run, non-ASCII letters included) is refused, not dropped: posting, `--links` and `--json` stop until it is fixed (fail closed). Fixture `test-file-bugs.sh` S — thanarat (#759)
- new(doctor, bug ledger): **a guard for mxcli's silent MxBuild fallback**: when the project's Mendix version is not in `~/.mxcli/mxbuild/` (and no Studio Pro of that version exists), `mxcli docker build` / `docker check` fall back to the newest cached MxBuild and print its path but no version warning (`run --local` is exact-or-refuse); `bin/doctor.sh` now warns, naming the model version, the cached ones and `./mxcli setup mxbuild -p <app>.mpr` (`mxtk_mxbuild_cache_gap`, warning only, never changes a gate). Ledger entry `BUG-DRAFT-mxbuild-silent-version-fallback`; upstream issue not filed. — a four-app Mendix build (field run), independent review
- fix(gate-check): **a fresh clone no longer FAILs Stage 3 as "stale render"**: `blueprint.html` / `process-map.html` freshness used `[ render -ot source ]`, a nanosecond mtime compare, and a checkout stamps files in directory order (`process-map.html` lands before `process-model.json`), so an unedited project read stale on every clone (reproduced: 200-wireframe checkout, 10 of 10 clones). New `bin/lib/render-fresh.sh` keeps `-ot` as the first test (a pass never flips, no git call) and, only where `-ot` would FAIL, asks git once per render (one `log`, one `status`) whether the source was committed after the render's commit; measured 0.19 s for 200 sources all `-ot`-stale on a 300-commit history (a per-source git version cost 12.9 s). Limit: a depth-1 clone cannot see a genuinely stale render. Field run: a four-app Mendix build checked out on a second machine. — a four-app Mendix build (field run), independent review
- fix(test-stack-up, e2e-harness-base): **the app port follows the app's configured port, and a multi-app start is no longer pinned to 8080**: with no `APP_PORTS`, `test-stack-up.sh` puts `RUN_LOCAL_APP_PORT`, then `APP_PORT` from the `.docker/.env` beside the `.mpr` (`apps/<App>/.docker/.env`; the project-root one as fallback) first, ahead of the old fixed `8080 8081 8084` tail. A fresh `run --local` start for a model under `apps/` with no handshake and no configured port used to fall back to 8080 for every app (`--app-port 8080`); it now exits 4 naming `RUN_LOCAL_APP_PORT`. `e2e-harness-base.md` states that a PAD/Docker runtime listens on 8080 *inside* the container only. Fixtures T28–T32 added. Field run: a four-app Mendix build on 8081–8084 whose fresh start failed with every app pinned to 8080. — a four-app Mendix build (field run), independent review
Expand Down
20 changes: 16 additions & 4 deletions bin/init-project.sh
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,10 @@ fi
# second run the variable was simply unset. The leak-guard registration further
# down reads it, and an unset name there writes a bare `\b\b` to the denylist:
# a pattern that matches every file. Assign it once, unconditionally.
PROJECT_NAME="$(basename "$PROJECT_DIR")"
#
# Resolved first (#759): `init-project.sh .` took basename "." as the name, and the denylist
# line `\b.\b` below then matched every space and hyphen in every bug report and every file.
PROJECT_NAME="$(basename "$(cd "$PROJECT_DIR" 2>/dev/null && pwd || printf '%s' "$PROJECT_DIR")")"

# Stamp the toolkit sha this scaffold was built from.
#
Expand Down Expand Up @@ -634,11 +637,20 @@ if [ -w "$TOOLKIT_ROOT" ]; then
# matching the name with `[^A-Za-z0-9]` boundaries — never fires, because the
# stored form is `\bName\b` and the character immediately before the name is
# the `b` of `\b`, which is alphanumeric. Every re-run appended a duplicate.
# -F also removes any need to escape names containing `.`, `+` or `-`.
if [ ! -f "$DENYFILE" ] || ! grep -qiF -- "$PROJECT_NAME" "$DENYFILE" 2>/dev/null; then
#
# The name is written regex-escaped (#759): a `.` or `+` in it is a metacharacter in the
# extended-regex list, and `my.app` would also match `my-app` and `myXapp`. The dedupe looks
# for that exact line. A name under 3 characters is not registered: as a pattern it matches
# ordinary words everywhere.
DENY_LINE="\\b$(printf '%s' "$PROJECT_NAME" | sed 's/[][\.*^$+?(){}|]/\\&/g')\\b"
if [ "${#PROJECT_NAME}" -lt 3 ]; then
echo ""
echo "NOTE: \"$PROJECT_NAME\" is too short to register with the toolkit leak guard (it would match"
echo " ordinary words). Add a longer pattern for this project to $DENYFILE by hand."
elif [ ! -f "$DENYFILE" ] || ! grep -qixF -- "$DENY_LINE" "$DENYFILE" 2>/dev/null; then
{
[ -f "$DENYFILE" ] || printf '# Leak-guard denylist — gitignored. See bin/check-no-client-data.sh header.\n'
printf '\\b%s\\b\n' "$PROJECT_NAME"
printf '%s\n' "$DENY_LINE"
} >> "$DENYFILE" 2>/dev/null && {
echo ""
echo "Registered \"$PROJECT_NAME\" with the toolkit leak guard ($DENYFILE)."
Expand Down
46 changes: 42 additions & 4 deletions bin/lib/file-bugs.py
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,29 @@ def name_pat(name, sub=None):
return r"(?:(?<![A-Za-z0-9])(?i:" + core + r")|(?<=[a-z0-9])" + cap + r")(?![a-z0-9])"


def attr_pat(name):
"""Attribute names (#759). They come from the model, and many are plain English (Round, Used), so
the rule is tighter than name_pat's. A distinctive one still matches in any case and with anything
after it (Kundennummer2, kundennummerold, KundennummerCheck), but only where an identifier segment
starts: the attribute Round no longer turns "Workaround" into "WorkaAttr1", and ACT_Round_Save,
$w/Round and myRound are still caught (so is a whole word "round": case-insensitive, as before). A weak one (short or a common word: Used) matches as a whole
word or segment in its model spelling only, so "used" in prose survives and $w/Used does not.
Known limits, accepted for readable reports (were redacted before #759): a distinctive attribute
inside a lowercase compound (`prekundennummer`) and a weak one in all caps (`USED`) are left in
clear. Module and entity names still go through name_pat; hosts, emails and paths are unaffected."""
n = norm(name)
core = LB.join(re.escape(c) for c in n)
seg = r"(?<=[a-z0-9])" + LB.join(re.escape(c) for c in n[:1].upper() + n[1:])
if not is_weak(n):
return r"(?:(?<![A-Za-z0-9])(?i:" + core + r")|" + seg + r")"
return r"(?:(?<![A-Za-z0-9])" + core + r"|" + seg + r")(?![a-z0-9])"


# A deny pattern that matches this is not a name: it matches ordinary text (#759). `\b.\b`, written by
# `init-project.sh .` before the fix, turned every space and hyphen into <redacted>.
DENY_PROBES = ("", " ", "a-b c")


def _toolkit_names():
here = os.path.dirname(os.path.abspath(__file__))
root = os.path.normpath(os.path.join(here, "..", ".."))
Expand Down Expand Up @@ -300,10 +323,23 @@ def _load_deny(self):

def add(pat, src):
try:
pats.append(re.compile(pat, re.I))
return
rx = re.compile(pat, re.I)
except re.error:
pass
rx = None
if rx is not None:
if not any(rx.search(p) for p in DENY_PROBES):
pats.append(rx)
elif not re.search(r"[^\W_]{3,}", pat):
# holds no name at all (`\b.\b`, `\b\b`): dropping it loses nothing. Unicode-aware, so
# `Öko|` counts as a name (review of #815)
self.warnings.append("deny pattern `%s` matches ordinary text (empty, a space, a hyphen or a "
"one-letter word), so it is ignored. Remove that line from %s"
% (pat[:60], src))
else:
# holds a name but also matches everything (`Acme|Initrode|`, `(Acme)?`): fail closed,
# since dropping it would post those names in clear (review of #815)
self.bad_deny.append("%s: %r matches ordinary text" % (src, pat[:60]))
return
# invalid as a whole: nothing is kept, posting/--links/--json are refused until it is fixed
self.bad_deny.append("%s: %r" % (src, pat[:60]))

Expand Down Expand Up @@ -441,9 +477,11 @@ def _compile(self):
self.matchers = []
if self.app:
self.matchers.append((self.app, "App1", name_pat(self.app)))
for table in (self.mods, self.ents, self.docs, self.attrs):
for table in (self.mods, self.ents, self.docs):
for n, tok in table.items():
self.matchers.append((n, tok, name_pat(n)))
for n, tok in self.attrs.items():
self.matchers.append((n, tok, attr_pat(n)))
for u in self.users:
self.matchers.append((u, "<user>", name_pat(u)))
for h in self.hosts:
Expand Down
35 changes: 35 additions & 0 deletions tests/wave2/test-file-bugs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@
# repo has it or the user may create it, exact-title duplicate recorded not reposted, no gh or not
# signed in -> links and exit 0, no deny list refused; --add --log redaction and drop; the
# permissions deny --file/--all but not --auto
# S #759 a `\b.\b` deny line (from `init-project.sh .`) is dropped with a warning, but one that
# also holds a name (`Acmecorp|Initrode|`) is refused (fail closed); attribute names
# that are English words (Round, Used) leave prose alone and are still caught in code
set -uo pipefail
TOOLKIT="$(cd "$(dirname "$0")/../.." && pwd)"
SUBJ="${1:-$TOOLKIT/bin/file-bugs.sh}"
Expand Down Expand Up @@ -681,6 +684,38 @@ chk "--import default cap is 20 bugs" '[ $RC -ne 0 ] && echo "$OUT" | grep -q "h
: > "$GHLOG"; OUT="$(rr --import "$W/unsafe.md" 2>&1)"
chk "--import refuses a bug that still fails the scan after re-redaction" 'echo "$OUT" | grep -q "^REFUSED: after re-redaction" && ! grep -qi "A8f3" "$GHLOG" && echo "$OUT" | grep -q "refused 1"'

echo "S. #759 ordinary English survives: a match-everything deny line, attribute names that are words"
# The reported title, byte for byte, and the toolkit-clone denylist `init-project.sh .` wrote.
T759="journey-runner: 'row' never matches a Mendix 11 list view item"
printf '# Leak-guard denylist — gitignored. See bin/check-no-client-data.sh header.\n\\b.\\b\n\\bFrobnozzle\\b\n' > "$W/tk-deny-dot"
G="$(MXTK_LEAKGUARD_DENYFILE="$W/tk-deny-dot" P2DENY='Snarfle\b' bodyof "$T759. The widget inside the list view item whose text matches is used. Frobnozzle too.")"
chk "#759 a \\b.\\b deny line leaves the reported text unchanged and the row is not refused" 'echo "$G" | grep -qF "$T759. The widget inside the list view item whose text matches is used."'
chk "#759 the dropped line is warned about, by pattern; the other lines in that file still redact" 'grep -qF "deny pattern \`\\b.\\b\` matches ordinary text" "$W/p2.err" && ! echo "$G" | grep -qi frobnozzle'
# Fail closed (review of #815): a probe-matching line that still names someone is refused, not dropped.
G="$(P2DENY='Acmecorp|Initrode|' bodyof "Acmecorp and Initrode staff saw it.")"
chk "#759 a deny line holding names that also matches everything (Acmecorp|Initrode|) is refused; the names never reach a body" '! echo "$G" | grep -q "Acmecorp\|Initrode" && grep -q "invalid deny pattern.*Acmecorp|Initrode|.*matches ordinary text" "$W/p2.err"'
G="$(P2DENY='Snarfle\b' LEAKGUARD_DENY='(Acmecorp)?' bodyof "Acmecorp saw it.")"
chk "#759 the same for \$LEAKGUARD_DENY (Acmecorp)?: refused, never posted in clear" '! echo "$G" | grep -q Acmecorp && grep -q "invalid deny pattern" "$W/p2.err"'
G="$(P2DENY='Öko|' bodyof "Öko staff saw it.")"
chk "#759 a non-ASCII name that matches everything (Öko|) is refused too, never dropped and posted in clear" '! echo "$G" | grep -q "Öko" && grep -q "invalid deny pattern.*matches ordinary text" "$W/p2.err"'
G="$(MXTK_LEAKGUARD_DENYFILE="$W/tk-deny-dot" P2DENY='Snarfle\b' bodyof "Plain text, no names.")"
chk "#759 a nameless line (\\b.\\b) is still dropped with a warning, not refused" 'echo "$G" | grep -qF "Plain text, no names." && grep -qF "deny pattern \`\\b.\\b\` matches ordinary text" "$W/p2.err" && ! grep -q "invalid deny pattern" "$W/p2.err"'
KMDL759='CREATE PERSISTENT ENTITY Quuxmod.Gadget (
Round: Integer,
Used: Boolean
);'
G="$(P2MDL="$KMDL759" bodyof 'The value is used here. ### Workaround: none. Flow ACT_Round_Save reads $w/Used and myRound.')"
chk "#759 attribute Round leaves Workaround alone; attribute Used leaves prose used alone" 'echo "$G" | grep -qF "The value is used here. ### Workaround: none."'
chk "#759 the same attributes are still redacted in code: ACT_Round_Save, \$w/Used, myRound" '! echo "$G" | grep -q "Round_Save\|/Used\|myRound" && echo "$G" | grep -q "ACT_Attr[0-9]*_Save"'
# init-project.sh no longer writes the line: the name is resolved, escaped, deduped, and a short one refused.
IP="$W/ip"; mkdir -p "$IP/Wobblecorp" "$IP/my.app"; : > "$W/ip-deny"
( cd "$IP/Wobblecorp" && MXTK_NO_GUIDE=1 MXTK_LEAKGUARD_DENYFILE="$W/ip-deny" bash "$TOOLKIT/bin/init-project.sh" . --ignore-sources </dev/null >/dev/null 2>&1
cd "$IP/Wobblecorp" && MXTK_NO_GUIDE=1 MXTK_LEAKGUARD_DENYFILE="$W/ip-deny" bash "$TOOLKIT/bin/init-project.sh" . --ignore-sources </dev/null >/dev/null 2>&1
MXTK_NO_GUIDE=1 MXTK_LEAKGUARD_DENYFILE="$W/ip-deny" bash "$TOOLKIT/bin/init-project.sh" "$IP/my.app" --ignore-sources </dev/null >/dev/null 2>&1
MXTK_NO_GUIDE=1 MXTK_LEAKGUARD_DENYFILE="$W/ip-deny" bash "$TOOLKIT/bin/init-project.sh" "$IP/ab" --ignore-sources </dev/null >/dev/null 2>&1 )
chk "#759 init-project.sh . registers the folder's name, once, never \\b.\\b" '[ "$(grep -cxF "\\bWobblecorp\\b" "$W/ip-deny")" = 1 ] && ! grep -qxF "\\b.\\b" "$W/ip-deny"'
chk "#759 init-project.sh escapes a dotted name and refuses one under 3 characters" 'grep -qxF "\\bmy\\.app\\b" "$W/ip-deny" && ! grep -q "ab\\\\b" "$W/ip-deny"'

echo "Z. no real issue was ever created"
chk "every gh call went through the stub (log exists, no network tool used)" '[ -s "$GHLOG" ]'

Expand Down
Loading