diff --git a/CHANGELOG.md b/CHANGELOG.md index e9e148df..50434983 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m `docs/RELEASE-PROCESS.md` §8. ## Unreleased +- fix(file-bugs, init-project): **bug-report redaction no longer garbles ordinary English**: `init-project.sh .` took the name `.` and wrote `\b.\b` to the toolkit clone's denylist, which file-bugs.sh reads, so every space and hyphen became `` and the scan refused the report as a deny-list match; separately, an attribute named `Round` turned "Workaround" into `WorkaAttr390` and one named `Used` redacted the word "used". `init-project.sh` now resolves the folder before taking its name, writes it regex-escaped (`\bmy\.app\b`), dedupes on that exact line and refuses a name under 3 characters. `file-bugs.py` ignores, with a warning, any deny pattern that matches empty text, a space or `a-b c` (this repairs clones that already hold the line), and matches attribute names only where an identifier segment starts (a short or common-word one in its model spelling only); `ACT_Round_Save`, `$w/Used`, `Kundennummer2` are still redacted; known limit: a distinctive attribute inside a lowercase compound (`prekundennummer`) and an all-caps weak one (`USED`) are no longer redacted. A probe-matching deny line that still holds a name (`Acme|Initrode|`, `(Acme)?`, `Öko|` — any 3-letter run, non-ASCII letters included) is refused, not dropped: posting, `--links` and `--json` stop until it is fixed (fail closed). Fixture `test-file-bugs.sh` S — thanarat (#759) - new(doctor, bug ledger): **a guard for mxcli's silent MxBuild fallback**: when the project's Mendix version is not in `~/.mxcli/mxbuild/` (and no Studio Pro of that version exists), `mxcli docker build` / `docker check` fall back to the newest cached MxBuild and print its path but no version warning (`run --local` is exact-or-refuse); `bin/doctor.sh` now warns, naming the model version, the cached ones and `./mxcli setup mxbuild -p .mpr` (`mxtk_mxbuild_cache_gap`, warning only, never changes a gate). Ledger entry `BUG-DRAFT-mxbuild-silent-version-fallback`; upstream issue not filed. — a four-app Mendix build (field run), independent review - fix(gate-check): **a fresh clone no longer FAILs Stage 3 as "stale render"**: `blueprint.html` / `process-map.html` freshness used `[ render -ot source ]`, a nanosecond mtime compare, and a checkout stamps files in directory order (`process-map.html` lands before `process-model.json`), so an unedited project read stale on every clone (reproduced: 200-wireframe checkout, 10 of 10 clones). New `bin/lib/render-fresh.sh` keeps `-ot` as the first test (a pass never flips, no git call) and, only where `-ot` would FAIL, asks git once per render (one `log`, one `status`) whether the source was committed after the render's commit; measured 0.19 s for 200 sources all `-ot`-stale on a 300-commit history (a per-source git version cost 12.9 s). Limit: a depth-1 clone cannot see a genuinely stale render. Field run: a four-app Mendix build checked out on a second machine. — a four-app Mendix build (field run), independent review - fix(test-stack-up, e2e-harness-base): **the app port follows the app's configured port, and a multi-app start is no longer pinned to 8080**: with no `APP_PORTS`, `test-stack-up.sh` puts `RUN_LOCAL_APP_PORT`, then `APP_PORT` from the `.docker/.env` beside the `.mpr` (`apps//.docker/.env`; the project-root one as fallback) first, ahead of the old fixed `8080 8081 8084` tail. A fresh `run --local` start for a model under `apps/` with no handshake and no configured port used to fall back to 8080 for every app (`--app-port 8080`); it now exits 4 naming `RUN_LOCAL_APP_PORT`. `e2e-harness-base.md` states that a PAD/Docker runtime listens on 8080 *inside* the container only. Fixtures T28–T32 added. Field run: a four-app Mendix build on 8081–8084 whose fresh start failed with every app pinned to 8080. — a four-app Mendix build (field run), independent review diff --git a/bin/init-project.sh b/bin/init-project.sh index 85671931..c82ffe73 100755 --- a/bin/init-project.sh +++ b/bin/init-project.sh @@ -162,7 +162,10 @@ fi # second run the variable was simply unset. The leak-guard registration further # down reads it, and an unset name there writes a bare `\b\b` to the denylist: # a pattern that matches every file. Assign it once, unconditionally. -PROJECT_NAME="$(basename "$PROJECT_DIR")" +# +# Resolved first (#759): `init-project.sh .` took basename "." as the name, and the denylist +# line `\b.\b` below then matched every space and hyphen in every bug report and every file. +PROJECT_NAME="$(basename "$(cd "$PROJECT_DIR" 2>/dev/null && pwd || printf '%s' "$PROJECT_DIR")")" # Stamp the toolkit sha this scaffold was built from. # @@ -634,11 +637,20 @@ if [ -w "$TOOLKIT_ROOT" ]; then # matching the name with `[^A-Za-z0-9]` boundaries — never fires, because the # stored form is `\bName\b` and the character immediately before the name is # the `b` of `\b`, which is alphanumeric. Every re-run appended a duplicate. - # -F also removes any need to escape names containing `.`, `+` or `-`. - if [ ! -f "$DENYFILE" ] || ! grep -qiF -- "$PROJECT_NAME" "$DENYFILE" 2>/dev/null; then + # + # The name is written regex-escaped (#759): a `.` or `+` in it is a metacharacter in the + # extended-regex list, and `my.app` would also match `my-app` and `myXapp`. The dedupe looks + # for that exact line. A name under 3 characters is not registered: as a pattern it matches + # ordinary words everywhere. + DENY_LINE="\\b$(printf '%s' "$PROJECT_NAME" | sed 's/[][\.*^$+?(){}|]/\\&/g')\\b" + if [ "${#PROJECT_NAME}" -lt 3 ]; then + echo "" + echo "NOTE: \"$PROJECT_NAME\" is too short to register with the toolkit leak guard (it would match" + echo " ordinary words). Add a longer pattern for this project to $DENYFILE by hand." + elif [ ! -f "$DENYFILE" ] || ! grep -qixF -- "$DENY_LINE" "$DENYFILE" 2>/dev/null; then { [ -f "$DENYFILE" ] || printf '# Leak-guard denylist — gitignored. See bin/check-no-client-data.sh header.\n' - printf '\\b%s\\b\n' "$PROJECT_NAME" + printf '%s\n' "$DENY_LINE" } >> "$DENYFILE" 2>/dev/null && { echo "" echo "Registered \"$PROJECT_NAME\" with the toolkit leak guard ($DENYFILE)." diff --git a/bin/lib/file-bugs.py b/bin/lib/file-bugs.py index 1b2b9340..9793a5ac 100644 --- a/bin/lib/file-bugs.py +++ b/bin/lib/file-bugs.py @@ -194,6 +194,29 @@ def name_pat(name, sub=None): return r"(?:(?. +DENY_PROBES = ("", " ", "a-b c") + + def _toolkit_names(): here = os.path.dirname(os.path.abspath(__file__)) root = os.path.normpath(os.path.join(here, "..", "..")) @@ -300,10 +323,23 @@ def _load_deny(self): def add(pat, src): try: - pats.append(re.compile(pat, re.I)) - return + rx = re.compile(pat, re.I) except re.error: - pass + rx = None + if rx is not None: + if not any(rx.search(p) for p in DENY_PROBES): + pats.append(rx) + elif not re.search(r"[^\W_]{3,}", pat): + # holds no name at all (`\b.\b`, `\b\b`): dropping it loses nothing. Unicode-aware, so + # `Öko|` counts as a name (review of #815) + self.warnings.append("deny pattern `%s` matches ordinary text (empty, a space, a hyphen or a " + "one-letter word), so it is ignored. Remove that line from %s" + % (pat[:60], src)) + else: + # holds a name but also matches everything (`Acme|Initrode|`, `(Acme)?`): fail closed, + # since dropping it would post those names in clear (review of #815) + self.bad_deny.append("%s: %r matches ordinary text" % (src, pat[:60])) + return # invalid as a whole: nothing is kept, posting/--links/--json are refused until it is fixed self.bad_deny.append("%s: %r" % (src, pat[:60])) @@ -441,9 +477,11 @@ def _compile(self): self.matchers = [] if self.app: self.matchers.append((self.app, "App1", name_pat(self.app))) - for table in (self.mods, self.ents, self.docs, self.attrs): + for table in (self.mods, self.ents, self.docs): for n, tok in table.items(): self.matchers.append((n, tok, name_pat(n))) + for n, tok in self.attrs.items(): + self.matchers.append((n, tok, attr_pat(n))) for u in self.users: self.matchers.append((u, "", name_pat(u))) for h in self.hosts: diff --git a/tests/wave2/test-file-bugs.sh b/tests/wave2/test-file-bugs.sh index 626c8113..e1b47dd5 100755 --- a/tests/wave2/test-file-bugs.sh +++ b/tests/wave2/test-file-bugs.sh @@ -37,6 +37,9 @@ # repo has it or the user may create it, exact-title duplicate recorded not reposted, no gh or not # signed in -> links and exit 0, no deny list refused; --add --log redaction and drop; the # permissions deny --file/--all but not --auto +# S #759 a `\b.\b` deny line (from `init-project.sh .`) is dropped with a warning, but one that +# also holds a name (`Acmecorp|Initrode|`) is refused (fail closed); attribute names +# that are English words (Round, Used) leave prose alone and are still caught in code set -uo pipefail TOOLKIT="$(cd "$(dirname "$0")/../.." && pwd)" SUBJ="${1:-$TOOLKIT/bin/file-bugs.sh}" @@ -681,6 +684,38 @@ chk "--import default cap is 20 bugs" '[ $RC -ne 0 ] && echo "$OUT" | grep -q "h : > "$GHLOG"; OUT="$(rr --import "$W/unsafe.md" 2>&1)" chk "--import refuses a bug that still fails the scan after re-redaction" 'echo "$OUT" | grep -q "^REFUSED: after re-redaction" && ! grep -qi "A8f3" "$GHLOG" && echo "$OUT" | grep -q "refused 1"' +echo "S. #759 ordinary English survives: a match-everything deny line, attribute names that are words" +# The reported title, byte for byte, and the toolkit-clone denylist `init-project.sh .` wrote. +T759="journey-runner: 'row' never matches a Mendix 11 list view item" +printf '# Leak-guard denylist — gitignored. See bin/check-no-client-data.sh header.\n\\b.\\b\n\\bFrobnozzle\\b\n' > "$W/tk-deny-dot" +G="$(MXTK_LEAKGUARD_DENYFILE="$W/tk-deny-dot" P2DENY='Snarfle\b' bodyof "$T759. The widget inside the list view item whose text matches is used. Frobnozzle too.")" +chk "#759 a \\b.\\b deny line leaves the reported text unchanged and the row is not refused" 'echo "$G" | grep -qF "$T759. The widget inside the list view item whose text matches is used."' +chk "#759 the dropped line is warned about, by pattern; the other lines in that file still redact" 'grep -qF "deny pattern \`\\b.\\b\` matches ordinary text" "$W/p2.err" && ! echo "$G" | grep -qi frobnozzle' +# Fail closed (review of #815): a probe-matching line that still names someone is refused, not dropped. +G="$(P2DENY='Acmecorp|Initrode|' bodyof "Acmecorp and Initrode staff saw it.")" +chk "#759 a deny line holding names that also matches everything (Acmecorp|Initrode|) is refused; the names never reach a body" '! echo "$G" | grep -q "Acmecorp\|Initrode" && grep -q "invalid deny pattern.*Acmecorp|Initrode|.*matches ordinary text" "$W/p2.err"' +G="$(P2DENY='Snarfle\b' LEAKGUARD_DENY='(Acmecorp)?' bodyof "Acmecorp saw it.")" +chk "#759 the same for \$LEAKGUARD_DENY (Acmecorp)?: refused, never posted in clear" '! echo "$G" | grep -q Acmecorp && grep -q "invalid deny pattern" "$W/p2.err"' +G="$(P2DENY='Öko|' bodyof "Öko staff saw it.")" +chk "#759 a non-ASCII name that matches everything (Öko|) is refused too, never dropped and posted in clear" '! echo "$G" | grep -q "Öko" && grep -q "invalid deny pattern.*matches ordinary text" "$W/p2.err"' +G="$(MXTK_LEAKGUARD_DENYFILE="$W/tk-deny-dot" P2DENY='Snarfle\b' bodyof "Plain text, no names.")" +chk "#759 a nameless line (\\b.\\b) is still dropped with a warning, not refused" 'echo "$G" | grep -qF "Plain text, no names." && grep -qF "deny pattern \`\\b.\\b\` matches ordinary text" "$W/p2.err" && ! grep -q "invalid deny pattern" "$W/p2.err"' +KMDL759='CREATE PERSISTENT ENTITY Quuxmod.Gadget ( + Round: Integer, + Used: Boolean +);' +G="$(P2MDL="$KMDL759" bodyof 'The value is used here. ### Workaround: none. Flow ACT_Round_Save reads $w/Used and myRound.')" +chk "#759 attribute Round leaves Workaround alone; attribute Used leaves prose used alone" 'echo "$G" | grep -qF "The value is used here. ### Workaround: none."' +chk "#759 the same attributes are still redacted in code: ACT_Round_Save, \$w/Used, myRound" '! echo "$G" | grep -q "Round_Save\|/Used\|myRound" && echo "$G" | grep -q "ACT_Attr[0-9]*_Save"' +# init-project.sh no longer writes the line: the name is resolved, escaped, deduped, and a short one refused. +IP="$W/ip"; mkdir -p "$IP/Wobblecorp" "$IP/my.app"; : > "$W/ip-deny" +( cd "$IP/Wobblecorp" && MXTK_NO_GUIDE=1 MXTK_LEAKGUARD_DENYFILE="$W/ip-deny" bash "$TOOLKIT/bin/init-project.sh" . --ignore-sources /dev/null 2>&1 + cd "$IP/Wobblecorp" && MXTK_NO_GUIDE=1 MXTK_LEAKGUARD_DENYFILE="$W/ip-deny" bash "$TOOLKIT/bin/init-project.sh" . --ignore-sources /dev/null 2>&1 + MXTK_NO_GUIDE=1 MXTK_LEAKGUARD_DENYFILE="$W/ip-deny" bash "$TOOLKIT/bin/init-project.sh" "$IP/my.app" --ignore-sources /dev/null 2>&1 + MXTK_NO_GUIDE=1 MXTK_LEAKGUARD_DENYFILE="$W/ip-deny" bash "$TOOLKIT/bin/init-project.sh" "$IP/ab" --ignore-sources /dev/null 2>&1 ) +chk "#759 init-project.sh . registers the folder's name, once, never \\b.\\b" '[ "$(grep -cxF "\\bWobblecorp\\b" "$W/ip-deny")" = 1 ] && ! grep -qxF "\\b.\\b" "$W/ip-deny"' +chk "#759 init-project.sh escapes a dotted name and refuses one under 3 characters" 'grep -qxF "\\bmy\\.app\\b" "$W/ip-deny" && ! grep -q "ab\\\\b" "$W/ip-deny"' + echo "Z. no real issue was ever created" chk "every gh call went through the stub (log exists, no network tool used)" '[ -s "$GHLOG" ]'