Skip to content

Sync ako/mxcli: generate connector scoping and dotted names; one in-memory catalog; mx check --json findings; report-quality skill - #1900

Merged
ako merged 15 commits into
mendixlabs:mainfrom
ako:main
Nov 19, 2026
Merged

ako merged 15 commits into
mendixlabs:mainfrom
ako:main

Conversation

@ako

@ako ako commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

Syncs ako/mxcli:main into upstream: 7 commits since #1899 — sql … generate connector fixes, an in-memory catalog that holds together, mx check --json findings, and a report-quality skill.

sql … generate connector

  • Scoped to the clause it was given. views (customer) generated every table as well and then appended the named view, so one connector held two entities and two queries of the same name — not a model Mendix can load. tables (customer) alone happened to be correct, which made it look like a views-only quirk.
  • Objects whose names contain dots are generated, and any object that is skipped is reported. The address was flattened to one string and split again on every dot, so a table named after a Kafka topic (main.local__cdc.<Module>.<Entity>.1.local) resolved to a schema of 1, the lookup found nothing, and the object was silently omitted — 4 of 9 on one lake.

Catalog: an in-memory catalog is one database

On a run that built the catalog rather than loading it, one lint iterator failed with "no such table: attributes" while its neighbours worked on the same handle in the same run: each connection of the pool opened its own empty in-memory database. It is now one shared database. This also unblocks MPR017 on a project's first lint run.

check and verify

  • check rejects a mapping custom handler that feeds a parameter a member of the wrong type (CE0282, MDL107).
  • Findings come from mx check --json, keyed on module, document, element and unit ids, which are stable across runs and copies. The text output doesn't name the document of a flow element, so two microflows with the same defect printed the same line. Golden files come from a real MxBuild 11.16.1 run of the check-gap fixtures.

Docs

  • A report-quality skill for the scored report and the traps behind its number; assess-quality keeps only the manual review (it had drifted: it quoted 41 lint rules, there are 64 now).
  • The documented CI score check compares the decimal score in jq: [ "$SCORE" -lt 70 ] failed with "integer expression expected" on a score like 99.2, so the gate never worked

claude and others added 15 commits November 19, 2026 06:18
…anual review

Every report kind had a report-* skill except the scored report, which
was covered only by assess-quality: a 451-line mix of running `mxcli
report`, restating lint rules by id, and a manual review checklist. It
had drifted. It quoted 41 lint rules (64 now), knew nothing of accepting
a finding in the project brain, and did not point to the six reports
that now answer several of its manual steps.

report-quality covers running the scored report and the traps behind
its number:
- without .claude/lint-rules/ only the built-in rules run, so the score
  looks better;
- --modules drops project-wide findings;
- mx check is not part of the score;
- the penalty is per finding and does not scale with the app;
- an intended finding is accepted in the brain, and code still being
  written belongs in a lint baseline.

assess-quality now starts from the reports (a table of question, report
and skill) and keeps only what no rule can judge. Rule-id lists are gone:
`lint --list-rules` is their home. Its assessment template reads the
measured half from the reports and ends with what was not assessed.

bootstrap-app, graph-analysis and the docs now point to the skill that
answers each question.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_011oK82qTgUsRMETmzvjS27w
The documented CI check did `[ "$SCORE" -lt 70 ]`. The overall score has
one decimal (99.2), so bash fails with "integer expression expected" and
the gate never works. It now compares in jq (`.overallScore >= 70`).
The page also links the report-quality skill and names the software
map among the other report kinds.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_011oK82qTgUsRMETmzvjS27w
`VIEWS (customer)` generated every table as well, then appended the named
view, so one connector held two entities and two queries called
LakeSql.Customer / GetAllCustomers -- not a model Mendix can load. On a
DuckGraphLake lake, whose objects are all views: `views (customer)` gave 6
queries with one duplicate, `views (customer, order)` 7 with two.
`tables (customer)` alone was correct, which made it look like a views-only
quirk.

Three causes:

- `cfg.Tables == nil` meant "every table" with nothing suppressing it when
  Views was set; there was no views-only mode. Either clause now scopes the
  whole command.
- The table reader ignored information_schema's table_type -- the third
  column ShowTables already returns -- so a view was in the tables set before
  the view pass added it again. Discovery now partitions that one listing,
  ReadAllTableSchemas excludes views, and the new ReadAllSchemas is what the
  no-clause path enumerates. Filtering without that would have turned
  `generate connector` into `no tables or views found` on a CDC lake, where
  every object is a view; the driver's own view listing is not the answer
  either -- pg_views returns 14 internal temp views on the lake where
  information_schema.tables returns the 9 real objects.
- ReadTableSchema recorded no schema for a bare name, so `TABLES (customer)`
  and `VIEWS (cdcpublish.customer)` were different addresses and could not be
  seen as one object. It now resolves one, which also finishes #2414: that fix
  qualified the discovery path only, so an explicit `TABLES (customer)` still
  emitted the unrunnable `FROM customer`. refuseAmbiguousTable already asked
  which schemas hold a name, for all three drivers, and discarded the answer
  when there was exactly one; `schemasHolding` keeps it.

An object named in both clauses is now generated once, and the reported
counts say which objects were tables and which were views.

Four controls, each failing a different test: clause scoping reverted ->
ViewsClauseDoesNotPullInEveryTable; dedupe removed ->
ObjectInBothClausesEmittedOnce; table_type ignored ->
NoClauseGeneratesEverythingOnce and both reader tests; bare-name resolution
removed -> BareClauseNameIsQualified and ObjectInBothClausesEmittedOnce.

Fixes #2415

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_011oK82qQLKUyoXXGmbmX68t
…ction

Closes #2417.

On a run that BUILT the catalog rather than loading it, one iterator failed
while its neighbours worked on the same handle in the same run:

  Error: lint could not read the catalog for AttributesFor:
         SQL logic error: no such table: attributes (1)

while ctx.Entities() returned rows, and the next identical run against the
cached catalog was correct. It reproduced on testdata/testapp-views and not on
testdata/pedapp.

catalog.New() opened sql.Open("sqlite", ":memory:"). Every CONNECTION to
":memory:" is its own separate, empty database, and sql.Open hands back a POOL,
so the schema createTables wrote landed on whichever connection it got and a
query the pool later served on another saw nothing. Which iterator broke was
pool timing, hence the per-project difference. The failure therefore presented
as a MISSING TABLE on a schema that provably existed: sqlite_master listed all
88 views in the failing run.

The in-memory catalog now opens a uniquely named shared-cache database
(file:mxcli-catalog-<pid>-<n>?mode=memory&cache=shared) and pins one connection
open, because a shared cache is freed when its last connection closes. The name
keeps two catalogs in one process separate.

Two knock-on fixes, both consequences of the same mixing of pool and
transaction access:

- Builder.updateSnapshotCount read a count through the POOL while the build
  transaction was open, so it could not see the rows that transaction had just
  written: a fresh catalog's snapshot ObjectCount was 0. It now reads through
  b.txQuery, which is also the correct scope.
- foreign_keys was set with db.Exec, which reaches only the one pooled
  connection it runs on, so the pragma was off for every query served on
  another -- on the file path too. It now rides in the DSN, as the busy timeout
  already did for exactly this stated reason.

NOT the fix, recorded because it looks like one: db.SetMaxOpenConns(1) also
makes the catalog one database, and deadlocks. The readers nest -- Entities()
yields from an open cursor while the loop body calls AttributesFor() -- so the
inner read waits forever for the connection the outer cursor holds, and a lint
run that took seconds hung past 600s. Shared cache alone still hung on
updateSnapshotCount's pool read until that was routed through the transaction.
Both are covered by tests, including one that fails on a deadline rather than
hanging.

MPR017's interim guard is reframed, not deleted: the stale advice it carried
(re-run to use the cached catalog) is gone, while the invariant stays -- a rule
that cannot read its input must not be able to report a pass.

Measured
- Three new tests, red before the fix with the production message verbatim
  ("no such table: attributes") and with foreign_keys reading 0.
- The FIRST lint of a fresh project now reports what only the second used to.
- Control: reverting ONLY the DSN change, keeping the builder fix, brings the
  original error back.
- ./mdl/catalog/... and ./mdl/linter/... pass.

Correction to the issue as filed: it claimed a CI gate would see a pass on a
partially unreadable catalog. It would not. cmd_lint.go already prints
"Results are INCOMPLETE" and exits 1 on any recorded query error; measured
exit code 1. The claim came from reading the printed "No issues found." without
checking the exit status.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_011oK82sY9fcHtMuZcMNRm2B
…iles

The text lines of mx check do not name the document of a flow element, so two
microflows with the same defect print the same line. mx check --json carries
module, document, element and unit IDs, stable across runs and copies. Replace
the text parser with a JSON reader keyed on IDs, add golden files from a real
MxBuild 11.16.1 run of the check-gap fixtures, and correct the verify proposal.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_011oK82stfDEB1TGfJLVwUxc
fix(sql): scope GENERATE CONNECTOR to the clause it was given
…ny skipped

An object whose own name contains a dot was silently omitted from a generated
connector. On a DuckGraphLake lake the raw topic tables are named after the
Kafka topic, so 4 of the 9 listed objects were missing, with no warning and no
error -- the only warnings printed were about unmappable columns.

The address was flattened to one string and split again on every dot, so
`main.local__cdc.CdcPublish.Customer.1.local` resolved to schema `1`, name
`local`; DescribeTable found nothing and readListed's `continue` discarded the
error. Nothing was wrong with the database's side of it: information_schema
has the object's 7 columns under that exact table_name, and the quoted select
returns its 5 rows.

Two more faults sat behind that one, and fixing only the addressing would have
been worse than the bug -- the object would have come out with invalid SQL and
an invalid entity name, where before it was merely absent:

- QualifiedName joined the parts unquoted, so the query would have read
  `FROM main.local__cdc.CdcPublish.Customer.1.local`, which is
  `syntax error at or near ".1."`. Identifiers that are not bare words are now
  quoted, with an embedded quote doubled. A bare word is still left alone,
  reserved or not: `cdcpublish.order` parses, which #2414 measured, and
  quoting everything would change every query this tool has generated.
- toPascalCase treated only `_ - space` as separators, so TableToEntityName
  gave `LocalCdc.cdcpublish.customer.1.local`. A dot is a separator now, in
  the separator loop and in the early return that only capitalises the first
  letter -- without the second, `my.table` became `My.table` and never
  reached the loop.

The silent half is fixed separately from the parsing: readListed returns every
listed object it could not read, GenerateResult carries them as
SkippedObjects, and the executor prints one warning per object. An object the
listing names and the output omits is invisible to everything downstream, so
it is reported whatever the reason.

Four controls, each failing a different test: split on every dot ->
SplitTableName_QuotedNameKeepsItsDots; quoteIdent never quotes ->
QualifiedName_QuotesANameThatNeedsIt; `.` not a separator ->
TableToEntityName_DotsAreWordSeparators; the skipped list dropped ->
UnreadableObjectIsReported.

Live against the lake: 5 of 9 objects generated before, 9 of 9 after, and all
9 generated queries run against the database they came from.

Fixes #2422

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_011oK82qQLKUyoXXGmbmX68t
…ny skipped (#2433)

fix(sql): generate the objects whose names contain dots, and report any skipped
check: reject a custom handler feeding a parameter a member of the wrong type (CE0282)
docs: a report-quality skill; assess-quality keeps only the manual review
feat(docker): read mx check --json for findings, with real-output golden files
fix(catalog): an in-memory catalog is one database, not one per connection
@ako
ako merged commit 245d5dc into mendixlabs:main Nov 19, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants