Repository navigation
Sync ako/mxcli: generate connector scoping and dotted names; one in-memory catalog; mx check --json findings; report-quality skill - #1900
Merged
Merged
Conversation
…anual review Every report kind had a report-* skill except the scored report, which was covered only by assess-quality: a 451-line mix of running `mxcli report`, restating lint rules by id, and a manual review checklist. It had drifted. It quoted 41 lint rules (64 now), knew nothing of accepting a finding in the project brain, and did not point to the six reports that now answer several of its manual steps. report-quality covers running the scored report and the traps behind its number: - without .claude/lint-rules/ only the built-in rules run, so the score looks better; - --modules drops project-wide findings; - mx check is not part of the score; - the penalty is per finding and does not scale with the app; - an intended finding is accepted in the brain, and code still being written belongs in a lint baseline. assess-quality now starts from the reports (a table of question, report and skill) and keeps only what no rule can judge. Rule-id lists are gone: `lint --list-rules` is their home. Its assessment template reads the measured half from the reports and ends with what was not assessed. bootstrap-app, graph-analysis and the docs now point to the skill that answers each question. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_011oK82qTgUsRMETmzvjS27w
The documented CI check did `[ "$SCORE" -lt 70 ]`. The overall score has one decimal (99.2), so bash fails with "integer expression expected" and the gate never works. It now compares in jq (`.overallScore >= 70`). The page also links the report-quality skill and names the software map among the other report kinds. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_011oK82qTgUsRMETmzvjS27w
`VIEWS (customer)` generated every table as well, then appended the named view, so one connector held two entities and two queries called LakeSql.Customer / GetAllCustomers -- not a model Mendix can load. On a DuckGraphLake lake, whose objects are all views: `views (customer)` gave 6 queries with one duplicate, `views (customer, order)` 7 with two. `tables (customer)` alone was correct, which made it look like a views-only quirk. Three causes: - `cfg.Tables == nil` meant "every table" with nothing suppressing it when Views was set; there was no views-only mode. Either clause now scopes the whole command. - The table reader ignored information_schema's table_type -- the third column ShowTables already returns -- so a view was in the tables set before the view pass added it again. Discovery now partitions that one listing, ReadAllTableSchemas excludes views, and the new ReadAllSchemas is what the no-clause path enumerates. Filtering without that would have turned `generate connector` into `no tables or views found` on a CDC lake, where every object is a view; the driver's own view listing is not the answer either -- pg_views returns 14 internal temp views on the lake where information_schema.tables returns the 9 real objects. - ReadTableSchema recorded no schema for a bare name, so `TABLES (customer)` and `VIEWS (cdcpublish.customer)` were different addresses and could not be seen as one object. It now resolves one, which also finishes #2414: that fix qualified the discovery path only, so an explicit `TABLES (customer)` still emitted the unrunnable `FROM customer`. refuseAmbiguousTable already asked which schemas hold a name, for all three drivers, and discarded the answer when there was exactly one; `schemasHolding` keeps it. An object named in both clauses is now generated once, and the reported counts say which objects were tables and which were views. Four controls, each failing a different test: clause scoping reverted -> ViewsClauseDoesNotPullInEveryTable; dedupe removed -> ObjectInBothClausesEmittedOnce; table_type ignored -> NoClauseGeneratesEverythingOnce and both reader tests; bare-name resolution removed -> BareClauseNameIsQualified and ObjectInBothClausesEmittedOnce. Fixes #2415 Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_011oK82qQLKUyoXXGmbmX68t
…ction Closes #2417. On a run that BUILT the catalog rather than loading it, one iterator failed while its neighbours worked on the same handle in the same run: Error: lint could not read the catalog for AttributesFor: SQL logic error: no such table: attributes (1) while ctx.Entities() returned rows, and the next identical run against the cached catalog was correct. It reproduced on testdata/testapp-views and not on testdata/pedapp. catalog.New() opened sql.Open("sqlite", ":memory:"). Every CONNECTION to ":memory:" is its own separate, empty database, and sql.Open hands back a POOL, so the schema createTables wrote landed on whichever connection it got and a query the pool later served on another saw nothing. Which iterator broke was pool timing, hence the per-project difference. The failure therefore presented as a MISSING TABLE on a schema that provably existed: sqlite_master listed all 88 views in the failing run. The in-memory catalog now opens a uniquely named shared-cache database (file:mxcli-catalog-<pid>-<n>?mode=memory&cache=shared) and pins one connection open, because a shared cache is freed when its last connection closes. The name keeps two catalogs in one process separate. Two knock-on fixes, both consequences of the same mixing of pool and transaction access: - Builder.updateSnapshotCount read a count through the POOL while the build transaction was open, so it could not see the rows that transaction had just written: a fresh catalog's snapshot ObjectCount was 0. It now reads through b.txQuery, which is also the correct scope. - foreign_keys was set with db.Exec, which reaches only the one pooled connection it runs on, so the pragma was off for every query served on another -- on the file path too. It now rides in the DSN, as the busy timeout already did for exactly this stated reason. NOT the fix, recorded because it looks like one: db.SetMaxOpenConns(1) also makes the catalog one database, and deadlocks. The readers nest -- Entities() yields from an open cursor while the loop body calls AttributesFor() -- so the inner read waits forever for the connection the outer cursor holds, and a lint run that took seconds hung past 600s. Shared cache alone still hung on updateSnapshotCount's pool read until that was routed through the transaction. Both are covered by tests, including one that fails on a deadline rather than hanging. MPR017's interim guard is reframed, not deleted: the stale advice it carried (re-run to use the cached catalog) is gone, while the invariant stays -- a rule that cannot read its input must not be able to report a pass. Measured - Three new tests, red before the fix with the production message verbatim ("no such table: attributes") and with foreign_keys reading 0. - The FIRST lint of a fresh project now reports what only the second used to. - Control: reverting ONLY the DSN change, keeping the builder fix, brings the original error back. - ./mdl/catalog/... and ./mdl/linter/... pass. Correction to the issue as filed: it claimed a CI gate would see a pass on a partially unreadable catalog. It would not. cmd_lint.go already prints "Results are INCOMPLETE" and exits 1 on any recorded query error; measured exit code 1. The claim came from reading the printed "No issues found." without checking the exit status. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_011oK82sY9fcHtMuZcMNRm2B
…iles The text lines of mx check do not name the document of a flow element, so two microflows with the same defect print the same line. mx check --json carries module, document, element and unit IDs, stable across runs and copies. Replace the text parser with a JSON reader keyed on IDs, add golden files from a real MxBuild 11.16.1 run of the check-gap fixtures, and correct the verify proposal. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_011oK82stfDEB1TGfJLVwUxc
…ong type (CE0282, MDL107) Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_011oK82tETa46YQmMS58pQjL
fix(sql): scope GENERATE CONNECTOR to the clause it was given
…ny skipped An object whose own name contains a dot was silently omitted from a generated connector. On a DuckGraphLake lake the raw topic tables are named after the Kafka topic, so 4 of the 9 listed objects were missing, with no warning and no error -- the only warnings printed were about unmappable columns. The address was flattened to one string and split again on every dot, so `main.local__cdc.CdcPublish.Customer.1.local` resolved to schema `1`, name `local`; DescribeTable found nothing and readListed's `continue` discarded the error. Nothing was wrong with the database's side of it: information_schema has the object's 7 columns under that exact table_name, and the quoted select returns its 5 rows. Two more faults sat behind that one, and fixing only the addressing would have been worse than the bug -- the object would have come out with invalid SQL and an invalid entity name, where before it was merely absent: - QualifiedName joined the parts unquoted, so the query would have read `FROM main.local__cdc.CdcPublish.Customer.1.local`, which is `syntax error at or near ".1."`. Identifiers that are not bare words are now quoted, with an embedded quote doubled. A bare word is still left alone, reserved or not: `cdcpublish.order` parses, which #2414 measured, and quoting everything would change every query this tool has generated. - toPascalCase treated only `_ - space` as separators, so TableToEntityName gave `LocalCdc.cdcpublish.customer.1.local`. A dot is a separator now, in the separator loop and in the early return that only capitalises the first letter -- without the second, `my.table` became `My.table` and never reached the loop. The silent half is fixed separately from the parsing: readListed returns every listed object it could not read, GenerateResult carries them as SkippedObjects, and the executor prints one warning per object. An object the listing names and the output omits is invisible to everything downstream, so it is reported whatever the reason. Four controls, each failing a different test: split on every dot -> SplitTableName_QuotedNameKeepsItsDots; quoteIdent never quotes -> QualifiedName_QuotesANameThatNeedsIt; `.` not a separator -> TableToEntityName_DotsAreWordSeparators; the skipped list dropped -> UnreadableObjectIsReported. Live against the lake: 5 of 9 objects generated before, 9 of 9 after, and all 9 generated queries run against the database they came from. Fixes #2422 Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_011oK82qQLKUyoXXGmbmX68t
…ny skipped (#2433) fix(sql): generate the objects whose names contain dots, and report any skipped
check: reject a custom handler feeding a parameter a member of the wrong type (CE0282)
docs: a report-quality skill; assess-quality keeps only the manual review
feat(docker): read mx check --json for findings, with real-output golden files
fix(catalog): an in-memory catalog is one database, not one per connection
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Syncs
ako/mxcli:maininto upstream: 7 commits since #1899 —sql … generate connectorfixes, an in-memory catalog that holds together,mx check --jsonfindings, and a report-quality skill.sql … generate connector
views (customer)generated every table as well and then appended the named view, so one connector held two entities and two queries of the same name — not a model Mendix can load.tables (customer)alone happened to be correct, which made it look like a views-only quirk.main.local__cdc.<Module>.<Entity>.1.local) resolved to a schema of1, the lookup found nothing, and the object was silently omitted — 4 of 9 on one lake.Catalog: an in-memory catalog is one database
On a run that built the catalog rather than loading it, one lint iterator failed with "no such table: attributes" while its neighbours worked on the same handle in the same run: each connection of the pool opened its own empty in-memory database. It is now one shared database. This also unblocks MPR017 on a project's first lint run.
check and verify
mx check --json, keyed on module, document, element and unit ids, which are stable across runs and copies. The text output doesn't name the document of a flow element, so two microflows with the same defect printed the same line. Golden files come from a real MxBuild 11.16.1 run of the check-gap fixtures.Docs
[ "$SCORE" -lt 70 ]failed with "integer expression expected" on a score like 99.2, so the gate never worked