From 3c5e8641be94f5ef4c43239e7e3835782330ef6d Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 04:27:07 +0000 Subject: [PATCH 1/9] =?UTF-8?q?docs(proposals):=20measured=20additions=20?= =?UTF-8?q?=E2=80=94=20startup=20rule,=20implicit=20id,=20topic=20naming,?= =?UTF-8?q?=20export?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Findings measured against the fixture project (ako/TestApp 619864b, Mendix 11.16.1) and Studio Pro's editors, folded into the Kafka event proposal: - §1: the Business Events module is started by StartupBusinessEvents from the after-startup microflow; nothing calls ConsumeBusinessEvent. The omission fails silently and mx check is clean, so §5.1 gains a "module started" rule. This answers §9.5, now marked as such. - §1: topic is businessevents.., with the ChannelName constant as a one-topic dev shortcut. - §5.2: `id` is UI-only. Studio Pro lists it per entity, but the model stores five properties for Order and three for Customer. Authoring it would break CLAUDE.md rule 4. - §5.2: the topic is cdc...., so a major bump starts a new topic — and RENAME ENTITY reaches the same outcome via CE1585. Both now stated next to the version policy. - §5.4: the consumer clobber is measured (ako/mxcli#2381), with the publisher round-tripping to Unchanged as the control; and a publisher's channel id cannot be read, although it is the topic name. - §5.5 (new, part E): export a published service's contract. Studio Pro exports all three published types; four of five OData artifacts are fetched from a running app, AsyncAPI and the GraphQL schema are rendered. ODataPublish$PublishedContract is in the metamodel but not populated. - §5.1: `subscribe` now has two meanings; flagged for a decision. - §8: tests for each of the above, each with its control. - §9: a local Kafka broker retires item 5 and gives §8 a runtime check, but not items 1-4. Docs only; no code changes. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011oK82qQLKUyoXXGmbmX68t --- .../PROPOSAL_kafka_event_integration.md | 181 +++++++++++++++++- 1 file changed, 174 insertions(+), 7 deletions(-) diff --git a/docs/11-proposals/PROPOSAL_kafka_event_integration.md b/docs/11-proposals/PROPOSAL_kafka_event_integration.md index e040dbfde..c7fa67b6b 100644 --- a/docs/11-proposals/PROPOSAL_kafka_event_integration.md +++ b/docs/11-proposals/PROPOSAL_kafka_event_integration.md @@ -22,6 +22,7 @@ nothing else. This proposal covers the rest: | B | Author a consumed business event service (contract + subscriptions) | not possible | | C | Author a CDC service (sources, exposed attributes/associations, baseline) | not possible | | D | Run-configuration Kafka settings (`Settings$KafkaConfiguration`) | not possible | +| E | Export a published service's contract to a file | not possible, for any service type | **Written to be built without Studio Pro.** Every BSON shape below was captured from a Studio Pro 11.16.1-authored project, and the raw units are committed under @@ -58,6 +59,35 @@ from the module's constants and the contract's channel id. MDL needs no Kafka se for business events. (The constants are already authorable with `alter settings constant @BusinessEvents.ServerUrl …`.) +**The module must be started, and nothing in the model says so.** The consumers are +created by the Java action `BusinessEvents.StartupBusinessEvents`, called from the app's +after-startup microflow, with `BusinessEvents.BSd_ShutdownSubscribers` as the +before-shutdown counterpart. Nothing calls `BusinessEvents.ConsumeBusinessEvent`. In the +fixture: + +``` +AfterStartupMicroflow: 'CdcPublish.ASu_StartBusinessEvents' +BeforeShutdownMicroflow: 'BusinessEvents.BSd_ShutdownSubscribers' + +create or modify microflow CdcPublish.ASu_StartBusinessEvents () +returns Boolean +begin + call java action BusinessEvents.StartupBusinessEvents(); + return true; +end; +``` + +Both are authorable in MDL today, and the omission fails **silently**: the app boots, no +consumer group is joined, no event arrives, and `mx check` is clean. Hence the startup +check rule in §5.1. (This answers §9.5.) + +**Topic naming, and a dev-loop shortcut.** With no Event Broker config map the topic is +`businessevents..`, where `` is the 32-hex +`ChannelName` in the publisher's definition. Setting the `BusinessEvents.ChannelName` +constant instead routes *every* message of the app to that one topic — simpler against a +local broker, at the cost of one table for all messages. Either way mxcli cannot read a +publisher's channel id today; see §5.4. + ## 2. Fixtures All from `github.com/ako/TestApp` commit `619864b`, Mendix 11.16.1, every element @@ -290,6 +320,14 @@ $$ self-contained (design-mdl-syntax §3); a skill shows the three-statement recipe. - `create or modify` carries the operation `$ID`s by message name, as `businessEventIdentityCarry` already does for the publisher. +- **Two spellings of `subscribe`.** Today's `message X (…) subscribe entity …` already + means "this app consumes X of its *own* definition" (`CanPublish: true`, plus a + `ServiceOperation` with `Operation: subscribe`) — Studio Pro's publisher editor has both + a Publish and a Subscribe table for exactly that, so one definition can do both. The + body item proposed here writes the *same* `ServiceOperation`, but against an imported + contract with `Definition: null`. The two are told apart only by which property key was + used, which R11 enforces — unambiguous to the parser, not to a reader. Either give the + contract-based form a distinct keyword, or have `describe` label which kind it is. **Check rules** (new `MDL-BE*` codes; mx does not catch these): @@ -300,6 +338,7 @@ $$ | entity shape | entity is persistent, `extends BusinessEvents.ConsumedBusinessEvent`, and has an attribute of the mapped type for each payload property | | handler shape | microflow has exactly one parameter, of the consumed entity | | module present | the `BusinessEvents` marketplace module is in the project | +| module started | `BusinessEvents.StartupBusinessEvents` is reachable from the configured after-startup microflow (§1) — without it the app consumes nothing, silently, and every other check passes | ### 5.2 CDC service (part C) @@ -329,6 +368,13 @@ describe change data capture service CdcPublish.OrdersCDCService; project name (see §10) and preserved on modify. - A member name resolves to an attribute or association of that entity (attributes win; ambiguity is a check error). +- **`id` is implicit and must not be authored.** Studio Pro's editor lists `id (Long)` as + a checked, disabled row under every tracked entity, but it is *not* a stored property: + the fixture's `Order` shows six rows in the UI and holds five + (`OrderId, CustomerName, ProductName, Price, Order_Customer`), `Customer` four rows and + three. The object id reaches a consumer from the runtime's Debezium schema, not from the + model. Writing an `id` property would violate CLAUDE.md rule 4 (Studio Pro's exact + property set); accepting one in the member list is a check error. - **The baseline is computed, not authored.** After building the live sources, mxcli rebuilds `Baseline` so it is structurally equal to them (this is what Studio Pro's *accept changes* does) and sets the pointers. Version policy, since `mx check` ignores @@ -341,6 +387,23 @@ describe change data capture service CdcPublish.OrdersCDCService; - unchanged sources keep their stored version and baseline element `$ID`s. The fixture is consistent with this (Order is 1.1 after its association was added; Customer 1.0; service 1.1), but the rule is inferred — §9. +- **A major bump renames the topic.** Studio Pro's editor shows the topic beside the + revision, per entity: + + ``` + CdcPublish.Order Order 1.1 cdc.TestApp.Order.1.{space} + CdcPublish.Customer Customer 1.0 cdc.TestApp.Customer.1.{space} + ``` + + So the topic is `cdc....`: `AppName`, `ExposedName` + and the *major* are all topic components. A `major+1` therefore starts a **new topic**, + and downstream — a lake, a subscriber — that is a new table with no history; changing + `AppName` or an `ExposedName` does the same. `describe` should print the topic template + next to the revision, so this is as visible from MDL as it is in Studio Pro. +- **`RENAME ENTITY` is a trap.** The entity's `GUID` survives a rename, so CDC keeps + working, but `ExposedName` was defaulted from the entity name and now disagrees with the + baseline → CE1585 → accepting the change is a rename → `major+1` → new topic. Renaming + an entity that a CDC service exposes deserves a check warning. - Gate: Mendix ≥ 11.10 (where `Baseline` was introduced and CE1585 enforces it). Below that, refuse with a message rather than guess the older shape. @@ -363,6 +426,10 @@ Overlay rule (MODELSDK_ENGINE_ARCHITECTURE.md, "never invent a key"): write only the stored configuration already carries a `KafkaConfiguration` part; otherwise refuse with the Mendix version that introduced it. +**Priority: lowest of the five.** A dev loop does not need it — +`mxcli run --local --runtime-setting Kafka.BootstrapServers=…` already sets this per run. +Part D is for the committed, shared model. + ### 5.4 Read side (part A — do first) - **Defect today:** `describe business event service BusinessEventsClient.OrdersEventClient` @@ -372,7 +439,18 @@ with the Mendix version that introduced it. round-trip is broken and an LLM reading it learns nothing. Fix with 5.1's describe form; until 5.1's grammar lands, print the contract and subscriptions as comments. Note `create or modify` of a *publisher* over an existing consumer must refuse, not - overwrite the contract. + overwrite the contract. **Measured** (ako/mxcli#2381): a hand-written `create or modify` + against the fixture's consumer drops `Document` (2696 chars → `""`) *and* + `OperationImplementations[0].Microflow`, and invents a `Definition`, while + `check --references` passes and `exec` reports success. The control is that the + publisher round-trips to `Unchanged` on the same build, so this is the property carry, + not general churn. +- **A publisher's channel id cannot be read.** `show contract channels from ` + parses the cached contract and prints the 32-hex id; the same command on a *publisher* + errors ("no cached AsyncAPI contract … it may be a publisher"), although the id is + stored right there in `Definition.Channels[].ChannelName`. That id is the topic name + (§1), so it is what a downstream consumer or lake config needs. Surface it on the + publisher too — a read, not a design question. - `list business event services` gains a `Kind` column (published / consumed). - `list business event clients` (today: prints "not yet implemented") is removed or becomes a filter for consumed services — see §10. @@ -381,6 +459,63 @@ with the Mendix version that introduced it. impact analysis see CDC exposure, and a row in `MDL_FEATURE_MATRIX.md`. - Catalog: bump `CatalogSchemaVersion`. +### 5.5 Export a published service's contract (part E) + +mxcli has no export direction for any contract. `EXPORT` in the grammar is only +`EXPORT MAPPING`, and every contract code path runs the import direction +(`mdl/openapi/parser.go`, `mdl/types/edmx.go`, `mdl/types/asyncapi.go`; `show contract …` +reads a *cached* document on a consumed service). Studio Pro exports all three published +service types, so this is a gap against Studio Pro rather than a new idea: + +| Document | Studio Pro export actions | Where | +|---|---|---| +| `Rest$PublishedRestService` | `swagger.json` (OpenAPI 2), `openapi.json` (OpenAPI 3) | document context menu | +| `ODataPublish$PublishedODataService2` | service feed XML, service feed JSON, `$metadata`, `openapi.json` (OpenAPI 3), GraphQL schema | context menu, and Settings → Export | +| `BusinessEvents$BusinessEventService` (publisher) | AsyncAPI document, YAML or JSON | editor button "Export AsyncAPI Document" | +| `ChangeDataCapture$EntityChangeDataCaptureService` | none | — | + +**Two mechanisms, decided per artifact.** The published OData Settings tab pairs each +Export button with the URL it saves, which names the mechanism: + +- service feed → `/odata//` +- `$metadata` → `/odata//$metadata` +- OpenAPI → `/odata-doc/odata//openapi.json` (note `odata-doc`, not `odata`) +- GraphQL schema → an Export button with **no URL** + +So four of the five OData artifacts are fetched from a running app, and the GraphQL schema +is rendered. AsyncAPI is rendered too — nothing serves it — which is why B2 exists and why +this part cannot be done by fetching alone. + +Nothing caches these on the published side. `ODataPublish$PublishedContract` is in the +metamodel with `Metadata`, `OpenApi` and `GraphQL` strings and looks like a free lunch; it +is **not populated** — the fixture project's three published OData services have no +`Contract` part at all. Do not build on it. + +Proposed surface, a CLI command rather than an MDL statement: it writes a file, it is a CI +step, and `--format` / `-o` are awkward as MDL properties. + +```bash +mxcli export contract Mod.Service [--format ] [-o file] [--from-running ] +``` + +- the valid `--format` set follows from the document type, defaulting to what Studio Pro's + editor button produces (`asyncapi` for a publisher, `openapi3` for REST and OData); +- `asyncapi` and `graphql` render from the model; `metadata`, `openapi2`, `openapi3` and + the service feeds fetch, `--from-running` defaulting to a `run --local` instance; +- AsyncAPI defaults to YAML, matching the fixture and Studio Pro's save dialog; + `--format asyncapi-json` for the other choice. + +**Why it is on this proposal's critical path.** §5.1's `ContractFrom:` only works when +publisher and consumer live in the *same* project. Two apps — the case business events +exist for — need app A's contract as a file that app B's script names in `ContractFile:`, +and nothing in mxcli can produce that file. B2 builds the renderer; this exposes it. + +**Test it against mxbuild, not only the golden fixture.** §8's byte-equality test against +`asyncapi-OrdersEventService.yaml` covers one message with one `Long` attribute. The test +that matters for the two-app loop is: render from the fixture publisher → store as a +consumer's `Document` → `mx check` → 0 errors, with the control that an empty render +reproduces the `business events service should have definitions` exception (§6). + ## 6. Measured `mx check` behaviour (mxbuild 11.16.1) On a copy of the fixture project; each variant mutates one unit's BSON. Control: the @@ -410,7 +545,7 @@ entities from `testapp-describe.mdl`), mutate with a few lines of Go over | Step | Files | Notes | |---|---|---| -| A1 consumer describe/list | `mdl/executor/cmd_businessevents.go` | fix the lossy describe first; it is a data-loss bug on round-trip | +| A1 consumer describe/list | `mdl/executor/cmd_businessevents.go` | fix the lossy describe first; it is a data-loss bug on round-trip (ako/mxcli#2381). Also surface the publisher's channel id (§5.4) | | A2 CDC read | new `mdl/backend/modelsdk/cdc_read.go`, `mdl/backend/service.go` (interface), mock, `mdl/executor/cmd_cdc.go`, catalog builder + tables, registry | gen types exist; add the 4 missing properties (§4) first so reads see them | | B1 grammar/AST | `MDLParser.g4` (`Contract`/`ContractFile`/`ContractFrom` keys, `DOLLAR_STRING` as a property value, `subscribe` body item), `ast_businessevents.go`, `visitor_businessevents.go` | | | B2 AsyncAPI renderer | new `mdl/businessevents/asyncapi.go` | golden test = `asyncapi-OrdersEventService.yaml`, byte-for-byte | @@ -419,7 +554,8 @@ entities from `testapp-describe.mdl`), mutate with a few lines of Go over | C1 grammar/AST/visitor | new `ast_cdc.go`, `visitor_cdc.go` | | | C2 write | new `mdl/backend/modelsdk/cdc_write.go` | live sources + computed baseline + pointers; GUIDs read from the domain model; wire through `canon.Reconcile`; consider a `canon.identityFields` row (identity = `Entity` / `Attribute` / `Association` by name) | | C3 version policy | `cdc_write.go` | §5.2 | -| D1 settings | settings overlay | §5.3 | +| D1 settings | settings overlay | §5.3 — lowest priority | +| E1 export contract | new `cmd/mxcli/cmd_export_contract.go`, reusing B2's renderer | §5.5; `asyncapi` and `graphql` render, the rest fetch from a running app | | docs | `docs-site/src/`, `cmd/mxcli/syntax/`, a skill `.claude/skills/mendix/business-events/SKILL.md` | the consumer recipe and the runtime facts in §1 | MCP backend: out of scope here. Studio Pro 11.16.1's PED has **no** create path for @@ -444,12 +580,39 @@ so CDC over `--mcp` is likely feasible as a follow-up — it needs a live Studio (ADR-0008 conditional write; include a control that a real change does write). - **Describe round-trip:** describe → exec → describe is a fixed point for both documents; `mdl-examples/doctype-tests/` gets a consumer and a CDC script. +- **Consumer clobber, with a control** (ako/mxcli#2381): `create or modify` over the + fixture consumer preserves `Document` and `ServiceOperation.Microflow`, or refuses; + the control is that the fixture *publisher* still round-trips to `Unchanged`, so the + test fails for the right reason. +- **Startup rule, with a control:** a project whose after-startup microflow does not reach + `BusinessEvents.StartupBusinessEvents` fails the §5.1 rule, and the fixture (which wires + `CdcPublish.ASu_StartBusinessEvents`) passes it. +- **`id` is not written:** the CDC writer emits five properties for `Order` and three for + `Customer` — no `id` — against an MDL member list that does not mention it, and rejects + one that does (§5.2). +- **AsyncAPI renderer against mxbuild (part E):** render the fixture publisher's contract, + store it as a consumer's `Document`, `mx check` → 0 errors; control = an empty render + reproduces `business events service should have definitions` (§6). - **Integration (`-tags integration`):** each §6 variant as a test — the valid writes pass `mx check`; a writer forced to omit the baseline reproduces CE1585 (the control that proves the test detects the defect). +- **Runtime (optional, Linux):** with a local Kafka-compatible broker (§9), produce an + event with the module's CloudEvents headers and assert the handler microflow ran — the + only test that proves an authored consumer actually consumes. ## 9. Needs Studio Pro later (not blocking) +**What a local broker retires, and what it cannot.** DuckGraphLake +(`github.com/ako/DuckGraphLake`) serves producers *and* consumers on a Kafka endpoint, +tested against the Kafka Java client 4.0.1 — the Business Events module's own — with +classic consumer groups, `OffsetCommit`/`OffsetFetch`, and MCP `produce` / `read_topic` / +`list_groups`. An integration test can therefore inject an event carrying the headers the +module expects (`ce_type`, `ce_source`, `ce_id`, `ce_specversion` 1.0, +`content-type: application/json`) and assert the handler microflow ran, with **no second +Mendix app and no Studio Pro**. That gives §8 a runtime check and retires item 5 below. +It does **not** retire items 1–4: those are about what Studio Pro *writes*, which only a +Studio Pro export can settle. + 1. AsyncAPI rendering of Integer, Decimal, Boolean, String, DateTime and Enumeration payload fields: add one attribute of each to `OrdersEventService`, export, diff against the generator. Until then the renderer refuses those types for @@ -459,15 +622,19 @@ so CDC over `--mcp` is likely feasible as a follow-up — it needs a live Studio 3. CDC `Type` for Integer/Boolean/DateTime/Enumeration/Binary/AutoNumber/HashString. 4. Whether Studio Pro (not just mxbuild) opens a document with the pointers present but pointing at mxcli-minted baseline `$ID`s — expected yes; confirm once. -5. What registers the subscriptions at startup: no microflow in the project calls - `BusinessEvents.ConsumeBusinessEvent`, and no 11.15 runtime jar references it by - name. Not needed to author the model; needed for the skill's troubleshooting section. +5. ~~What registers the subscriptions at startup.~~ **Answered — see §1.** + `BusinessEvents.StartupBusinessEvents`, called from the after-startup microflow; + `ConsumeBusinessEvent` is never called. Verified against the fixture, which wires + `CdcPublish.ASu_StartBusinessEvents` as `AfterStartupMicroflow`. It is now a check + rule (§5.1), not an open question. ## 10. Open questions 1. **`AppName`:** the fixture holds the project name (`TestApp`). Write the `.mpr` basename, or read it from somewhere authoritative (project settings)? Must satisfy - CE1577's pattern either way. + CE1577's pattern either way. Not cosmetic: it is the second segment of every topic the + service publishes to (§5.2), so getting it wrong publishes, silently, to topics nobody + is reading. 2. **`list business event clients`:** remove it, or keep it as `list … services` filtered to consumed ones? (Recommendation: remove; one noun, a `Kind` column.) 3. Should a service with `ContractFrom: X` also *create* the From d7bdfdce87c5d19c2b3fafd527d2d725665a54d9 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 04:39:04 +0000 Subject: [PATCH 2/9] =?UTF-8?q?docs(proposals):=20settle=20the=20two=20ope?= =?UTF-8?q?n=20design=20points=20=E2=80=94=20consume=20keyword,=20export?= =?UTF-8?q?=20as=20CLI?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both decided by the maintainer in session: - §5.1: the contract-based body item is `consume entity … microflow …`, not `subscribe`. `subscribe` already means "this app consumes a message of its own definition", and Studio Pro's publisher editor has both a Publish and a Subscribe table for exactly that, so reusing it would give two spellings that write the same ServiceOperation. `consume` matches the metamodel's own vocabulary (ConsumedBusinessEventService, ConsumedBusinessEvent, the CBE_ entity prefix). Only the keyword changes: the stored Operation stays "subscribe", so no BSON changes. Lexing: a CONSUME token has to be added — the grammar has CONSUMED but not CONSUME — and longest-match keeps `consumed` on the existing token. - §5.5: `export contract` is a CLI command, not an MDL statement. Examples, the check-rule table and the B1 plan row updated to match. Docs only; no code changes. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011oK82qQLKUyoXXGmbmX68t --- .../PROPOSAL_kafka_event_integration.md | 42 ++++++++++++------- 1 file changed, 26 insertions(+), 16 deletions(-) diff --git a/docs/11-proposals/PROPOSAL_kafka_event_integration.md b/docs/11-proposals/PROPOSAL_kafka_event_integration.md index c7fa67b6b..508540f67 100644 --- a/docs/11-proposals/PROPOSAL_kafka_event_integration.md +++ b/docs/11-proposals/PROPOSAL_kafka_event_integration.md @@ -273,7 +273,7 @@ create or modify business event service BusinessEventsClient.OrdersEventClient ContractFrom: CdcPublish.OrdersEventService ) { - subscribe OrderPlaceEvent + consume OrderPlaceEvent entity BusinessEventsClient.CBE_OrderPlaceEvent microflow BusinessEventsClient.HandleBE_OrderPlaceEvent; }; @@ -284,7 +284,7 @@ create business event service Orders.OrdersEventClient ContractFile: 'contracts/OrdersEventService.yaml' ) { - subscribe OrderPlaceEvent entity Orders.CBE_OrderPlaceEvent microflow Orders.HandleBE_OrderPlaceEvent; + consume OrderPlaceEvent entity Orders.CBE_OrderPlaceEvent microflow Orders.HandleBE_OrderPlaceEvent; }; -- what describe prints, whichever key created it @@ -298,7 +298,7 @@ info: $$ ) { - subscribe OrderPlaceEvent entity Orders.CBE_OrderPlaceEvent microflow Orders.HandleBE_OrderPlaceEvent; + consume OrderPlaceEvent entity Orders.CBE_OrderPlaceEvent microflow Orders.HandleBE_OrderPlaceEvent; }; ``` @@ -313,28 +313,37 @@ $$ Java action's `extra code`); the property-list value rule needs to accept that token. `describe` always prints `Contract: $$ … $$`, so its output needs no files and round-trips. Check refuses a contract containing `$$` rather than escape it. -- `subscribe entity … [microflow …]` writes one `ServiceOperation`. +- `consume entity … [microflow …]` writes one `ServiceOperation` + (`Operation: "subscribe"` — see the keyword note below). The consumed entity and handler are ordinary MDL, declared before the service (`create persistent entity … extends BusinessEvents.ConsumedBusinessEvent (…)`, `create microflow …`). mxcli does not generate them implicitly — statements stay self-contained (design-mdl-syntax §3); a skill shows the three-statement recipe. - `create or modify` carries the operation `$ID`s by message name, as `businessEventIdentityCarry` already does for the publisher. -- **Two spellings of `subscribe`.** Today's `message X (…) subscribe entity …` already - means "this app consumes X of its *own* definition" (`CanPublish: true`, plus a - `ServiceOperation` with `Operation: subscribe`) — Studio Pro's publisher editor has both - a Publish and a Subscribe table for exactly that, so one definition can do both. The - body item proposed here writes the *same* `ServiceOperation`, but against an imported - contract with `Definition: null`. The two are told apart only by which property key was - used, which R11 enforces — unambiguous to the parser, not to a reader. Either give the - contract-based form a distinct keyword, or have `describe` label which kind it is. +- **The contract-based form is `consume`, not `subscribe`** (decided). Today's + `message X (…) subscribe entity …` already means "this app consumes X of its *own* + definition" (`CanPublish: true`, plus a `ServiceOperation` with `Operation: subscribe`) + — Studio Pro's publisher editor has both a Publish and a Subscribe table for exactly + that, so one definition can do both. Reusing `subscribe` for the contract form would + give two spellings that write the *same* `ServiceOperation` and differ only by which + property key was used: unambiguous to the parser, which R11 enforces, but not to a + reader. `consume` follows Mendix's own vocabulary for this side — the metamodel's + `ConsumedBusinessEventService` and `ConsumedBusinessEvent`, and the `CBE_` entity prefix + Studio Pro generates for it. **Only the MDL keyword differs**: the stored `Operation` + is `"subscribe"` either way (§3), so nothing about the BSON changes. + Lexing: the grammar already has a `CONSUMED` token (consumed OData/REST/MCP services) + but no `CONSUME`; add one, and ANTLR's longest-match keeps `consumed` resolving to the + existing token. Like every MDL keyword it stops being a bare identifier, so an + attribute or variable named `consume` needs quoting from then on — parser-level only, + not a platform-reserved name (CLAUDE.md, "Quoting Escapes Parser Keywords"). **Check rules** (new `MDL-BE*` codes; mx does not catch these): | Rule | Condition | |---|---| | contract parses | the YAML is AsyncAPI 2.x with at least one channel (else mxbuild crashes) | -| message exists | each `subscribe` names a message in `components.messages` (mx passes it, §6) | +| message exists | each `consume` names a message in `components.messages` (mx passes it, §6) | | entity shape | entity is persistent, `extends BusinessEvents.ConsumedBusinessEvent`, and has an attribute of the mapped type for each payload property | | handler shape | microflow has exactly one parameter, of the consumed entity | | module present | the `BusinessEvents` marketplace module is in the project | @@ -491,8 +500,9 @@ metamodel with `Metadata`, `OpenApi` and `GraphQL` strings and looks like a free is **not populated** — the fixture project's three published OData services have no `Contract` part at all. Do not build on it. -Proposed surface, a CLI command rather than an MDL statement: it writes a file, it is a CI -step, and `--format` / `-o` are awkward as MDL properties. +**Decided: a CLI command, not an MDL statement.** It writes a file, it is a CI step (app +A's pipeline emits the contract, app B's script names it in `ContractFile:`), and +`--format` / `-o` are awkward as MDL properties. ```bash mxcli export contract Mod.Service [--format ] [-o file] [--from-running ] @@ -547,7 +557,7 @@ entities from `testapp-describe.mdl`), mutate with a few lines of Go over |---|---|---| | A1 consumer describe/list | `mdl/executor/cmd_businessevents.go` | fix the lossy describe first; it is a data-loss bug on round-trip (ako/mxcli#2381). Also surface the publisher's channel id (§5.4) | | A2 CDC read | new `mdl/backend/modelsdk/cdc_read.go`, `mdl/backend/service.go` (interface), mock, `mdl/executor/cmd_cdc.go`, catalog builder + tables, registry | gen types exist; add the 4 missing properties (§4) first so reads see them | -| B1 grammar/AST | `MDLParser.g4` (`Contract`/`ContractFile`/`ContractFrom` keys, `DOLLAR_STRING` as a property value, `subscribe` body item), `ast_businessevents.go`, `visitor_businessevents.go` | | +| B1 grammar/AST | `MDLParser.g4` (`Contract`/`ContractFile`/`ContractFrom` keys, `DOLLAR_STRING` as a property value, `consume` body item), `ast_businessevents.go`, `visitor_businessevents.go` | | | B2 AsyncAPI renderer | new `mdl/businessevents/asyncapi.go` | golden test = `asyncapi-OrdersEventService.yaml`, byte-for-byte | | B3 write | `mdl/backend/modelsdk/business_event_write.go` | `Definition` null, `Document`, subscribe ops; identity carry by message name | | B4 check rules | `mdl/executor/validate_*` | §5.1 table | From 0a2ce069d946a89939052e2ebfe787302ce81d8e Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 04:40:17 +0000 Subject: [PATCH 3/9] check: refuse a mapping element whose association does not connect it to its parent (CE0295) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_011oK82tETa46YQmMS58pQjL --- ...apping-association-not-allowed-ce0295.json | 1 + .../json-structures-and-mappings/SKILL.md | 27 ++- CHANGELOG.md | 1 + .../integration/create-export-mapping.md | 1 + .../mapping-association-not-allowed.fail.mdl | 42 ++++ .../mapping-association-not-allowed.mdl | 32 +++ mdl/executor/validate_mapping_associations.go | 229 ++++++++++++++++++ .../validate_mapping_associations_test.go | 168 +++++++++++++ mdl/executor/validate_program.go | 2 + 9 files changed, 492 insertions(+), 11 deletions(-) create mode 100644 .claude/skills/fix-issue/findings/mdl-executor/2026-11-18-mapping-association-not-allowed-ce0295.json create mode 100644 mdl-examples/bug-tests/mapping-association-not-allowed.fail.mdl create mode 100644 mdl-examples/bug-tests/mapping-association-not-allowed.mdl create mode 100644 mdl/executor/validate_mapping_associations.go create mode 100644 mdl/executor/validate_mapping_associations_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-mapping-association-not-allowed-ce0295.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-mapping-association-not-allowed-ce0295.json new file mode 100644 index 000000000..8f53ac14f --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-mapping-association-not-allowed-ce0295.json @@ -0,0 +1 @@ +{"area": "mdl/executor", "date": "2026-11-18", "refs": ["ako/mxcli#1258"], "ce": ["CE0295"], "symptom": "An import or export mapping element `Assoc/Entity` whose association did not connect the entity above it to its own entity passed `check` and `exec` and was written; `mx check` then reported CE0295 \"Association '...' is not allowed\". Neither-end, parent-only and element-only associations were all rejected; the matching pair, and an end that is a generalization of the parent, the element's entity or both, built clean.", "file": "mdl/executor/validate_mapping_associations.go ValidateMappingAssociations (MDL100), called from validate_program.go with the lazy project opener. Tests: validate_mapping_associations_test.go; fixtures mdl-examples/bug-tests/mapping-association-not-allowed(.fail).mdl", "cause": "The export and import mapping builders resolve nothing about an element's association beyond qualifying its name; the message-definition path already refuses an association that connects neither direction (resolveAssociationCardinality), but the mapping builders never got that check. The four existing CE0295 findings were all the same mxcli-written array-container bug, so the rule for hand-written mappings was never recorded.", "insight": "**When the field-hit ranking shows a code only in findings about mxcli's own writer, the user-script form of the same error is probably unrefused: measure the hand-written shapes before assuming the writer fix covered it.** Measured on MxBuild 11.16.1 in separate project copies (export and import, one defect per copy plus a control): neither end, parent-only and element-only are CE0295; the plain pair is clean; specialization of the parent, the element or both is clean. My first control was wrong - an object element over a plain many-to-one Reference is also CE0295 (the association must reach at most one object; the repo's own example uses owner both) - which is a separate cardinality rule left uncovered, and the reason to run a control before trusting it. The resolver reuses associationCatalogFor and findEntityByQN, keeps the spelling the names were written in for project lookups (lower-casing them made the chain read as unreadable and the rule silent), and refuses only when both generalization chains were read to their roots. Sweep: 1,314 repo .mdl files, 0 MDL100 hits (weak without a project; the measurement is the evidence). Control: stubbing the function fails the script, group, project and parent-specialization tests."} diff --git a/.claude/skills/mendix/json-structures-and-mappings/SKILL.md b/.claude/skills/mendix/json-structures-and-mappings/SKILL.md index a494b11f4..7da0b2642 100644 --- a/.claude/skills/mendix/json-structures-and-mappings/SKILL.md +++ b/.claude/skills/mendix/json-structures-and-mappings/SKILL.md @@ -13,17 +13,17 @@ description: "Create and manage JSON structures, import mappings and export mapp | 33-110 | Key Concepts | | 111-122 | Export Mappings | | 123-187 | JSON Structures | -| 188-229 | Message Definitions | -| 230-393 | Import Mappings | -| 394-491 | Export Mappings | -| 492-509 | Starting a Mapping Below the Payload Root | -| 510-593 | Microflow Actions | -| 594-606 | Browse | -| 607-654 | Export Workflow: PE → NPE → JSON | -| 655-661 | Realistic Example: Countries REST API | -| 662-685 | Placing Documents in Folders | -| 686-701 | Documentation | -| 702-716 | Common Mistakes | +| 188-234 | Message Definitions | +| 235-398 | Import Mappings | +| 399-496 | Export Mappings | +| 497-514 | Starting a Mapping Below the Payload Root | +| 515-598 | Microflow Actions | +| 599-611 | Browse | +| 612-659 | Export Workflow: PE → NPE → JSON | +| 660-666 | Realistic Example: Countries REST API | +| 667-690 | Placing Documents in Folders | +| 691-706 | Documentation | +| 707-721 | Common Mistakes | # JSON Structures, Import Mappings & Export Mappings @@ -206,6 +206,11 @@ target entity** — the stored cardinality follows the direction of traversal an the association's type, so a `Reference` gives a single object one way and a list the other, while a `ReferenceSet` is a list both ways. +The association must also connect the entity the element sits under to the element's +own entity (either direction; a generalization of either is fine). Any other one is +CE0295 "Association '...' is not allowed"; `check` refuses it as MDL100, in import and +export mappings alike. + **Mendix 11.15 removed collections.** Each definition is its own document, and a mapping names it in two parts. The collection form is refused on 11.15, and the document form below 11.15: diff --git a/CHANGELOG.md b/CHANGELOG.md index 715c66e5e..80bc8983d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Fixed +- **`check` rejects a mapping element whose association does not connect it to its parent** (ako/mxcli#1258) — in an import or export mapping, `Assoc/Entity` has to connect the entity the element sits under to its own entity. An association between other entities, or one that touches only the parent or only the element's entity, passed `check` and `exec` and the build reported CE0295 "Association '...' is not allowed". New MDL100, which reads the association from the script or the project and follows generalizations of either entity (measured clean); an association it cannot find is never refused. A JSON object element over a to-many association is also CE0295 and is not covered. Measured on MxBuild 11.16.1, import and export. - **`check` rejects `owner both` between an entity and itself or its generalization** (ako/mxcli#1258) — `create association M.A_A from M.A to M.A owner both`, or from a specialization to its parent (either direction, any chain, Reference or ReferenceSet), passed `check` and `exec` and the build reported CE0002. New MDL099, which needs no project; an unrelated pair and every other owner are unaffected. Only generalizations the script declares are followed. Measured on MxBuild 11.16.1. - **The catalog cache sees a write made in the same second as the build** (ako/mxcli#2374) — freshness compared the project file's modification time as whole seconds, so a read that followed a write inside the second of the previous catalog build answered from the catalog as it was before the write (`marketplace update` finding no dependent associations about one run in six; `describe` and catalog queries could do the same). The time is now stored and compared to the nanosecond. A cache written by an older mxcli holds whole seconds and is rebuilt once per project. - **`alter page … set (visible: roles (…))` sets the module roles** (ako/mxcli#2371) — the alter form stored `roles (…)` as the visibility expression, reported success, and left a page `mx check` rejected with CE0117. It now writes the "Visible for module roles" setting as `create page` does, keeping the widget's condition, and `roles (…) and ` sets both. diff --git a/docs-site/src/reference/integration/create-export-mapping.md b/docs-site/src/reference/integration/create-export-mapping.md index 68661c8ee..0d29249c6 100644 --- a/docs-site/src/reference/integration/create-export-mapping.md +++ b/docs-site/src/reference/integration/create-export-mapping.md @@ -94,6 +94,7 @@ If `OR MODIFY` is specified and the mapping already exists, it is updated in pla `Assoc/module.ChildEntity AS alias` : Maps associated objects to a nested JSON object or array using the named alias. + The association must connect the entity the element sits under to the element's own entity (in either direction; an end may be a generalization of either entity). Any other association is refused by `check` (MDL100) because Mendix reports CE0295 "Association '...' is not allowed". ## Examples diff --git a/mdl-examples/bug-tests/mapping-association-not-allowed.fail.mdl b/mdl-examples/bug-tests/mapping-association-not-allowed.fail.mdl new file mode 100644 index 000000000..9253b20fa --- /dev/null +++ b/mdl-examples/bug-tests/mapping-association-not-allowed.fail.mdl @@ -0,0 +1,42 @@ +mdl 1; +-- NEGATIVE TEST — `mxcli check` must REFUSE this file (MDL100), with no project. +-- +-- An object mapping element's association must connect the entity it sits under +-- to its own entity. Written otherwise, `check` and `exec` accept it and the +-- build reports +-- +-- [CE0295] "Association '...' is not allowed." +-- +-- Measured on MxBuild 11.16.1, export AND import mappings, each of: the +-- association touches neither entity, touches only the parent, touches only the +-- element's entity. The matching pair is clean (see +-- mapping-association-not-allowed.mdl). +create json structure MyFirstModule.JSON_NotAllowed sample '{"orderId": 1, "customer": {"name": "A"}}'; +create non-persistent entity MyFirstModule.NaRoot (OrderId: integer); +create non-persistent entity MyFirstModule.NaCust (Name: string(50)); +create non-persistent entity MyFirstModule.NaOther (Name: string(50)); +create non-persistent entity MyFirstModule.NaOther2 (Name: string(50)); +create association MyFirstModule.NaOther_NaOther2 from MyFirstModule.NaOther to MyFirstModule.NaOther2; +create association MyFirstModule.NaCust_NaOther from MyFirstModule.NaCust to MyFirstModule.NaOther; +create association MyFirstModule.NaCust_NaRoot from MyFirstModule.NaCust to MyFirstModule.NaRoot owner both; + +create export mapping MyFirstModule.EMM_NaNeither with json structure MyFirstModule.JSON_NotAllowed { + MyFirstModule.NaRoot { + orderId = OrderId, + MyFirstModule.NaOther_NaOther2/MyFirstModule.NaCust as customer { name = Name } + } +}; + +create export mapping MyFirstModule.EMM_NaParentOnly with json structure MyFirstModule.JSON_NotAllowed { + MyFirstModule.NaRoot { + orderId = OrderId, + MyFirstModule.NaCust_NaRoot/MyFirstModule.NaOther as customer { name = Name } + } +}; + +create import mapping MyFirstModule.IMM_NaChildOnly with json structure MyFirstModule.JSON_NotAllowed { + create MyFirstModule.NaRoot { + OrderId = orderId, + create MyFirstModule.NaCust_NaOther/MyFirstModule.NaCust = customer { Name = name } + } +}; diff --git a/mdl-examples/bug-tests/mapping-association-not-allowed.mdl b/mdl-examples/bug-tests/mapping-association-not-allowed.mdl new file mode 100644 index 000000000..1ad66d71a --- /dev/null +++ b/mdl-examples/bug-tests/mapping-association-not-allowed.mdl @@ -0,0 +1,32 @@ +mdl 1; +-- Control for mapping-association-not-allowed.fail.mdl: the association connects the +-- parent and the element's entity, directly and through generalizations. Builds with +-- 0 errors on MxBuild 11.16.1 (measured: plain, element specialized, parent +-- specialized, both specialized; import mapping likewise). +create json structure MyFirstModule.JSON_Allowed sample '{"orderId": 1, "customer": {"name": "A"}}'; +create non-persistent entity MyFirstModule.AlRoot (OrderId: integer); +create non-persistent entity MyFirstModule.AlSubRoot extends MyFirstModule.AlRoot (SubV: integer); +create non-persistent entity MyFirstModule.AlCust (Name: string(50)); +create non-persistent entity MyFirstModule.AlSubCust extends MyFirstModule.AlCust (SubName: string(50)); +create association MyFirstModule.AlCust_AlRoot from MyFirstModule.AlCust to MyFirstModule.AlRoot owner both; + +create export mapping MyFirstModule.EMM_Al with json structure MyFirstModule.JSON_Allowed { + MyFirstModule.AlRoot { + orderId = OrderId, + MyFirstModule.AlCust_AlRoot/MyFirstModule.AlCust as customer { name = Name } + } +}; + +create export mapping MyFirstModule.EMM_AlBothSpecialized with json structure MyFirstModule.JSON_Allowed { + MyFirstModule.AlSubRoot { + orderId = OrderId, + MyFirstModule.AlCust_AlRoot/MyFirstModule.AlSubCust as customer { name = Name } + } +}; + +create import mapping MyFirstModule.IMM_Al with json structure MyFirstModule.JSON_Allowed { + create MyFirstModule.AlRoot { + OrderId = orderId, + create MyFirstModule.AlCust_AlRoot/MyFirstModule.AlCust = customer { Name = name } + } +}; diff --git a/mdl/executor/validate_mapping_associations.go b/mdl/executor/validate_mapping_associations.go new file mode 100644 index 000000000..212abfe4c --- /dev/null +++ b/mdl/executor/validate_mapping_associations.go @@ -0,0 +1,229 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// CE0295 "Association '…' is not allowed." — an object mapping element, in an +// import or an export mapping, whose association does not connect the entity it +// sits under to its own entity. Measured on MxBuild 11.16.1, both mapping kinds: +// +// the association touches neither entity CE0295 +// touches the parent, not the element's entity CE0295 +// touches the element's entity, not the parent CE0295 +// connects the two, in either direction clean +// an end is a generalization of the parent, of the +// element's entity, or of both clean +// +// `check` and `exec` accepted every rejected shape and wrote the mapping. (A +// Reference whose cardinality does not fit the element — a JSON object over a +// to-many association — is CE0295 too, but is a different rule and is not +// covered here.) +// +// The association is looked up in the script first, then the project; the +// generalization chain of each entity the same way. An association that cannot +// be found, or a chain that cannot be read to its root, is never refused: the +// question is then unanswered, not answered "no". +const mappingAssociationRule = "MDL100" + +// mappingNode is the part of an import or export element this rule reads. +type mappingNode struct { + entity, assoc, name string + children []mappingNode +} + +func exportMappingNode(d *ast.ExportMappingElementDef) mappingNode { + n := mappingNode{entity: d.Entity, assoc: d.Association, name: d.JsonName} + for _, c := range d.Children { + if c != nil { + n.children = append(n.children, exportMappingNode(c)) + } + } + return n +} + +func importMappingNode(d *ast.ImportMappingElementDef) mappingNode { + n := mappingNode{entity: d.Entity, assoc: d.Association, name: d.JsonName} + for _, c := range d.Children { + if c != nil { + n.children = append(n.children, importMappingNode(c)) + } + } + return n +} + +// ValidateMappingAssociations reports each mapping element whose association +// connects neither the entity above it nor its own entity (CE0295). open, which +// may be nil, supplies the project for what the script does not declare. +func ValidateMappingAssociations(prog *ast.Program, open func() backend.FullBackend) []linter.Violation { + if prog == nil { + return nil + } + r := newMappingAssocResolver(prog, open) + var out []linter.Violation + for _, stmt := range prog.Statements { + switch s := stmt.(type) { + case *ast.CreateExportMappingStmt: + if s.RootElement != nil { + out = append(out, r.walk("export mapping", s.Name, exportMappingNode(s.RootElement))...) + } + case *ast.CreateImportMappingStmt: + if s.RootElement != nil { + out = append(out, r.walk("import mapping", s.Name, importMappingNode(s.RootElement))...) + } + } + } + return out +} + +type mappingAssocResolver struct { + scriptEntities map[string]string // lower-cased entity QN -> lower-cased generalization QN ("" = none) + scriptAssocs map[string][2]string // lower-cased association QN -> lower-cased FROM, TO entity QNs + open func() backend.FullBackend + opened bool + project entityLookupBackend + catalog associationCatalog +} + +func newMappingAssocResolver(prog *ast.Program, open func() backend.FullBackend) *mappingAssocResolver { + r := &mappingAssocResolver{ + scriptEntities: map[string]string{}, scriptAssocs: map[string][2]string{}, open: open, + } + for _, stmt := range prog.Statements { + switch s := stmt.(type) { + case *ast.CreateEntityStmt: + gen := "" + if s.Generalization != nil { + gen = qualifyIn(s.Name.Module, s.Generalization.String()) + } + r.scriptEntities[strings.ToLower(s.Name.String())] = gen + case *ast.CreateAssociationStmt: + from := qualifyIn(s.Name.Module, s.Parent.String()) + to := qualifyIn(s.Name.Module, s.Child.String()) + r.scriptAssocs[strings.ToLower(s.Name.String())] = [2]string{strings.ToLower(from), strings.ToLower(to)} + } + } + return r +} + +// qualifyIn gives a bare name the module it is written in, as the builders do. +func qualifyIn(module, name string) string { + if name == "" || strings.Contains(name, ".") || module == "" { + return name + } + return module + "." + name +} + +func (r *mappingAssocResolver) projectBackend() entityLookupBackend { + if !r.opened { + r.opened = true + if r.open != nil { + if b := r.open(); b != nil { + if lb, ok := b.(entityLookupBackend); ok { + r.project = lb + r.catalog = associationCatalogFor(lb) + } + } + } + } + return r.project +} + +// ends returns the FROM and TO entities of an association, lower-cased. +func (r *mappingAssocResolver) ends(assocQN string) (from, to string, ok bool) { + key := strings.ToLower(assocQN) + if e, found := r.scriptAssocs[key]; found { + return e[0], e[1], true + } + if r.projectBackend() == nil { + return "", "", false + } + if sh, found := r.catalog.shapes[key]; found { + return strings.ToLower(sh.from), strings.ToLower(sh.to), true + } + return "", "", false +} + +// chain is the entity followed by its generalizations, keyed lower-cased, and +// whether the walk reached the root. The project is asked with the spelling the +// names were written in; only the comparison ignores case. +func (r *mappingAssocResolver) chain(entityQN string) (map[string]bool, bool) { + set := map[string]bool{} + for cur := entityQN; cur != ""; { + key := strings.ToLower(cur) + if set[key] { + return set, true // a cycle: Mendix rejects it elsewhere + } + set[key] = true + if gen, ok := r.scriptEntities[key]; ok { + cur = gen + continue + } + b := r.projectBackend() + if b == nil { + return set, false + } + ent, found := findEntityByQN(b, cur) + if !found { + return set, false + } + cur = ent.GeneralizationRef + } + return set, true +} + +func (r *mappingAssocResolver) walk(kind string, name ast.QualifiedName, root mappingNode) []linter.Violation { + var out []linter.Violation + var visit func(n mappingNode, parentEntity string) + visit = func(n mappingNode, parentEntity string) { + for _, c := range n.children { + childEntity := qualifyIn(name.Module, c.entity) + if c.entity != "" && c.assoc != "" && parentEntity != "" { + if v, bad := r.check(kind, name, c, qualifyIn(name.Module, c.assoc), parentEntity, childEntity); bad { + out = append(out, v) + } + } + next := parentEntity + if c.entity != "" { + next = childEntity + } + visit(c, next) + } + } + visit(root, qualifyIn(name.Module, root.entity)) + return out +} + +func (r *mappingAssocResolver) check(kind string, name ast.QualifiedName, el mappingNode, assocQN, parentEntity, entity string) (linter.Violation, bool) { + from, to, ok := r.ends(assocQN) + if !ok { + return linter.Violation{}, false + } + parent, parentDone := r.chain(parentEntity) + own, ownDone := r.chain(entity) + if (parent[from] && own[to]) || (parent[to] && own[from]) { + return linter.Violation{}, false + } + if !parentDone || !ownDone { + return linter.Violation{}, false + } + return linter.Violation{ + RuleID: mappingAssociationRule, + Severity: linter.SeverityError, + Message: fmt.Sprintf( + "%s %s: element %q maps %s through %s, but that association runs between %s and %s, "+ + "not between %s and %s — mxbuild reports CE0295 \"Association '%s' is not allowed\"", + kind, name.String(), el.name, entity, assocQN, from, to, parentEntity, entity, assocQN), + Location: linter.Location{DocumentType: kind, DocumentName: name.String()}, + Suggestion: fmt.Sprintf( + "Use an association that connects %s and %s (or entities they extend), or map the element under the entity that association does connect.", + parentEntity, entity), + }, true +} diff --git a/mdl/executor/validate_mapping_associations_test.go b/mdl/executor/validate_mapping_associations_test.go new file mode 100644 index 000000000..c8a32bd5a --- /dev/null +++ b/mdl/executor/validate_mapping_associations_test.go @@ -0,0 +1,168 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// CE0295 "Association '…' is not allowed." on an object mapping element whose +// association does not connect the entity it sits under to its own entity. +// Measured on MxBuild 11.16.1, export AND import mappings, every variant built +// and rejected by `mx check` while `check` and `exec` accepted it: +// +// association touches neither entity CE0295 +// touches the parent, not the element's entity CE0295 +// touches the element's entity, not the parent CE0295 +// connects the two (either direction) clean <- control +const mapHead = `mdl 1; +create module Mp; +create json structure Mp.JS sample '{"id": 1, "customer": {"name": "A"}}'; +create non-persistent entity Mp.Root (Id: integer); +create non-persistent entity Mp.Cust (Name: string(50)); +create non-persistent entity Mp.SubCust extends Mp.Cust (Extra: string(50)); +create non-persistent entity Mp.Other (Name: string(50)); +create non-persistent entity Mp.Other2 (Name: string(50)); +create association Mp.Cust_Root from Mp.Cust to Mp.Root owner both; +create association Mp.Other_Other2 from Mp.Other to Mp.Other2; +create association Mp.Cust_Other from Mp.Cust to Mp.Other; +` + +func exportMap(assoc string) string { + return "create export mapping Mp.EMM with json structure Mp.JS {\n Mp.Root {\n id = Id,\n " + + assoc + " as customer { name = Name }\n }\n};\n" +} + +func importMap(assoc string) string { + return "create import mapping Mp.IMM with json structure Mp.JS {\n create Mp.Root {\n Id = id,\n create " + + assoc + " = customer { Name = name }\n }\n};\n" +} + +func mapProgram(t *testing.T, src string) *ast.Program { + t.Helper() + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse: %v\n%s", errs, src) + } + return prog +} + +func TestValidateMappingAssociations_Script(t *testing.T) { + cases := []struct { + name string + assoc string + want bool // refused + }{ + {"the association connects parent and element (control)", "Mp.Cust_Root/Mp.Cust", false}, + {"neither end is the parent or the element", "Mp.Other_Other2/Mp.Cust", true}, + {"touches the parent, not the element's entity", "Mp.Cust_Root/Mp.Other", true}, + {"touches the element's entity, not the parent", "Mp.Cust_Other/Mp.Cust", true}, + // The association's FROM is Mp.Cust, which the element's entity extends. + {"an end is a generalization of the element's entity (control)", "Mp.Cust_Root/Mp.SubCust", false}, + } + for _, kind := range []struct { + name string + body func(string) string + rule string + }{{"export", exportMap, "export mapping"}, {"import", importMap, "import mapping"}} { + for _, c := range cases { + t.Run(kind.name+": "+c.name, func(t *testing.T) { + v := ValidateMappingAssociations(mapProgram(t, mapHead+kind.body(c.assoc)), nil) + if got := len(v) > 0; got != c.want { + t.Fatalf("violations=%v, want refused=%v", v, c.want) + } + if c.want { + if v[0].RuleID != mappingAssociationRule || !strings.Contains(v[0].Message, "CE0295") || + v[0].Location.DocumentType != kind.rule { + t.Errorf("rule, CE code or location wrong: %+v", v[0]) + } + } + }) + } + } +} + +// An entity-less group node passes the enclosing entity through, as the export +// builder does: the association below it still has to connect that entity. +func TestValidateMappingAssociations_GroupPassesTheParentThrough(t *testing.T) { + wrap := func(assoc string) string { + return "create export mapping Mp.EMM with json structure Mp.JS {\n Mp.Root {\n id = Id,\n group AS wrapper {\n " + + assoc + " as customer { name = Name }\n }\n }\n};\n" + } + if v := ValidateMappingAssociations(mapProgram(t, mapHead+wrap("Mp.Cust_Root/Mp.Cust")), nil); len(v) != 0 { + t.Errorf("control refused: %v", v) + } + if v := ValidateMappingAssociations(mapProgram(t, mapHead+wrap("Mp.Other_Other2/Mp.Cust")), nil); len(v) != 1 { + t.Errorf("a wrong association under a group was not refused: %v", v) + } +} + +// An association the script does not declare and no project supplies is not +// looked at: refusing what cannot be read would be a false report. +func TestValidateMappingAssociations_UnknownAssociationIsNeverRefused(t *testing.T) { + src := strings.Replace(mapHead, "create association Mp.Cust_Root from Mp.Cust to Mp.Root owner both;\n", "", 1) + + exportMap("Mp.Cust_Root/Mp.Cust") + if v := ValidateMappingAssociations(mapProgram(t, src), nil); len(v) != 0 { + t.Errorf("an association that cannot be read was refused: %v", v) + } +} + +// The association and the entities live in the project, not the script. +func TestValidateMappingAssociations_ProjectAssociation(t *testing.T) { + mod := &model.Module{BaseElement: model.BaseElement{ID: "mod-pj"}, Name: "Pj"} + ent := func(id, name, gen string) *domainmodel.Entity { + return &domainmodel.Entity{BaseElement: model.BaseElement{ID: model.ID(id)}, ContainerID: "dm-pj", Name: name, GeneralizationRef: gen} + } + root, cust, other := ent("e-root", "Root", ""), ent("e-cust", "Cust", ""), ent("e-other", "Other", "") + dm := &domainmodel.DomainModel{ + BaseElement: model.BaseElement{ID: "dm-pj"}, ContainerID: mod.ID, + Entities: []*domainmodel.Entity{root, cust, other}, + Associations: []*domainmodel.Association{ + {BaseElement: model.BaseElement{ID: "a1"}, Name: "Cust_Root", ParentID: cust.ID, ChildID: root.ID}, + }, + } + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListDomainModelsFunc: func() ([]*domainmodel.DomainModel, error) { return []*domainmodel.DomainModel{dm}, nil }, + GetModuleByNameFunc: func(string) (*model.Module, error) { return mod, nil }, + GetDomainModelFunc: func(model.ID) (*domainmodel.DomainModel, error) { return dm, nil }, + } + open := func() backend.FullBackend { return mb } + mapping := func(el string) string { + return "mdl 1;\ncreate json structure Pj.JS sample '{\"id\": 1, \"customer\": {\"name\": \"A\"}}';\n" + + "create export mapping Pj.EMM with json structure Pj.JS {\n Pj.Root {\n id = Id,\n " + el + + " as customer { name = Name }\n }\n};\n" + } + if v := ValidateMappingAssociations(mapProgram(t, mapping("Pj.Cust_Root/Pj.Cust")), open); len(v) != 0 { + t.Errorf("control: the project association connects the two, got %v", v) + } + if v := ValidateMappingAssociations(mapProgram(t, mapping("Pj.Cust_Root/Pj.Other")), open); len(v) != 1 { + t.Errorf("Cust_Root does not reach Other: want one violation, got %v", v) + } +} + +// Measured clean on MxBuild 11.16.1: the mapping's root is a specialization of +// the association's end (the parent side of the chain). +func TestValidateMappingAssociations_ParentSpecialization(t *testing.T) { + head := strings.Replace(mapHead, "create non-persistent entity Mp.Cust ", + "create non-persistent entity Mp.SubRoot extends Mp.Root (SubV: integer);\ncreate non-persistent entity Mp.Cust ", 1) + mapping := func(assoc string) string { + return "create export mapping Mp.EMM with json structure Mp.JS {\n Mp.SubRoot {\n id = Id,\n " + + assoc + " as customer { name = Name }\n }\n};\n" + } + if v := ValidateMappingAssociations(mapProgram(t, head+mapping("Mp.Cust_Root/Mp.Cust")), nil); len(v) != 0 { + t.Errorf("control: an association ending on the root's generalization was refused: %v", v) + } + if v := ValidateMappingAssociations(mapProgram(t, head+mapping("Mp.Cust_Other/Mp.Cust")), nil); len(v) != 1 { + t.Errorf("an association that reaches neither the root nor what it extends: want one violation, got %v", v) + } +} diff --git a/mdl/executor/validate_program.go b/mdl/executor/validate_program.go index 8e5790456..8414fc883 100644 --- a/mdl/executor/validate_program.go +++ b/mdl/executor/validate_program.go @@ -48,6 +48,8 @@ func ValidateProgram(prog *ast.Program, projectPath string) []linter.Violation { voids := newVoidCodeActions(prog, project) // owner Both between an entity and itself or its generalization (CE0002). violations = append(violations, ValidateAssociationOwnerBoth(prog)...) + // A mapping element whose association does not connect it to its parent (CE0295). + violations = append(violations, ValidateMappingAssociations(prog, project)...) for _, stmt := range prog.Statements { // Check enumeration values for reserved words if enumStmt, ok := stmt.(*ast.CreateEnumerationStmt); ok { From fbc72e62c3bdbfdb6a5b415c53f4e460a7abd949 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 05:31:35 +0000 Subject: [PATCH 4/9] fix(businessevents): refuse a rewrite that would destroy a consumer's contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In Mendix 11 a consumed business event service is the same BusinessEvents$BusinessEventService document as a published one, told apart by `Definition: null` plus the producer's AsyncAPI contract in `Document`, with a handler microflow on each ServiceOperation. createBusinessEventService builds a published service unconditionally: it never sets svc.Document, and `svc.Definition = def` always runs. So `create or modify` over a consumer dropped the contract (2696 chars -> "") and the handler, and invented a Definition — while `check --references` passed and exec reported "Modified business event service". The contract is the only record of the channel id a subscriber binds to, and nothing in the model can put it back. Guard, don't drop (ADR-0005): refuse the statement when the stored service has a contract, naming the byte count and pointing at describe and drop. Carrying the Document instead would produce a Document+Definition hybrid Studio Pro never writes. The handler loss was a separate mechanism. MDL has had `MICROFLOW qualifiedName` on the businessEventMessageDef rule all along, so a statement omitting it is a partial statement, not a request to unbind: the stored Microflow is now carried when the statement is silent, the way carriedDocumentation does (#1018). Read side, same issue: - describe never printed `microflow`, so describe -> exec unbound the handler on a publisher-shaped service too. - describe of a consumer emitted only the statement header — no property list, no body, no `;` — because the whole statement sits inside `if found.Definition != nil`, so exec of that output failed with "mismatched input '' expecting '('". A consumer is now described as MDL comments: the subscriptions and the contract verbatim. It parses, and there is no statement to re-run into the bug. The early return is why the diff dedents the published branch. Evidence: the four tests were written first and all failed with the reported symptoms; _PublisherRewriteStillAllowed passed throughout, so the guard is not refusing everything. End-to-end on ako/TestApp 619864b the refused exec left the unit byte-identical to pristine (Document 2696, Microflow intact), and the publisher still round-trips to "Unchanged" (ADR-0008 elision intact). Closes #2381 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011oK82qQLKUyoXXGmbmX68t --- ...nsumer-contract-and-handler-clobbered.json | 1 + ...siness-event-consumer-contract-clobber.mdl | 58 +++ mdl/executor/cmd_businessevents.go | 186 +++++++-- .../cmd_businessevents_consumer_test.go | 373 ++++++++++++++++++ 4 files changed, 578 insertions(+), 40 deletions(-) create mode 100644 .claude/skills/fix-issue/findings/mdl-executor/2026-11-18-business-event-consumer-contract-and-handler-clobbered.json create mode 100644 mdl-examples/bug-tests/2381-business-event-consumer-contract-clobber.mdl create mode 100644 mdl/executor/cmd_businessevents_consumer_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-business-event-consumer-contract-and-handler-clobbered.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-business-event-consumer-contract-and-handler-clobbered.json new file mode 100644 index 000000000..27fcc820f --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-business-event-consumer-contract-and-handler-clobbered.json @@ -0,0 +1 @@ +{"area":"mdl/executor","date":"2026-11-18","symptom":"`create or modify business event service` over a CONSUMED service (Mendix 11: same BusinessEvents$BusinessEventService document, told apart by `Definition: null` + the producer's AsyncAPI contract in `Document`) dropped `Document` (2696 chars -> \"\") and `OperationImplementations[].Microflow`, and invented a Definition. `check --references` passed and exec reported `Modified business event service`. Separately, `describe` of such a service printed only the bare header `create or modify business event service M.X` — no property list, no body, no `;` — because the whole statement sits inside `if found.Definition != nil`, so exec of describe output failed with `mismatched input '' expecting '('`.","cause":"createBusinessEventService builds a published service unconditionally: svc.Document is never set (businessEventServiceToGen does not write it either) and `svc.Definition = def` always runs, so a consumer is rewritten into a publisher shape. The handler loss is a different mechanism — MDL has had `MICROFLOW qualifiedName` on the businessEventMessageDef rule all along, so a statement omitting it is a partial statement whose unstated value the rebuild wrote as \"\"; businessEventIdentityCarry carried the operation $ID but nothing else off the stored op.","file":"mdl/executor/cmd_businessevents.go (guard in createBusinessEventService; operationID -> operationFor; describeConsumedBusinessEventService; microflow in describeBusinessEventService)","fix":"Guard-don't-drop (ADR-0005): refuse the statement when the stored service has a non-empty Document, naming the byte count and pointing at describe and drop. Carry the stored Microflow when the statement is silent, the way carriedDocumentation does (#1018). describe emits `microflow X` for an operation that has one, and describes a consumer as MDL comments — subscriptions plus the contract verbatim — which parses and holds no statement to re-run.","test":"mdl/executor/cmd_businessevents_consumer_test.go: _RefusesClobberingStoredContract, _PublisherRewriteStillAllowed (control), _CarriesStoredHandlerMicroflow (+ authored-wins control), _ConsumerDescribesAsParseableComments (both describe languages; asserts NO create statement is emitted), _EmitsHandlerMicroflow. Pre-fix run: all four positives failed with the reported symptoms, the publisher control passed. End-to-end on ako/TestApp 619864b: the refused exec left the unit byte-identical to pristine (Document 2696, Microflow intact) and the publisher still round-trips to `Unchanged`.","insight":"Two documents sharing one $Type is the trap: every `found.Definition != nil` and every unconditional `svc.Definition = def` is a place where one role's code silently handles the other. Grep for the discriminator, not for the document type. The handler microflow looked like a missing MDL spelling and was not — check the grammar before designing a feature, because `MICROFLOW qualifiedName` was already on the rule and the real defect was a one-line carry. A describe that emits a statement its own executor would refuse is worse than one that emits none: comments parse, carry everything, and cannot be re-run into the bug. Do not reach for a Document carry instead of the guard — carrying it would produce a Document+Definition hybrid Studio Pro never writes.","refs":["ako/mxcli#2381","ako/mxcli#2384","docs-wiki/bug-patterns/rewrite-drops-unauthored-state.md","docs-wiki/bug-patterns/describe-round-trip-gaps.md"]} diff --git a/mdl-examples/bug-tests/2381-business-event-consumer-contract-clobber.mdl b/mdl-examples/bug-tests/2381-business-event-consumer-contract-clobber.mdl new file mode 100644 index 000000000..b5f281093 --- /dev/null +++ b/mdl-examples/bug-tests/2381-business-event-consumer-contract-clobber.mdl @@ -0,0 +1,58 @@ +mdl 1; +-- ============================================================================ +-- ako/mxcli#2381 — `create or modify business event service` destroyed a +-- CONSUMED service's AsyncAPI contract and its handler microflow. +-- +-- In Mendix 11 both roles are the same BusinessEvents$BusinessEventService +-- document. What makes one a consumer is `Definition: null` plus the producing +-- app's AsyncAPI contract in `Document`, with a handler microflow on each +-- ServiceOperation. The rebuild wrote neither, so a statement like the one +-- below — which can only describe a *published* service — silently produced: +-- +-- Document 2696 chars -> "" +-- OperationImplementations[0].Microflow M.Handle… -> "" +-- Definition null -> invented +-- +-- while `check --references` passed and `exec` reported "Modified business +-- event service". The control is that the fixture's *publisher* round-tripped +-- to "Unchanged" on the same build, so this was the property carry and not +-- general churn. +-- +-- Why it matters: the contract is the only record of the channel id, and the +-- channel id is what binds a subscriber to the producer's topic at deployment +-- (`businessevents..`). Nothing in the model can put it +-- back, so this is guard-don't-drop territory (ADR-0005): the statement is now +-- refused rather than applied. +-- +-- This file only checks that the statement still PARSES — the defect is in the +-- executor and needs a stored consumer, which a standalone check has not got. +-- +-- Manual repro (needs a project holding a Studio Pro-authored consumer; the +-- fixture is ako/TestApp at 619864b, BusinessEventsClient.OrdersEventClient): +-- +-- mxcli exec mdl-examples/bug-tests/2381-business-event-consumer-contract-clobber.mdl \ +-- -p TestApp.mpr +-- +-- before the fix: "Modified business event service: …" (contract gone) +-- after the fix: Error: … is a consumed business event service: its stored +-- AsyncAPI contract (2696 bytes) holds the channel id +-- subscribers bind to … +-- +-- `describe business event service BusinessEventsClient.OrdersEventClient` +-- now prints the subscriptions and the contract as comments: parseable +-- output with no statement to re-run. Before the fix it printed a bare +-- `create or modify business event service …` header that failed to parse +-- with "mismatched input '' expecting '('". +-- +-- Unit coverage: mdl/executor/cmd_businessevents_consumer_test.go +-- ============================================================================ + +create or modify business event service BusinessEventsClient.OrdersEventClient +( + ServiceName: 'OrdersEventService', + EventNamePrefix: '' +) +{ + message OrderPlaceEvent (OrderId: Long, CustomerId: Long) subscribe + entity BusinessEventsClient.CBE_OrderPlaceEvent; +}; diff --git a/mdl/executor/cmd_businessevents.go b/mdl/executor/cmd_businessevents.go index 267cdf2d3..bc1040dd1 100644 --- a/mdl/executor/cmd_businessevents.go +++ b/mdl/executor/cmd_businessevents.go @@ -203,58 +203,137 @@ func describeBusinessEventService(ctx *ExecContext, name ast.QualifiedName) erro return mdlerrors.NewNotFound("business event service", name.String()) } + // A consumed service has no Definition of its own: it carries the producer's + // AsyncAPI contract and one subscription per event. MDL cannot author that + // shape yet, and emitting the statement header alone produced output that + // does not parse. Describe it as comments instead — a parseable description + // that states everything, and no statement to re-run (ako/mxcli#2381). + if found.Definition == nil { + describeConsumedBusinessEventService(ctx, foundModule, found) + return nil + } + // Output MDL CREATE statement if found.Documentation != "" { outputJavadoc(ctx.Output, found.Documentation) } fmt.Fprintf(ctx.Output, "create or modify business event service %s.%s\n", foundModule, found.Name) - if found.Definition != nil { - fmt.Fprintf(ctx.Output, "(\n") - fmt.Fprintf(ctx.Output, " ServiceName: %s", mdlQuote(ctx, found.Definition.ServiceName)) - fmt.Fprintf(ctx.Output, ",\n EventNamePrefix: %s", mdlQuote(ctx, found.Definition.EventNamePrefix)) - fmt.Fprintf(ctx.Output, "\n)\n") + fmt.Fprintf(ctx.Output, "(\n") + fmt.Fprintf(ctx.Output, " ServiceName: %s", mdlQuote(ctx, found.Definition.ServiceName)) + fmt.Fprintf(ctx.Output, ",\n EventNamePrefix: %s", mdlQuote(ctx, found.Definition.EventNamePrefix)) + fmt.Fprintf(ctx.Output, "\n)\n") - fmt.Fprintf(ctx.Output, "{\n") + fmt.Fprintf(ctx.Output, "{\n") - // Build operation map: messageName -> operation info - opMap := make(map[string]*model.ServiceOperation) - for _, op := range found.OperationImplementations { - opMap[op.MessageName] = op - } + // Build operation map: messageName -> operation info + opMap := make(map[string]*model.ServiceOperation) + for _, op := range found.OperationImplementations { + opMap[op.MessageName] = op + } - // Output messages - for _, ch := range found.Definition.Channels { - for _, msg := range ch.Messages { - // Format attributes - var attrs []string - for _, a := range msg.Attributes { - attrs = append(attrs, fmt.Sprintf("%s: %s", a.AttributeName, a.AttributeType)) - } + // Output messages + for _, ch := range found.Definition.Channels { + for _, msg := range ch.Messages { + // Format attributes + var attrs []string + for _, a := range msg.Attributes { + attrs = append(attrs, fmt.Sprintf("%s: %s", a.AttributeName, a.AttributeType)) + } - // Determine operation from OperationImplementations - opStr := "publish" - entityStr := "" - if op, ok := opMap[msg.MessageName]; ok { - if op.Operation == "subscribe" { - opStr = "subscribe" - } - if op.Entity != "" { - entityStr = fmt.Sprintf("\n entity %s", op.Entity) - } + // Determine operation from OperationImplementations + opStr := "publish" + entityStr := "" + microflowStr := "" + if op, ok := opMap[msg.MessageName]; ok { + if op.Operation == "subscribe" { + opStr = "subscribe" + } + if op.Entity != "" { + entityStr = fmt.Sprintf("\n entity %s", op.Entity) + } + // The grammar has `MICROFLOW qualifiedName` on the message + // rule; describe never printed it, so describe → exec + // unbound the handler (ako/mxcli#2381). + if op.Microflow != "" { + microflowStr = fmt.Sprintf("\n microflow %s", op.Microflow) } - - fmt.Fprintf(ctx.Output, " message %s (%s) %s%s;\n", - msg.MessageName, strings.Join(attrs, ", "), opStr, entityStr) } - } - fmt.Fprintf(ctx.Output, "};\n") + fmt.Fprintf(ctx.Output, " message %s (%s) %s%s%s;\n", + msg.MessageName, strings.Join(attrs, ", "), opStr, entityStr, microflowStr) + } } + fmt.Fprintf(ctx.Output, "};\n") + return nil } +// describeConsumedBusinessEventService writes a consumed service as MDL comments: +// the subscriptions, and the stored AsyncAPI contract verbatim. The contract is +// the irreplaceable part — its channel keys are the topics a subscriber binds to, +// and nothing else in the model records them — so it is printed in full rather +// than summarised. +func describeConsumedBusinessEventService(ctx *ExecContext, moduleName string, svc *model.BusinessEventService) { + comment := func(format string, args ...any) { + line := fmt.Sprintf(format, args...) + if line == "" { + fmt.Fprintln(ctx.Output, "--") + return + } + fmt.Fprintf(ctx.Output, "-- %s\n", line) + } + + comment("%s.%s is a CONSUMED business event service.", moduleName, svc.Name) + comment("") + comment("It stores no definition of its own: the producing app's AsyncAPI contract") + comment("and one subscription per event it handles. MDL cannot author this shape") + comment("yet, so what follows is a description and not a statement — there is") + comment("nothing here to re-run, and `create or modify` over it is refused.") + + if svc.Documentation != "" { + comment("") + comment("Documentation:") + for _, line := range strings.Split(strings.ReplaceAll(svc.Documentation, "\r\n", "\n"), "\n") { + comment(" %s", line) + } + } + + comment("") + if len(svc.OperationImplementations) == 0 { + comment("Subscriptions: none.") + } else { + comment("Subscriptions:") + for _, op := range svc.OperationImplementations { + comment(" %s (%s)", op.MessageName, orPlaceholder(op.Operation, "subscribe")) + comment(" entity %s", orPlaceholder(op.Entity, "(none)")) + comment(" microflow %s", orPlaceholder(op.Microflow, "(none)")) + } + } + + comment("") + if strings.TrimSpace(svc.Document) == "" { + comment("Contract: none stored. mxbuild refuses a consumed service without one") + comment("(\"business events service should have definitions\").") + return + } + comment("Contract (AsyncAPI, verbatim). Each key under `channels:` is the channel") + comment("id the producer minted; the Kafka topic is derived from it, so this is the") + comment("only record of what a subscriber binds to:") + for _, line := range strings.Split(strings.ReplaceAll(strings.TrimRight(svc.Document, "\n"), "\r\n", "\n"), "\n") { + comment(" %s", line) + } +} + +// orPlaceholder is value, or placeholder when value is empty. +func orPlaceholder(value, placeholder string) string { + if value == "" { + return placeholder + } + return value +} + // createBusinessEventService creates a new business event service from an AST statement. func createBusinessEventService(ctx *ExecContext, stmt *ast.CreateBusinessEventServiceStmt) error { if !ctx.ConnectedForWrite() { @@ -293,6 +372,23 @@ func createBusinessEventService(ctx *ExecContext, stmt *ast.CreateBusinessEventS if !stmt.CreateOrModify { return mdlerrors.NewAlreadyExistsMsg("business event service", moduleName+"."+stmt.Name.Name, fmt.Sprintf("business event service already exists: %s.%s (use create or modify to update)", moduleName, stmt.Name.Name)) } + // Guard, don't drop (ADR-0005): a stored AsyncAPI contract marks a + // *consumed* service, and this statement can only build a published one. + // Rewriting would null the contract and every handler microflow with it — + // and the contract is the only record of the channel id a subscriber binds + // to, so nothing in the model could put it back (ako/mxcli#2381). + if strings.TrimSpace(existing.Document) != "" { + return mdlerrors.NewValidationf( + "%s.%s stores an AsyncAPI contract (%d bytes) — the shape of a consumed "+ + "business event service, which MDL cannot author yet. That contract holds the "+ + "channel id its subscribers bind to, and this statement can only build a "+ + "published service, so applying it would destroy the contract and the handler "+ + "microflow of every subscription, with no way to recover either from the model. "+ + "Use `describe business event service %s.%s` to see what is stored; "+ + "`drop business event service %s.%s` first if you do mean to replace it with a published one.", + moduleName, stmt.Name.Name, len(existing.Document), + moduleName, stmt.Name.Name, moduleName, stmt.Name.Name) + } existingID = existing.ID existingExcluded = existing.Excluded existingContainerID = existing.ContainerID @@ -379,7 +475,14 @@ func createBusinessEventService(ctx *ExecContext, stmt *ast.CreateBusinessEventS Entity: msgDef.Entity, Microflow: msgDef.Microflow, } - op.ID = carry.operationID(msgDef.MessageName) + if storedOp := carry.operationFor(msgDef.MessageName); storedOp != nil { + op.ID = storedOp.ID + // A statement silent about the handler keeps the stored one, the + // way a silent doc comment does (#1018). + if op.Microflow == "" { + op.Microflow = storedOp.Microflow + } + } op.TypeName = "BusinessEvents$ServiceOperation" svc.OperationImplementations = append(svc.OperationImplementations, op) } @@ -520,16 +623,19 @@ func (c *businessEventIdentityCarry) channelFor(name string, ch *model.BusinessE return s.message } -// operationID is the $ID of the stored operation implementation for the named -// message, handed out once, or "". -func (c *businessEventIdentityCarry) operationID(messageName string) model.ID { +// operationFor is the stored operation implementation for the named message, +// handed out once, or nil. The caller takes its $ID, and its Microflow when the +// statement is silent: MDL has `MICROFLOW qualifiedName` on the message rule, so +// a rewrite that omits it is a partial statement, not a request to unbind the +// handler (ako/mxcli#2381). +func (c *businessEventIdentityCarry) operationFor(messageName string) *model.ServiceOperation { key := strings.ToLower(messageName) op, ok := c.ops[key] if !ok { - return "" + return nil } delete(c.ops, key) - return op.ID + return op } // storedAttributeID is the $ID of the stored attribute named name, or "". diff --git a/mdl/executor/cmd_businessevents_consumer_test.go b/mdl/executor/cmd_businessevents_consumer_test.go new file mode 100644 index 000000000..f9d837929 --- /dev/null +++ b/mdl/executor/cmd_businessevents_consumer_test.go @@ -0,0 +1,373 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/langver" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" +) + +// ako/mxcli#2381. In Mendix 11 a *consumed* business event service is the same +// BusinessEvents$BusinessEventService document as a published one: what makes it +// a consumer is `Definition: null` plus the producer's AsyncAPI contract in +// `Document`, with a handler microflow per ServiceOperation. +// +// The rebuild writes neither, so `create or modify` over a consumer dropped the +// contract and the handler and invented a Definition — with `check --references` +// clean and `exec` reporting success. The contract is the only place the channel +// id lives, and the channel id is what binds the subscriber to the producer's +// topic at deployment, so losing it breaks the subscription. +// +// The posture is guard-don't-drop (ADR-0005): refuse the statement rather than +// quietly produce a smaller document. + +const storedContract = `asyncapi: 2.2.0 +info: + title: "OrdersEventService" + version: "1.0.0" +channels: + f2477dc90f714d69bb12df0df1ee9cce: + subscribe: + operationId: receiveOrderPlaceEventEvents +` + +// storedConsumer is the fixture shape: Definition nil, Document set, one +// subscribe operation bound to an entity and a handler microflow. +func storedConsumer(containerID model.ID) *model.BusinessEventService { + return &model.BusinessEventService{ + BaseElement: model.BaseElement{ID: "sp-consumer"}, + ContainerID: containerID, + Name: "OrdersEventClient", + ExportLevel: "Hidden", + Definition: nil, + Document: storedContract, + OperationImplementations: []*model.ServiceOperation{{ + BaseElement: model.BaseElement{ID: "sp-op"}, + MessageName: "OrderPlaceEvent", + Operation: "subscribe", + Entity: "M.CBE_OrderPlaceEvent", + Microflow: "M.HandleBE_OrderPlaceEvent", + }}, + } +} + +func TestCreateOrModifyBusinessEventService_RefusesClobberingStoredContract(t *testing.T) { + mod := mkModule("M") + stored := storedConsumer(mod.ID) + h := mkHierarchy(mod) + withContainer(h, stored.ContainerID, mod.ID) + + var written *model.BusinessEventService + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListBusinessEventServicesFunc: func() ([]*model.BusinessEventService, error) { + return []*model.BusinessEventService{stored}, nil + }, + UpdateBusinessEventServiceFunc: func(svc *model.BusinessEventService) error { + written = svc + return nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + err := createBusinessEventService(ctx, &ast.CreateBusinessEventServiceStmt{ + Name: ast.QualifiedName{Module: "M", Name: "OrdersEventClient"}, + ServiceName: "OrdersEventService", + CreateOrModify: true, + Messages: []*ast.BusinessEventMessageDef{{ + MessageName: "OrderPlaceEvent", + Operation: "subscribe", + Entity: "M.CBE_OrderPlaceEvent", + Attributes: []*ast.BusinessEventAttributeDef{ + {Name: "OrderId", TypeName: "Long"}, + }, + }}, + }) + + if err == nil { + t.Fatalf("rewrite of a consumer was accepted; it must refuse.\nwritten = %+v", written) + } + if written != nil { + t.Errorf("the write was attempted anyway: Document=%q Definition=%v", written.Document, written.Definition != nil) + } + // The message has to say what is at stake and how to proceed, or the refusal + // is just a wall: the contract is unrecoverable from the model once gone. + for _, want := range []string{"contract", "M.OrdersEventClient"} { + if !strings.Contains(strings.ToLower(err.Error()), strings.ToLower(want)) { + t.Errorf("refusal does not mention %q: %v", want, err) + } + } +} + +// The guard must not fire on an ordinary publisher: it keys on a stored +// contract, not on the statement. Without this control the guard could refuse +// every rewrite and still pass the test above. +func TestCreateOrModifyBusinessEventService_PublisherRewriteStillAllowed(t *testing.T) { + mod := mkModule("M") + stored := &model.BusinessEventService{ + BaseElement: model.BaseElement{ID: "sp-publisher"}, + ContainerID: mod.ID, + Name: "OrdersEventService", + ExportLevel: "Hidden", + Document: "", // a publisher stores no contract + Definition: &model.BusinessEventDefinition{ + BaseElement: model.BaseElement{ID: "sp-def"}, + ServiceName: "OrdersEventService", + Channels: []*model.BusinessEventChannel{{ + BaseElement: model.BaseElement{ID: "sp-ch"}, + ChannelName: "f2477dc90f714d69bb12df0df1ee9cce", + Messages: []*model.BusinessEventMessage{{ + BaseElement: model.BaseElement{ID: "sp-msg"}, + MessageName: "OrderPlaceEvent", + Attributes: []*model.BusinessEventAttribute{ + {BaseElement: model.BaseElement{ID: "sp-attr"}, AttributeName: "OrderId", AttributeType: "Long"}, + }, + }}, + }}, + }, + OperationImplementations: []*model.ServiceOperation{{ + BaseElement: model.BaseElement{ID: "sp-op"}, + MessageName: "OrderPlaceEvent", + Operation: "publish", + Entity: "M.PBE_OrderPlaceEvent", + }}, + } + h := mkHierarchy(mod) + withContainer(h, stored.ContainerID, mod.ID) + + var written *model.BusinessEventService + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListBusinessEventServicesFunc: func() ([]*model.BusinessEventService, error) { + return []*model.BusinessEventService{stored}, nil + }, + UpdateBusinessEventServiceFunc: func(svc *model.BusinessEventService) error { + written = svc + return nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + err := createBusinessEventService(ctx, &ast.CreateBusinessEventServiceStmt{ + Name: ast.QualifiedName{Module: "M", Name: "OrdersEventService"}, + ServiceName: "OrdersEventService", + CreateOrModify: true, + Messages: []*ast.BusinessEventMessageDef{{ + MessageName: "OrderPlaceEvent", + Operation: "publish", + Entity: "M.PBE_OrderPlaceEvent", + Attributes: []*ast.BusinessEventAttributeDef{ + {Name: "OrderId", TypeName: "Long"}, + }, + }}, + }) + assertNoError(t, err) + if written == nil { + t.Fatal("a publisher rewrite was refused; the guard keys on the wrong thing") + } +} + +// A statement silent about the handler keeps the stored one, the way a silent +// doc comment does (#1018). MDL has had `MICROFLOW qualifiedName` on the message +// rule all along, so this is a carry gap, not a missing spelling. +func TestCreateOrModifyBusinessEventService_CarriesStoredHandlerMicroflow(t *testing.T) { + mod := mkModule("M") + stored := &model.BusinessEventService{ + BaseElement: model.BaseElement{ID: "sp-svc"}, + ContainerID: mod.ID, + Name: "OrderEvents", + ExportLevel: "Hidden", + Definition: &model.BusinessEventDefinition{ + BaseElement: model.BaseElement{ID: "sp-def"}, + ServiceName: "OrderEvents", + Channels: []*model.BusinessEventChannel{{ + BaseElement: model.BaseElement{ID: "sp-ch"}, + ChannelName: "aaaabbbbccccddddeeeeffff00001111", + Messages: []*model.BusinessEventMessage{{ + BaseElement: model.BaseElement{ID: "sp-msg"}, + MessageName: "OrderPlaced", + Attributes: []*model.BusinessEventAttribute{ + {BaseElement: model.BaseElement{ID: "sp-attr"}, AttributeName: "OrderId", AttributeType: "Long"}, + }, + }}, + }}, + }, + OperationImplementations: []*model.ServiceOperation{{ + BaseElement: model.BaseElement{ID: "sp-op"}, + MessageName: "OrderPlaced", + Operation: "subscribe", + Entity: "M.CBE_OrderPlaced", + Microflow: "M.HandleBE_OrderPlaced", + }}, + } + h := mkHierarchy(mod) + withContainer(h, stored.ContainerID, mod.ID) + + var written *model.BusinessEventService + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListBusinessEventServicesFunc: func() ([]*model.BusinessEventService, error) { + return []*model.BusinessEventService{stored}, nil + }, + UpdateBusinessEventServiceFunc: func(svc *model.BusinessEventService) error { + written = svc + return nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + stmtFor := func(microflow string) *ast.CreateBusinessEventServiceStmt { + return &ast.CreateBusinessEventServiceStmt{ + Name: ast.QualifiedName{Module: "M", Name: "OrderEvents"}, + ServiceName: "OrderEvents", + CreateOrModify: true, + Messages: []*ast.BusinessEventMessageDef{{ + MessageName: "OrderPlaced", + Operation: "subscribe", + Entity: "M.CBE_OrderPlaced", + Microflow: microflow, + Attributes: []*ast.BusinessEventAttributeDef{ + {Name: "OrderId", TypeName: "Long"}, + }, + }}, + } + } + + // Silent statement: the stored handler survives. + assertNoError(t, createBusinessEventService(ctx, stmtFor(""))) + if written == nil || len(written.OperationImplementations) != 1 { + t.Fatalf("no operation written: %+v", written) + } + if got := written.OperationImplementations[0].Microflow; got != "M.HandleBE_OrderPlaced" { + t.Errorf("handler microflow = %q, want the stored M.HandleBE_OrderPlaced", got) + } + + // Control: a statement that names one wins, so the carry is not pinning the + // stored value against an authored change. + written = nil + assertNoError(t, createBusinessEventService(ctx, stmtFor("M.OtherHandler"))) + if got := written.OperationImplementations[0].Microflow; got != "M.OtherHandler" { + t.Errorf("authored handler microflow = %q, want M.OtherHandler", got) + } +} + +// describe printed only `create or modify business event service M.X` for a +// consumer — no property list, no body, no `;` — because the whole statement is +// written inside `if found.Definition != nil`. `mxcli exec` of that output is a +// parse error, so the round trip is broken and an LLM reading it learns nothing. +// +// Until the consumer form has a grammar (proposal part B), describe states the +// contract and the subscriptions as comments: output that parses, and that says +// why there is no statement. +func TestDescribeBusinessEventService_ConsumerDescribesAsParseableComments(t *testing.T) { + for _, lang := range []langver.Version{langver.V0, langver.V1} { + mod := mkModule("M") + stored := storedConsumer(mod.ID) + h := mkHierarchy(mod) + withContainer(h, stored.ContainerID, mod.ID) + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListBusinessEventServicesFunc: func() ([]*model.BusinessEventService, error) { + return []*model.BusinessEventService{stored}, nil + }, + } + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + ctx.describeLang = &lang + assertNoError(t, describeBusinessEventService(ctx, ast.QualifiedName{Module: "M", Name: "OrdersEventClient"})) + + src := buf.String() + if lang == langver.V1 { + src = langver.HeaderFor(langver.V1) + "\n" + src + } + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("mdl %d: consumer describe output does not parse: %v\n%s", lang, errs, src) + } + for _, s := range prog.Statements { + if _, ok := s.(*ast.CreateBusinessEventServiceStmt); ok { + t.Fatalf("mdl %d: consumer describe emitted a create statement; "+ + "re-running it would destroy the contract (#2381)\n%s", lang, src) + } + } + // What the reader needs is in there: the channel id (the topic), the + // subscription, its handler, and why no statement was written. + for _, want := range []string{ + "f2477dc90f714d69bb12df0df1ee9cce", + "OrderPlaceEvent", + "M.CBE_OrderPlaceEvent", + "M.HandleBE_OrderPlaceEvent", + } { + if !strings.Contains(src, want) { + t.Errorf("mdl %d: consumer describe omits %q:\n%s", lang, want, src) + } + } + } +} + +// A handler microflow is authorable (`MICROFLOW qualifiedName` on the message +// rule) but describe never printed one, so describe → exec dropped it on a +// publisher-shaped service too — a loss independent of the consumer case. +func TestDescribeBusinessEventService_EmitsHandlerMicroflow(t *testing.T) { + mod := mkModule("M") + svc := &model.BusinessEventService{ + BaseElement: model.BaseElement{ID: nextID("bes")}, + ContainerID: mod.ID, + Name: "OrderEvents", + Definition: &model.BusinessEventDefinition{ + ServiceName: "OrderEvents", + Channels: []*model.BusinessEventChannel{{ + ChannelName: "ch1", + Messages: []*model.BusinessEventMessage{{ + MessageName: "OrderPlaced", + Attributes: []*model.BusinessEventAttribute{{AttributeName: "OrderId", AttributeType: "Long"}}, + }}, + }}, + }, + OperationImplementations: []*model.ServiceOperation{{ + MessageName: "OrderPlaced", + Operation: "subscribe", + Entity: "M.CBE_OrderPlaced", + Microflow: "M.HandleBE_OrderPlaced", + }}, + } + h := mkHierarchy(mod) + withContainer(h, svc.ContainerID, mod.ID) + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListBusinessEventServicesFunc: func() ([]*model.BusinessEventService, error) { + return []*model.BusinessEventService{svc}, nil + }, + } + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, describeBusinessEventService(ctx, ast.QualifiedName{Module: "M", Name: "OrderEvents"})) + + src := langver.HeaderFor(langver.V1) + "\n" + buf.String() + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("describe output does not parse: %v\n%s", errs, src) + } + var stmt *ast.CreateBusinessEventServiceStmt + for _, s := range prog.Statements { + if c, ok := s.(*ast.CreateBusinessEventServiceStmt); ok { + stmt = c + } + } + if stmt == nil { + t.Fatalf("no business event service in:\n%s", src) + } + if len(stmt.Messages) != 1 { + t.Fatalf("re-parsed %d messages, want 1\n%s", len(stmt.Messages), src) + } + if got := stmt.Messages[0].Microflow; got != "M.HandleBE_OrderPlaced" { + t.Errorf("re-parsed handler microflow = %q, want M.HandleBE_OrderPlaced\n%s", got, src) + } +} From c6a057292a344d6ebb2570d8fdd6aca2df57d387 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 05:38:50 +0000 Subject: [PATCH 5/9] check: refuse a primitive where a list operation needs an object or a list (CE0023) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_011oK82tETa46YQmMS58pQjL --- ...6-11-18-list-operand-primitive-ce0023.json | 1 + CHANGELOG.md | 1 + cmd/mxcli/syntax/features_microflow.go | 5 +- .../bug-tests/list-operand-primitive.fail.mdl | 37 +++++ .../bug-tests/list-operand-primitive.mdl | 23 +++ mdl/executor/validate_list_operand_kinds.go | 108 ++++++++++++++ .../validate_list_operand_kinds_test.go | 139 ++++++++++++++++++ mdl/executor/validate_microflow.go | 2 + mdl/executor/validate_nanoflow.go | 1 + 9 files changed, 316 insertions(+), 1 deletion(-) create mode 100644 .claude/skills/fix-issue/findings/mdl-executor/2026-11-18-list-operand-primitive-ce0023.json create mode 100644 mdl-examples/bug-tests/list-operand-primitive.fail.mdl create mode 100644 mdl-examples/bug-tests/list-operand-primitive.mdl create mode 100644 mdl/executor/validate_list_operand_kinds.go create mode 100644 mdl/executor/validate_list_operand_kinds_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-list-operand-primitive-ce0023.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-list-operand-primitive-ce0023.json new file mode 100644 index 000000000..26b3620ce --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-list-operand-primitive-ce0023.json @@ -0,0 +1 @@ +{"area": "mdl/executor", "date": "2026-11-18", "refs": ["ako/mxcli#1258"], "ce": ["CE0023"], "symptom": "`$R = UNION $L WITH $S` (and INTERSECT, SUBTRACT, EQUALS, `CONTAINS $S IN $L`) with a primitive variable as the operand that is not the list passed `check` and `exec`, and the build reported CE0023 \"Selected variable 'S' must be of type Object or List.\" The same shape with an Object or List operand built clean.", "file": "mdl/executor/validate_list_operand_kinds.go checkListOperandKinds (MDL101), called from validate_microflow.go and validate_nanoflow.go after the primitive kinds are seeded. Tests: validate_list_operand_kinds_test.go; fixtures mdl-examples/bug-tests/list-operand-primitive(.fail).mdl", "cause": "The only CE0023 findings were the `contains` String overload (ledger #53), which the builder rewrites into a Change Variable; nothing judged the operand type of a list operation that really is one. The validators already had the kind of every parameter and declare (varKinds) and used it for MDL063 and the numeric-assignment rules, but not here.", "insight": "**When a code is in the field-hit list only because of an overload workaround, measure the general shape the workaround stands next to: the operand-type matrix was cheaper than guessing the boundary.** One project, one flow per case, `mx check -j` read per document (the `locations[].document-name` field, not the error object): 42 first-and-second-operand cases, then 25 more over Boolean, Decimal, DateTime, Long and enumeration. Result: for UNION, INTERSECT, SUBTRACT, EQUALS and CONTAINS the second operand being any primitive is CE0023 (7 types x 5 operations = 35 of 35, nanoflows the same), an Object or List there is clean (`union $L with $O` builds), a wrong FIRST operand is CE0097 and COUNT/SUM over a non-list is CE0013 (both left for their own rules; `range $S limit 2` was clean). Staging did not interfere because every flow was its own document. Visitor mapping to remember: `SUBTRACT $A FROM $B` and `CONTAINS $A IN $B` store the LIST in InputVariable and the other operand in SecondVariable. An enumeration parameter `LO.Color` is indistinguishable from an object at parse time, so it is silent (5 of 63 measured cases, listed in the PR); a declared `enum` variable is judged. Sweep: 1,317 repo .mdl files, 0 MDL101 hits. Control: stubbing the check fails 60 sub-tests; the String `contains` function still passes."} diff --git a/CHANGELOG.md b/CHANGELOG.md index 80bc8983d..2d84e77f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Fixed +- **`check` rejects a primitive where a list operation needs an object or a list** (ako/mxcli#1258) — `$R = UNION $L WITH $S`, and the same for `INTERSECT`, `SUBTRACT`, `EQUALS` and `CONTAINS $S IN $L`, with a String, Integer, Long, Decimal, Boolean or DateTime variable as the operand that is not the list, passed `check` and `exec` and the build reported CE0023 "Selected variable '…' must be of type Object or List." New MDL101, in microflows and nanoflows. An Object or a List there is fine, `contains($Hay, $Needle)` over two Strings is still the string function, and an enumeration parameter is not judged (it reads as an entity type until the project says otherwise). Measured on MxBuild 11.16.1. - **`check` rejects a mapping element whose association does not connect it to its parent** (ako/mxcli#1258) — in an import or export mapping, `Assoc/Entity` has to connect the entity the element sits under to its own entity. An association between other entities, or one that touches only the parent or only the element's entity, passed `check` and `exec` and the build reported CE0295 "Association '...' is not allowed". New MDL100, which reads the association from the script or the project and follows generalizations of either entity (measured clean); an association it cannot find is never refused. A JSON object element over a to-many association is also CE0295 and is not covered. Measured on MxBuild 11.16.1, import and export. - **`check` rejects `owner both` between an entity and itself or its generalization** (ako/mxcli#1258) — `create association M.A_A from M.A to M.A owner both`, or from a specialization to its parent (either direction, any chain, Reference or ReferenceSet), passed `check` and `exec` and the build reported CE0002. New MDL099, which needs no project; an unrelated pair and every other owner are unaffected. Only generalizations the script declares are followed. Measured on MxBuild 11.16.1. - **The catalog cache sees a write made in the same second as the build** (ako/mxcli#2374) — freshness compared the project file's modification time as whole seconds, so a read that followed a write inside the second of the previous catalog build answered from the catalog as it was before the write (`marketplace update` finding no dependent associations about one run in six; `describe` and catalog queries could do the same). The time is now stored and compared to the nanosecond. A cache written by an older mxcli holds whole seconds and is rebuilt once per project. diff --git a/cmd/mxcli/syntax/features_microflow.go b/cmd/mxcli/syntax/features_microflow.go index cc888cf34..5f4db8fc7 100644 --- a/cmd/mxcli/syntax/features_microflow.go +++ b/cmd/mxcli/syntax/features_microflow.go @@ -530,7 +530,7 @@ func init() { "reduce", "all", "any", "fold", // A CE number in the cached `syntax --json` index leads an agent // debugging the build error back to the right topic (issue #1002). - "$currentObject", "predicate", "CE0117", "CE0109", "MDL-LISTOP01", + "$currentObject", "predicate", "CE0117", "CE0109", "CE0023", "MDL101", "MDL-LISTOP01", "contains", "equals", "by", "where", "MDL-DEPR003", "MDL-DEPR004", "MDL-V1-LIST", }, Syntax: "$List = CREATE LIST OF Module.Entity;\n" + @@ -565,6 +565,9 @@ func init() { "-- RANGE $L LIMIT $Amount first $Amount\n" + "-- RANGE $L OFFSET $Offset skip $Offset, take the rest\n" + "-- RANGE with no bound is CE6520 at build time (mxcli check: MDL068).\n\n" + + "-- UNION, INTERSECT, SUBTRACT, EQUALS and CONTAINS take an Object or a List as\n" + + "-- the operand that is not the list; a String, Integer, Boolean or other\n" + + "-- primitive variable there is CE0023 at build time (mxcli check: MDL101).\n\n" + "BY names a member and the value it must have; anything else goes after\n" + "WHERE, which is evaluated once per item with the item bound to\n" + "$currentObject -- the same variable the aggregate expressions use:\n\n" + diff --git a/mdl-examples/bug-tests/list-operand-primitive.fail.mdl b/mdl-examples/bug-tests/list-operand-primitive.fail.mdl new file mode 100644 index 000000000..631d60f4c --- /dev/null +++ b/mdl-examples/bug-tests/list-operand-primitive.fail.mdl @@ -0,0 +1,37 @@ +mdl 1; +-- NEGATIVE TEST — `mxcli check` must REFUSE this file (MDL101), with no project. +-- +-- UNION, INTERSECT, SUBTRACT, EQUALS and CONTAINS take an Object or a List as +-- the operand that is not the list. A variable of a primitive type there passed +-- `check` and `exec`, was written as a list operation, and the build reported +-- +-- [CE0023] "Selected variable 'X' must be of type Object or List." +-- +-- Measured on MxBuild 11.16.1, microflows and nanoflows, for String, Integer, +-- Long, Decimal, Boolean, DateTime and enumeration alike. An Object or a List in +-- the same position is clean (see list-operand-primitive.mdl). +create persistent entity MyFirstModule.LoItem (Name: string(20)); + +create microflow MyFirstModule.LoUnionString ($L: list of MyFirstModule.LoItem, $X: string) begin + $R = UNION $L WITH $X; +end; + +create microflow MyFirstModule.LoIntersectInteger ($L: list of MyFirstModule.LoItem, $X: integer) begin + $R = INTERSECT $L WITH $X; +end; + +create microflow MyFirstModule.LoSubtractDecimal ($L: list of MyFirstModule.LoItem, $X: decimal) begin + $R = SUBTRACT $X FROM $L; +end; + +create microflow MyFirstModule.LoEqualsBoolean ($L: list of MyFirstModule.LoItem, $X: boolean) begin + $R = EQUALS $L AND $X; +end; + +create microflow MyFirstModule.LoContainsDateTime ($L: list of MyFirstModule.LoItem, $X: datetime) begin + $R = CONTAINS $X IN $L; +end; + +create nanoflow MyFirstModule.LoUnionLong ($L: list of MyFirstModule.LoItem, $X: long) begin + $R = UNION $L WITH $X; +end; diff --git a/mdl-examples/bug-tests/list-operand-primitive.mdl b/mdl-examples/bug-tests/list-operand-primitive.mdl new file mode 100644 index 000000000..666fbc512 --- /dev/null +++ b/mdl-examples/bug-tests/list-operand-primitive.mdl @@ -0,0 +1,23 @@ +mdl 1; +-- Control for list-operand-primitive.fail.mdl: an Object or a List in the position +-- where a primitive is refused, and the String function `contains($Hay, $Needle)`, +-- which parses as the same node and builds as a Change Variable. Builds with 0 errors +-- on MxBuild 11.16.1. +create persistent entity MyFirstModule.LpItem (Name: string(20)); + +create microflow MyFirstModule.LpUnionObject ($L: list of MyFirstModule.LpItem, $O: MyFirstModule.LpItem) begin + $R = UNION $L WITH $O; +end; + +create microflow MyFirstModule.LpUnionList ($L: list of MyFirstModule.LpItem, $M: list of MyFirstModule.LpItem) begin + $R = UNION $L WITH $M; +end; + +create microflow MyFirstModule.LpContainsObject ($L: list of MyFirstModule.LpItem, $O: MyFirstModule.LpItem) begin + $R = CONTAINS $O IN $L; +end; + +create microflow MyFirstModule.LpStringContains ($Hay: string, $Needle: string) begin + declare $M boolean = false; + set $M = contains($Hay, $Needle); +end; diff --git a/mdl/executor/validate_list_operand_kinds.go b/mdl/executor/validate_list_operand_kinds.go new file mode 100644 index 000000000..68119a67a --- /dev/null +++ b/mdl/executor/validate_list_operand_kinds.go @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/exprcheck" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// listOperandKindRule is the rule ID for "a primitive where a list operation +// needs an object or a list". +const listOperandKindRule = "MDL101" + +// checkListOperandKinds reports a list operation whose second operand is a +// variable of a primitive type — CE0023 "Selected variable 'X' must be of type +// Object or List." +// +// Measured on MxBuild 11.16.1, microflows and nanoflows, one flow per case: for +// UNION, INTERSECT, SUBTRACT, EQUALS and CONTAINS the operand that is not the +// list must be an Object or a List, and every primitive type — String, Integer, +// Long, Decimal, Boolean, DateTime, enumeration — is CE0023 there. An Object or +// a List is clean (`union $L with $O` builds). `check` and `exec` accepted every +// rejected case and wrote the activity. +// +// Only a parameter or declared variable of a known primitive kind is judged. A +// variable whose kind is not known here is never reported, and a primitive FIRST +// operand is a different error (CE0097), left to its own rule. +// +// `set $M = contains($Hay, $Needle)` over two Strings parses as the same node, +// and builds as the String function (addListOperationAction rewrites it when the +// haystack is a declared String): the needle being a String is right there, so a +// String haystack is skipped, by the same test the builder and MDL063 share. +func (v *microflowValidator) checkListOperandKinds(body []ast.MicroflowStatement) { + forEachMicroflowStatement(body, func(s ast.MicroflowStatement) { + lo, ok := s.(*ast.ListOperationStmt) + if !ok || lo.SecondVariable == "" { + return + } + switch lo.Operation { + case ast.ListOpUnion, ast.ListOpIntersect, ast.ListOpSubtract, ast.ListOpEquals: + case ast.ListOpContains: + if stringOverloadedListOp(lo.Operation) && v.varKinds[lo.InputVariable] == exprcheck.KindString { + return + } + default: + return + } + kind, known := v.varKinds[lo.SecondVariable] + if !known || !isPrimitiveOperandKind(kind) { + return + } + v.addViolation(listOperandKindRule, linter.SeverityError, + fmt.Sprintf("$%s is a %s, but %s needs an object or a list there — mxbuild rejects this with "+ + "CE0023 \"Selected variable '%s' must be of type Object or List.\" (measured for every primitive type)", + lo.SecondVariable, kindName(kind), listOperationWord(lo.Operation), lo.SecondVariable), + "Pass an object or a list variable; to test a primitive value, compare it in an expression instead") + }) +} + +// isPrimitiveOperandKind is the set measured as CE0023: Binary is left out +// because it was not measured, and anything else is not a primitive value. +func isPrimitiveOperandKind(k exprcheck.TypeKind) bool { + switch k { + case exprcheck.KindString, exprcheck.KindInteger, exprcheck.KindLong, exprcheck.KindDecimal, + exprcheck.KindBoolean, exprcheck.KindDateTime, exprcheck.KindEnumeration: + return true + } + return false +} + +func kindName(k exprcheck.TypeKind) string { + switch k { + case exprcheck.KindString: + return "String" + case exprcheck.KindInteger: + return "Integer" + case exprcheck.KindLong: + return "Long" + case exprcheck.KindDecimal: + return "Decimal" + case exprcheck.KindBoolean: + return "Boolean" + case exprcheck.KindDateTime: + return "DateTime" + case exprcheck.KindEnumeration: + return "enumeration value" + } + return "primitive value" +} + +func listOperationWord(op ast.ListOperationType) string { + switch op { + case ast.ListOpUnion: + return "UNION" + case ast.ListOpIntersect: + return "INTERSECT" + case ast.ListOpSubtract: + return "SUBTRACT" + case ast.ListOpEquals: + return "EQUALS" + case ast.ListOpContains: + return "CONTAINS" + } + return "this list operation" +} diff --git a/mdl/executor/validate_list_operand_kinds_test.go b/mdl/executor/validate_list_operand_kinds_test.go new file mode 100644 index 000000000..b31faea03 --- /dev/null +++ b/mdl/executor/validate_list_operand_kinds_test.go @@ -0,0 +1,139 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/linter" + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// CE0023 "Selected variable 'X' must be of type Object or List." Measured on +// MxBuild 11.16.1, microflows and nanoflows, one flow per case: the operand that +// is not the list — the second operand of UNION, INTERSECT, SUBTRACT, EQUALS, and +// the object of CONTAINS — is CE0023 for every primitive type (String, Integer, +// Long, Decimal, Boolean, DateTime, enumeration). An Object or a List there is +// clean. An enumeration PARAMETER is not judged: `LO.Color` reads as an entity +// type until the project says otherwise, so it is never reported. `check` and `exec` accepted every rejected case. +var listOperandOps = []struct{ name, tpl string }{ + {"union", "$R = UNION $L WITH $X;"}, + {"intersect", "$R = INTERSECT $L WITH $X;"}, + {"subtract", "$R = SUBTRACT $X FROM $L;"}, + {"equals", "$R = EQUALS $L AND $X;"}, + {"contains", "$R = CONTAINS $X IN $L;"}, +} + +var listOperandPrimitives = []struct{ name, typ string }{ + {"string", "String"}, {"integer", "Integer"}, {"long", "Long"}, {"decimal", "Decimal"}, + {"boolean", "Boolean"}, {"datetime", "DateTime"}, +} + +func listOperandFlow(t *testing.T, kind, xType, body string) []linter.Violation { + t.Helper() + src := fmt.Sprintf("mdl 1;\ncreate %s LO.F ($L: list of LO.Item, $X: %s) begin\n %s\nend;\n", kind, xType, body) + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse: %v\n%s", errs, src) + } + switch s := prog.Statements[0].(type) { + case *ast.CreateMicroflowStmt: + return ValidateMicroflow(s) + case *ast.CreateNanoflowStmt: + return ValidateNanoflow(s) + } + t.Fatalf("unexpected %T", prog.Statements[0]) + return nil +} + +func listOperandHits(vs []linter.Violation) []linter.Violation { + var out []linter.Violation + for _, v := range vs { + if v.RuleID == listOperandKindRule { + out = append(out, v) + } + } + return out +} + +func TestListOperandKinds_PrimitiveWhereObjectOrListIsRequired(t *testing.T) { + for _, kind := range []string{"microflow", "nanoflow"} { + for _, op := range listOperandOps { + for _, p := range listOperandPrimitives { + t.Run(kind+"/"+op.name+"/"+p.name, func(t *testing.T) { + hits := listOperandHits(listOperandFlow(t, kind, p.typ, op.tpl)) + if len(hits) != 1 { + t.Fatalf("want one MDL101 violation, got %v", hits) + } + if !strings.Contains(hits[0].Message, "CE0023") || !strings.Contains(hits[0].Message, "$X") { + t.Errorf("message should name CE0023 and the variable: %s", hits[0].Message) + } + }) + } + } + } +} + +// Controls: an Object or a List in the same position is clean on MxBuild, and +// so must not be reported. +func TestListOperandKinds_ObjectAndListOperandsAreFine(t *testing.T) { + for _, op := range listOperandOps { + for _, x := range []string{"LO.Item", "list of LO.Item"} { + t.Run(op.name+"/"+x, func(t *testing.T) { + if hits := listOperandHits(listOperandFlow(t, "microflow", x, op.tpl)); len(hits) != 0 { + t.Errorf("an object or list operand was reported: %v", hits) + } + }) + } + } +} + +// `$M = contains($Hay, $Needle)` over two Strings parses as a list operation and +// builds as the String function (ledger #53, addListOperationAction): the +// needle being a String is exactly right there. +func TestListOperandKinds_StringContainsIsNotAListOperation(t *testing.T) { + src := "mdl 1;\ncreate microflow LO.F ($Hay: String, $Needle: String) begin\n declare $M Boolean = false;\n set $M = contains($Hay, $Needle);\nend;\n" + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatal(errs) + } + if hits := listOperandHits(ValidateMicroflow(prog.Statements[0].(*ast.CreateMicroflowStmt))); len(hits) != 0 { + t.Errorf("string contains was reported as a list operation: %v", hits) + } +} + +// A variable whose kind is not known here (not a parameter or a declare) is never +// reported, and neither is a wrong FIRST operand: that is CE0097, a different rule. +func TestListOperandKinds_OnlyTheSecondOperandAndOnlyWhenKnown(t *testing.T) { + unknown := "mdl 1;\ncreate microflow LO.F ($L: list of LO.Item) begin\n $R = UNION $L WITH $Mystery;\nend;\n" + prog, errs := visitor.Build(unknown) + if len(errs) > 0 { + t.Fatal(errs) + } + if hits := listOperandHits(ValidateMicroflow(prog.Statements[0].(*ast.CreateMicroflowStmt))); len(hits) != 0 { + t.Errorf("a variable of unknown kind was reported: %v", hits) + } + if hits := listOperandHits(listOperandFlow(t, "microflow", "String", "$R = UNION $X WITH $L;")); len(hits) != 0 { + t.Errorf("a primitive FIRST operand is CE0097, not this rule: %v", hits) + } +} + +// An enumeration is CE0023 too (measured), but only a variable the script says is +// an enumeration can be told from an object here. +func TestListOperandKinds_EnumerationVariable(t *testing.T) { + declared := "mdl 1;\ncreate microflow LO.F ($L: list of LO.Item) begin\n declare $X enum LO.Color = LO.Color.Red;\n $R = UNION $L WITH $X;\nend;\n" + prog, errs := visitor.Build(declared) + if len(errs) > 0 { + t.Fatal(errs) + } + if hits := listOperandHits(ValidateMicroflow(prog.Statements[0].(*ast.CreateMicroflowStmt))); len(hits) != 1 { + t.Errorf("a declared enumeration variable should be reported once, got %v", hits) + } + // The same name as a parameter is indistinguishable from an object: silent. + if hits := listOperandHits(listOperandFlow(t, "microflow", "LO.Color", "$R = UNION $L WITH $X;")); len(hits) != 0 { + t.Errorf("an enumeration parameter cannot be told from an object and must not be reported: %v", hits) + } +} diff --git a/mdl/executor/validate_microflow.go b/mdl/executor/validate_microflow.go index 1f8e0f6c2..a7a671a36 100644 --- a/mdl/executor/validate_microflow.go +++ b/mdl/executor/validate_microflow.go @@ -189,6 +189,8 @@ func (v *microflowValidator) validate(body []ast.MicroflowStatement) { // by the build. See validate_microflow_ce_gaps.go for the measurements. v.checkReturnInLoop(body) v.checkDuplicateVariableNames(v.params, body) + // CE0023: a primitive where UNION/INTERSECT/SUBTRACT/EQUALS/CONTAINS need an object or list. + v.checkListOperandKinds(body) // ako/mxcli#962: the other half of #953's void-call finding — the name is // inert both ways, so reading it is CE0109. See validate_void_call_output.go. v.checkVoidCallOutputUse(v.params, body) diff --git a/mdl/executor/validate_nanoflow.go b/mdl/executor/validate_nanoflow.go index 0b05bb4ea..c3b7a1b6e 100644 --- a/mdl/executor/validate_nanoflow.go +++ b/mdl/executor/validate_nanoflow.go @@ -49,6 +49,7 @@ func validateNanoflowWith(stmt *ast.CreateNanoflowStmt, voids *voidCodeActions) // contains/find rewrite come from the parameters and the declares. v.seedPrimitiveKinds(stmt.Parameters, stmt.Body) v.checkDuplicateVariableNames(v.params, stmt.Body) + v.checkListOperandKinds(stmt.Body) // A nanoflow's show message has the same ShowMessageType (mendixlabs/mxcli#1280). v.checkShowMessageType(stmt.Body) // MDL-TEMPLATE01: a nanoflow's log, show message and validation feedback From 73cecec333f70fe7173e5ac4324badaf9603a6df Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 06:29:12 +0000 Subject: [PATCH 6/9] check: refuse an alter that leaves a published OData attribute published as the wrong type (CE5016) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_011oK82tETa46YQmMS58pQjL --- ...ublished-attribute-type-change-ce5016.json | 1 + CHANGELOG.md | 1 + cmd/mxcli/syntax/features_domain_model.go | 2 +- .../published-attribute-type-change.fail.mdl | 33 +++ .../published-attribute-type-change.mdl | 38 +++ mdl/executor/validate_program.go | 2 + .../validate_published_attribute_types.go | 260 ++++++++++++++++++ ...validate_published_attribute_types_test.go | 184 +++++++++++++ 8 files changed, 520 insertions(+), 1 deletion(-) create mode 100644 .claude/skills/fix-issue/findings/mdl-executor/2026-11-18-published-attribute-type-change-ce5016.json create mode 100644 mdl-examples/bug-tests/published-attribute-type-change.fail.mdl create mode 100644 mdl-examples/bug-tests/published-attribute-type-change.mdl create mode 100644 mdl/executor/validate_published_attribute_types.go create mode 100644 mdl/executor/validate_published_attribute_types_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-published-attribute-type-change-ce5016.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-published-attribute-type-change-ce5016.json new file mode 100644 index 000000000..b2fd738c2 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-published-attribute-type-change-ce5016.json @@ -0,0 +1 @@ +{"area": "mdl/executor", "date": "2026-11-18", "refs": ["ako/mxcli#1258"], "ce": ["CE5016"], "symptom": "`alter entity M.E modify attribute A: ` on an attribute a published OData service exposes passed `check` and `exec`; `mx check` then reported one CE5016 \"Attribute 'A' has type …, but is published as …\" per attribute. Measured: String to Integer, Decimal to Integer, Boolean to String, DateTime to String, String to Decimal. String(50) to String(80) was clean.", "file": "mdl/executor/validate_published_attribute_types.go ValidatePublishedAttributeTypes (MDL102), called from validate_program.go with the lazy project opener. Tests: validate_published_attribute_types_test.go; fixtures mdl-examples/bug-tests/published-attribute-type-change(.fail).mdl", "cause": "The publisher derives each attribute's EdmType once, when the service is written (mendixAttrTypeToEdm, adjudicated on 11.12.1), and nothing revisits it when the attribute's type later changes. The six earlier CE5016 findings were all that derivation being wrong; this is the derivation going stale.", "insight": "**When a derived value is stored beside its source, the gap is the edit that changes the source later, and the order of statements in a script decides whether it is a defect.** Measured on MxBuild 11.16.1 in copies of one project that started at 0 errors: `alter` then `create or modify published odata service` is 0 errors (the re-publish re-derives every type); the same two statements the other way round is CE5016. So the rule follows the script in order and clears when a later statement re-publishes, alters or drops the service, and stays silent when it cannot prove staleness. The first version passed its own tests and still missed the re-publish-then-alter case over a PROJECT entity, because the script never declared the attribute so its type was unknown and the entry was dropped: the end-to-end run against the measured scripts found it (9 scripts, all agree after the project-type fallback), the unit tests did not. Enum parameters, `rename attribute` and `drop attribute` (a different CE) are not covered. Sweep: 1,317 repo .mdl files, 0 MDL102 hits. Control: stubbing the check fails the script and project tests; stubbing only the project-type fallback fails the re-publish-then-alter test."} diff --git a/CHANGELOG.md b/CHANGELOG.md index 80bc8983d..16beeb90f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Fixed +- **`check` rejects an `alter entity … modify attribute` that leaves a published OData attribute published as the wrong type** (ako/mxcli#1258) — changing the type of an attribute a published service exposes (String to Integer, Decimal to Integer, Boolean or DateTime to String, String to Decimal) passed `check` and `exec` and the build reported CE5016 "Attribute '…' has type …, but is published as …", one per attribute. New MDL102, which reads the service from the script and from the project. A change inside one EDM type (String(50) to String(80); Integer to Long) is fine, and so is a `create or modify published odata service` AFTER the alter, which re-derives the types — before the alter it does not help. Measured on MxBuild 11.16.1. - **`check` rejects a mapping element whose association does not connect it to its parent** (ako/mxcli#1258) — in an import or export mapping, `Assoc/Entity` has to connect the entity the element sits under to its own entity. An association between other entities, or one that touches only the parent or only the element's entity, passed `check` and `exec` and the build reported CE0295 "Association '...' is not allowed". New MDL100, which reads the association from the script or the project and follows generalizations of either entity (measured clean); an association it cannot find is never refused. A JSON object element over a to-many association is also CE0295 and is not covered. Measured on MxBuild 11.16.1, import and export. - **`check` rejects `owner both` between an entity and itself or its generalization** (ako/mxcli#1258) — `create association M.A_A from M.A to M.A owner both`, or from a specialization to its parent (either direction, any chain, Reference or ReferenceSet), passed `check` and `exec` and the build reported CE0002. New MDL099, which needs no project; an unrelated pair and every other owner are unaffected. Only generalizations the script declares are followed. Measured on MxBuild 11.16.1. - **The catalog cache sees a write made in the same second as the build** (ako/mxcli#2374) — freshness compared the project file's modification time as whole seconds, so a read that followed a write inside the second of the previous catalog build answered from the catalog as it was before the write (`marketplace update` finding no dependent associations about one run in six; `describe` and catalog queries could do the same). The time is now stored and compared to the nanosecond. A cache written by an older mxcli holds whole seconds and is rebuilt once per project. diff --git a/cmd/mxcli/syntax/features_domain_model.go b/cmd/mxcli/syntax/features_domain_model.go index 109469729..1e4305614 100644 --- a/cmd/mxcli/syntax/features_domain_model.go +++ b/cmd/mxcli/syntax/features_domain_model.go @@ -99,7 +99,7 @@ func init() { "if not exists", "if exists", "idempotent", "alter entities", "bulk", "every entity", "all entities", "where persistent", }, - Syntax: "ALTER ENTITY Module.Name ADD ATTRIBUTE [IF NOT EXISTS] AttrName: Type [constraints];\nALTER ENTITY Module.Name DROP ATTRIBUTE [IF EXISTS] AttrName;\nALTER ENTITY Module.Name RENAME ATTRIBUTE OldName TO NewName;\nALTER ENTITY Module.Name MODIFY ATTRIBUTE AttrName: Type [DEFAULT val];\nALTER ENTITY Module.Name DROP DEFAULT ON ATTRIBUTE AttrName;\nALTER ENTITY Module.Name ADD INDEX [name] [ON] (attr1, attr2);\nALTER ENTITY Module.Name SET DOCUMENTATION 'text';\nALTER ENTITY Module.Name SET POSITION (x, y);\nALTER ENTITY Module.Name SET EXPORT LEVEL API|HIDDEN; -- see domain-model.export-level\nALTER ENTITY Module.Name SET (AllowCreateChangeLocally: true|false); -- external entity\nALTER ENTITY Module.Name ADD EVENT HANDLER [IF NOT EXISTS] ON BEFORE COMMIT CALL Module.MF RAISE ERROR;\nALTER ENTITY Module.Name DROP EVENT HANDLER [IF EXISTS] ON BEFORE COMMIT [CALL Module.MF];\nALTER ENTITIES [IN Module] ADD ATTRIBUTE [IF NOT EXISTS] AttrName: Type [, ...]\n [WHERE PERSISTENT | WHERE NON-PERSISTENT];\n\nALTER ENTITIES is the bulk form: one statement applied to every entity in a\nmodule instead of one statement per entity. Only ADD ATTRIBUTE is offered --\nDROP and RENAME aimed at a set are destructive by a typo, and SET POSITION on\nevery entity is meaningless. Pair it with IF NOT EXISTS so the script re-runs.\n\nWHERE filters by persistence, using the same words CREATE ENTITY uses. A VIEW\nentity matches NEITHER: its rows come from an OQL query, so it is not the\npersistent/non-persistent distinction this filter means.\n\nWITHOUT IN, the sweep covers the whole project but SKIPS System and every\nMarketplace module, reporting which -- an upgrade replaces those modules and\nwould take the attribute with it. Naming a module with IN is taken as meaning\nit, so a deliberate edit there is still possible.\n\nSET POSITION places the entity in the domain-model editor, and CREATE ENTITY\ntakes the same thing as an @Position(x, y) annotation. Both are the box's\nCENTRE, not its top-left corner. An entity created without one takes the next\nslot in a wrapping grid, which is a default rather than a layout: to arrange a\nwhole module from its association graph, run 'mxcli layout -p app.mpr'\n(--dry-run first; it replaces positions you set by hand).\n\nMODIFY ATTRIBUTE always takes a type — restate it even when you are only\nchanging the default. There is no 'MODIFY ATTRIBUTE X SET DEFAULT v' form:\nSET would be read as the type name. Use DROP DEFAULT to clear one.\n\nIF NOT EXISTS / IF EXISTS make the add/drop a no-op (skipped, not an error)\nwhen the attribute is already present / already gone — so a domain script\nre-runs cleanly. For a whole script, 'mxcli exec --continue-on-error' reports\neach failed statement and keeps going instead of halting at the first.\n\nAn entity may have several handlers for one event, as in Studio Pro: a handler\nis identified by its moment, event AND microflow, so ADD EVENT HANDLER with\nanother microflow adds a second one, and only the same microflow again is a\nduplicate. DROP EVENT HANDLER takes CALL Module.MF to say which one to remove;\nwithout it, a drop that matches several handlers is refused and names them.\n\nRENAME ATTRIBUTE also rewrites every reference to the attribute: the stored\nqualified names (microflow create/change members, page widgets, the entity's own\nvalidation and access rules) AND the bare steps inside XPath constraints, which\nare resolved to their owning entity first so another entity's identically-named\nattribute is left alone. A constraint that cannot be resolved is reported and\nleft unchanged, never guessed at. Uses inside microflow expressions ($obj/Attr)\nare free text and are NOT rewritten; mxbuild reports those as CE0117.", + Syntax: "ALTER ENTITY Module.Name ADD ATTRIBUTE [IF NOT EXISTS] AttrName: Type [constraints];\nALTER ENTITY Module.Name DROP ATTRIBUTE [IF EXISTS] AttrName;\nALTER ENTITY Module.Name RENAME ATTRIBUTE OldName TO NewName;\nALTER ENTITY Module.Name MODIFY ATTRIBUTE AttrName: Type [DEFAULT val];\nALTER ENTITY Module.Name DROP DEFAULT ON ATTRIBUTE AttrName;\nALTER ENTITY Module.Name ADD INDEX [name] [ON] (attr1, attr2);\nALTER ENTITY Module.Name SET DOCUMENTATION 'text';\nALTER ENTITY Module.Name SET POSITION (x, y);\nALTER ENTITY Module.Name SET EXPORT LEVEL API|HIDDEN; -- see domain-model.export-level\nALTER ENTITY Module.Name SET (AllowCreateChangeLocally: true|false); -- external entity\nALTER ENTITY Module.Name ADD EVENT HANDLER [IF NOT EXISTS] ON BEFORE COMMIT CALL Module.MF RAISE ERROR;\nALTER ENTITY Module.Name DROP EVENT HANDLER [IF EXISTS] ON BEFORE COMMIT [CALL Module.MF];\nALTER ENTITIES [IN Module] ADD ATTRIBUTE [IF NOT EXISTS] AttrName: Type [, ...]\n [WHERE PERSISTENT | WHERE NON-PERSISTENT];\n\nALTER ENTITIES is the bulk form: one statement applied to every entity in a\nmodule instead of one statement per entity. Only ADD ATTRIBUTE is offered --\nDROP and RENAME aimed at a set are destructive by a typo, and SET POSITION on\nevery entity is meaningless. Pair it with IF NOT EXISTS so the script re-runs.\n\nWHERE filters by persistence, using the same words CREATE ENTITY uses. A VIEW\nentity matches NEITHER: its rows come from an OQL query, so it is not the\npersistent/non-persistent distinction this filter means.\n\nWITHOUT IN, the sweep covers the whole project but SKIPS System and every\nMarketplace module, reporting which -- an upgrade replaces those modules and\nwould take the attribute with it. Naming a module with IN is taken as meaning\nit, so a deliberate edit there is still possible.\n\nSET POSITION places the entity in the domain-model editor, and CREATE ENTITY\ntakes the same thing as an @Position(x, y) annotation. Both are the box's\nCENTRE, not its top-left corner. An entity created without one takes the next\nslot in a wrapping grid, which is a default rather than a layout: to arrange a\nwhole module from its association graph, run 'mxcli layout -p app.mpr'\n(--dry-run first; it replaces positions you set by hand).\n\nMODIFY ATTRIBUTE always takes a type — restate it even when you are only\nchanging the default. There is no 'MODIFY ATTRIBUTE X SET DEFAULT v' form:\nSET would be read as the type name. Use DROP DEFAULT to clear one.\n\nChanging the type of an attribute a published OData service exposes leaves the\npublished type as it was, and mxbuild reports CE5016 unless both types publish\nas the same EDM type (String(50) to String(80); Integer to Long). Re-publish the\nservice AFTER the alter with CREATE OR MODIFY PUBLISHED ODATA SERVICE, which\nre-derives the types; the same statement before the alter does not help (mxcli\ncheck: MDL102).\n\nIF NOT EXISTS / IF EXISTS make the add/drop a no-op (skipped, not an error)\nwhen the attribute is already present / already gone — so a domain script\nre-runs cleanly. For a whole script, 'mxcli exec --continue-on-error' reports\neach failed statement and keeps going instead of halting at the first.\n\nAn entity may have several handlers for one event, as in Studio Pro: a handler\nis identified by its moment, event AND microflow, so ADD EVENT HANDLER with\nanother microflow adds a second one, and only the same microflow again is a\nduplicate. DROP EVENT HANDLER takes CALL Module.MF to say which one to remove;\nwithout it, a drop that matches several handlers is refused and names them.\n\nRENAME ATTRIBUTE also rewrites every reference to the attribute: the stored\nqualified names (microflow create/change members, page widgets, the entity's own\nvalidation and access rules) AND the bare steps inside XPath constraints, which\nare resolved to their owning entity first so another entity's identically-named\nattribute is left alone. A constraint that cannot be resolved is reported and\nleft unchanged, never guessed at. Uses inside microflow expressions ($obj/Attr)\nare free text and are NOT rewritten; mxbuild reports those as CE0117.", Example: "mdl 1;\nALTER ENTITY Shop.Customer ADD ATTRIBUTE Phone: String(20);\nALTER ENTITY Shop.Customer ADD ATTRIBUTE IF NOT EXISTS Phone: String(20); -- re-runnable\nALTER ENTITY Shop.Customer DROP ATTRIBUTE IF EXISTS OldField; -- re-runnable\nALTER ENTITY Shop.Customer RENAME ATTRIBUTE Email TO EmailAddress;\nALTER ENTITY Shop.Customer MODIFY ATTRIBUTE Phone: String(30) DEFAULT ''; -- type restated\nALTER ENTITY Shop.Customer DROP DEFAULT ON ATTRIBUTE Phone; -- clear a default\nALTER ENTITY Shop.Customer ADD INDEX ON (EmailAddress);\nALTER ENTITY Shop.Customer\n ADD EVENT HANDLER ON BEFORE COMMIT CALL Shop.Validate($currentObject) RAISE ERROR;\n\n-- give every persistent entity in a module an audit trail, in one statement\nALTER ENTITIES IN Shop\n ADD ATTRIBUTE IF NOT EXISTS CreatedDate: AutoCreatedDate,\n ADD ATTRIBUTE IF NOT EXISTS ChangedDate: AutoChangedDate\n WHERE PERSISTENT;", SeeAlso: []string{"domain-model.entity.create", "domain-model.entity.attributes"}, }) diff --git a/mdl-examples/bug-tests/published-attribute-type-change.fail.mdl b/mdl-examples/bug-tests/published-attribute-type-change.fail.mdl new file mode 100644 index 000000000..9a16dd3d8 --- /dev/null +++ b/mdl-examples/bug-tests/published-attribute-type-change.fail.mdl @@ -0,0 +1,33 @@ +mdl 1; +-- NEGATIVE TEST — `mxcli check` must REFUSE this file (MDL102), with no project. +-- +-- Changing the type of an attribute a published OData service already exposes +-- leaves the published type stale. `check` and `exec` accepted it, and the build +-- reported one +-- +-- [CE5016] "Attribute '...' has type ..., but is published as ..." +-- +-- per attribute. Measured on MxBuild 11.16.1: String to Integer, Decimal to +-- Integer, Boolean to String, DateTime to String, String to Decimal. A change +-- inside one EDM type is clean, and so is re-publishing the service AFTER the +-- alter (see published-attribute-type-change.mdl). +create persistent entity MyFirstModule.PaFail ( + Code: integer unique error message 'u' required error message 'r', + Label: string(50), + Amount: decimal +); + +create published odata service MyFirstModule.PaFailSvc ( + path: 'odata/pafail/', version: '1.0.0', ODataVersion: OData4, + namespace: 'MyFirstModule.PaFail', ServiceName: 'PaFailSvc', PublishAssociations: Yes, + Authentication: (basic, session) +) +{ + publish entity MyFirstModule.PaFail as 'Items' ( + ReadMode: source, InsertMode: not_supported, UpdateMode: not_supported, DeleteMode: not_supported + ) + expose (Code as 'id' (KEY, Filterable, Sortable), Label, Amount); +}; + +alter entity MyFirstModule.PaFail modify attribute Label: integer; +alter entity MyFirstModule.PaFail modify attribute Amount: string(30); diff --git a/mdl-examples/bug-tests/published-attribute-type-change.mdl b/mdl-examples/bug-tests/published-attribute-type-change.mdl new file mode 100644 index 000000000..7732c5d87 --- /dev/null +++ b/mdl-examples/bug-tests/published-attribute-type-change.mdl @@ -0,0 +1,38 @@ +mdl 1; +-- Control for published-attribute-type-change.fail.mdl: an alter inside one EDM type +-- (String(50) to String(80); Integer to Long are both Edm.Int64), and a type change +-- followed by `create or modify published odata service`, which re-derives the +-- published types. Builds with 0 errors on MxBuild 11.16.1. +create persistent entity MyFirstModule.PaOk ( + Code: integer unique error message 'u' required error message 'r', + Label: string(50), + Amount: decimal +); + +create published odata service MyFirstModule.PaOkSvc ( + path: 'odata/paok/', version: '1.0.0', ODataVersion: OData4, + namespace: 'MyFirstModule.PaOk', ServiceName: 'PaOkSvc', PublishAssociations: Yes, + Authentication: (basic, session) +) +{ + publish entity MyFirstModule.PaOk as 'Items' ( + ReadMode: source, InsertMode: not_supported, UpdateMode: not_supported, DeleteMode: not_supported + ) + expose (Code as 'id' (KEY, Filterable, Sortable), Label, Amount); +}; + +alter entity MyFirstModule.PaOk modify attribute Label: string(80); +alter entity MyFirstModule.PaOk modify attribute Code: long unique error message 'u' required error message 'r'; +alter entity MyFirstModule.PaOk modify attribute Amount: integer; + +create or modify published odata service MyFirstModule.PaOkSvc ( + path: 'odata/paok/', version: '1.0.0', ODataVersion: OData4, + namespace: 'MyFirstModule.PaOk', ServiceName: 'PaOkSvc', PublishAssociations: Yes, + Authentication: (basic, session) +) +{ + publish entity MyFirstModule.PaOk as 'Items' ( + ReadMode: source, InsertMode: not_supported, UpdateMode: not_supported, DeleteMode: not_supported + ) + expose (Code as 'id' (KEY, Filterable, Sortable), Label, Amount); +}; diff --git a/mdl/executor/validate_program.go b/mdl/executor/validate_program.go index 8414fc883..e8d9dd56c 100644 --- a/mdl/executor/validate_program.go +++ b/mdl/executor/validate_program.go @@ -50,6 +50,8 @@ func ValidateProgram(prog *ast.Program, projectPath string) []linter.Violation { violations = append(violations, ValidateAssociationOwnerBoth(prog)...) // A mapping element whose association does not connect it to its parent (CE0295). violations = append(violations, ValidateMappingAssociations(prog, project)...) + // An alter that leaves a published OData attribute published as the wrong type (CE5016). + violations = append(violations, ValidatePublishedAttributeTypes(prog, project)...) for _, stmt := range prog.Statements { // Check enumeration values for reserved words if enumStmt, ok := stmt.(*ast.CreateEnumerationStmt); ok { diff --git a/mdl/executor/validate_published_attribute_types.go b/mdl/executor/validate_published_attribute_types.go new file mode 100644 index 000000000..13002de51 --- /dev/null +++ b/mdl/executor/validate_published_attribute_types.go @@ -0,0 +1,260 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// publishedAttrTypeRule is the rule ID for "an alter that leaves a published +// attribute published as the wrong type". +const publishedAttrTypeRule = "MDL102" + +// CE5016 "Attribute X has type T, but is published as Y." Measured on MxBuild +// 11.16.1: after a published OData service exposes an attribute, `alter entity +// … modify attribute` to a type with another EDM type leaves the published type +// stale, and `mx check` reports one CE5016 per attribute — String to Integer, +// Decimal to Integer, Boolean to String, DateTime to String, String to Decimal. +// `check` and `exec` accepted all of them. A change inside one EDM type +// (String(50) to String(80); Integer to Long, both Edm.Int64) is clean. +// +// The way out is `create or modify published odata service` AFTER the alter, +// which re-derives every published type (0 errors); the same statement BEFORE +// the alter is CE5016 again. So the order of the script decides, and this check +// follows it: an alter is reported unless a later statement in the script +// re-publishes (or alters, or drops) the service it leaves stale. +// +// Services are read from the script and from the project (open may be nil, in +// which case only the script's own are known). Anything whose published or new +// type cannot be worked out is not reported: an unanswered question is not "no". +func ValidatePublishedAttributeTypes(prog *ast.Program, open func() backend.FullBackend) []linter.Violation { + if prog == nil { + return nil + } + s := &pubTypeScan{ + open: open, + pubs: map[pubAttrKey]map[string]string{}, + attrEdm: map[pubAttrKey]string{}, + display: map[string]string{}, + } + for _, stmt := range prog.Statements { + switch st := stmt.(type) { + case *ast.CreateEntityStmt: + for i := range st.Attributes { + if edm, ok := astTypeToEdm(st.Attributes[i].Type); ok { + s.attrEdm[pubAttrKey{lowerQN(st.Name.String()), strings.ToLower(st.Attributes[i].Name)}] = edm + } + } + case *ast.CreateODataServiceStmt: + s.republish(st) + case *ast.AlterODataServiceStmt: + s.forget(lowerQN(st.Name.String())) + case *ast.DropODataServiceStmt: + s.forget(lowerQN(st.Name.String())) + case *ast.AlterEntityStmt: + if st.Operation == ast.AlterEntityModifyAttribute && !st.Calculated { + s.modify(st) + } + } + } + var out []linter.Violation + for _, p := range s.pending { + out = append(out, linter.Violation{ + RuleID: publishedAttrTypeRule, + Severity: linter.SeverityError, + Message: fmt.Sprintf( + "attribute %s.%s is published by %s as %s, but this statement changes it to a type published as %s — "+ + "the published type is not updated, and mxbuild reports CE5016 \"Attribute '%s' has type …, but is published as %s\"", + p.entity, p.attr, p.service, p.from, p.to, p.attr, p.from), + Location: linter.Location{DocumentType: "entity", DocumentName: p.entity}, + Suggestion: fmt.Sprintf( + "Re-publish the service after this statement (`create or modify published odata service %s …`), "+ + "or stop exposing %s.", p.service, p.attr), + }) + } + return out +} + +type pubAttrKey struct{ entity, attr string } + +type pendingStale struct { + key pubAttrKey + svc string // lower-cased service QN + service, entity, attr string // as written + from, to string +} + +type pubTypeScan struct { + open func() backend.FullBackend + loaded bool + pubs map[pubAttrKey]map[string]string // attribute -> lower-cased service QN -> published EDM type + attrEdm map[pubAttrKey]string // the script's current EDM type per attribute + display map[string]string // lower-cased service QN -> as written + pending []pendingStale +} + +func lowerQN(s string) string { return strings.ToLower(s) } + +// load reads the project's published services, once. +func (s *pubTypeScan) load() { + if s.loaded { + return + } + s.loaded = true + if s.open == nil { + return + } + b := s.open() + if b == nil { + return + } + modules, err := b.ListModules() + if err != nil { + return + } + names := map[string]string{} + for _, m := range modules { + if m != nil { + names[string(m.ID)] = m.Name + } + } + services, err := b.ListPublishedODataServices() + if err != nil { + return + } + for _, svc := range services { + mod := names[string(svc.ContainerID)] + if svc == nil || mod == "" { + continue + } + qn := mod + "." + svc.Name + for _, et := range svc.EntityTypes { + for _, m := range et.Members { + if m.Kind != "attribute" || m.EdmType == "" { + continue + } + k := pubAttrKey{lowerQN(et.Entity), strings.ToLower(bareMemberName(m.Name))} + if s.pubs[k] == nil { + s.pubs[k] = map[string]string{} + } + s.pubs[k][lowerQN(qn)] = m.EdmType + s.display[lowerQN(qn)] = qn + } + } + } +} + +// forget drops what is known of a service and any staleness reported against it: +// a later statement that rewrites, alters or drops it is something this check +// cannot prove did not fix the types. +func (s *pubTypeScan) forget(svc string) { + s.load() + for k, m := range s.pubs { + delete(m, svc) + if len(m) == 0 { + delete(s.pubs, k) + } + } + kept := s.pending[:0] + for _, p := range s.pending { + if p.svc != svc { + kept = append(kept, p) + } + } + s.pending = kept +} + +// republish records a service the script creates: every exposed attribute is +// published as the type it has at that point in the script. +func (s *pubTypeScan) republish(st *ast.CreateODataServiceStmt) { + svc := lowerQN(st.Name.String()) + s.forget(svc) + s.display[svc] = st.Name.String() + for _, e := range st.Entities { + entity := lowerQN(e.Entity.String()) + for _, m := range e.Members { + k := pubAttrKey{entity, strings.ToLower(m.Name)} + edm, ok := s.attrEdm[k] + if !ok { + edm, ok = s.projectAttrEdm(e.Entity.String(), m.Name) + } + if !ok { + continue // an association, or an attribute whose type is not known here + } + if s.pubs[k] == nil { + s.pubs[k] = map[string]string{} + } + s.pubs[k][svc] = edm + } + } +} + +// projectAttrEdm is the EDM type an attribute of a project entity publishes as, +// for what the script never declared. The type must be read from the project: +// `create or modify` derives every published type from the attributes as they +// are when it runs. +func (s *pubTypeScan) projectAttrEdm(entityQN, attr string) (string, bool) { + if s.open == nil { + return "", false + } + b, ok := s.open().(entityLookupBackend) + if !ok || b == nil { + return "", false + } + ent, found := findEntityByQN(b, entityQN) + if !found { + return "", false + } + for _, a := range ent.Attributes { + if a != nil && strings.EqualFold(a.Name, attr) && a.Type != nil { + if edm := mendixAttrTypeToEdm(a.Type); edm != "" { + return edm, true + } + } + } + return "", false +} + +// modify applies one `modify attribute`: every service publishing it keeps the +// old type, which is reported when the new type publishes differently. +func (s *pubTypeScan) modify(st *ast.AlterEntityStmt) { + s.load() + k := pubAttrKey{lowerQN(st.Name.String()), strings.ToLower(st.AttributeName)} + to, ok := astTypeToEdm(st.DataType) + if !ok { + delete(s.attrEdm, k) + return + } + for svc, from := range s.pubs[k] { + if from != to { + s.pending = append(s.pending, pendingStale{ + key: k, svc: svc, service: s.display[svc], entity: st.Name.String(), attr: st.AttributeName, + from: from, to: to, + }) + } + } + s.attrEdm[k] = to +} + +// astTypeToEdm is mendixAttrTypeToEdm for a type written in the script. The +// published type of a kind it does not know is not guessed. +func astTypeToEdm(dt ast.DataType) (string, bool) { + switch dt.Kind { + case ast.TypeString, ast.TypeEnumeration: + return "Edm.String", true + case ast.TypeInteger, ast.TypeLong, ast.TypeAutoNumber: + return "Edm.Int64", true + case ast.TypeDecimal: + return "Edm.Decimal", true + case ast.TypeBoolean: + return "Edm.Boolean", true + case ast.TypeDateTime, ast.TypeDate: + return "Edm.DateTimeOffset", true + } + return "", false +} diff --git a/mdl/executor/validate_published_attribute_types_test.go b/mdl/executor/validate_published_attribute_types_test.go new file mode 100644 index 000000000..1d0c38654 --- /dev/null +++ b/mdl/executor/validate_published_attribute_types_test.go @@ -0,0 +1,184 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// CE5016 "Attribute X has type T, but is published as Y." Measured on MxBuild +// 11.16.1: changing a published attribute's type to another EDM type with +// `alter entity … modify attribute` leaves the published type stale, and `mx +// check` reports one CE5016 per attribute (String to Integer, Decimal to +// Integer, Boolean to String, DateTime to String, String to Decimal); `check` +// and `exec` accepted all of them. String(50) to String(80) is clean. The +// recovery path is `create or modify published odata service` AFTER the alter, +// which re-derives the types (0 errors); BEFORE the alter it is CE5016 again. +const pubTypesHead = `mdl 1; +create module Pt; +create persistent entity Pt.E ( + Code: integer unique error message 'u' required error message 'r', + Label: string(50), + Amount: decimal, + Flag: boolean, + Stamp: datetime +); +` + +const pubTypesService = `create or modify published odata service Pt.Svc ( + path: 'odata/pt/', version: '1.0.0', ODataVersion: OData4, namespace: 'Pt.S', ServiceName: 'Svc', + Authentication: (basic, session) +) +{ + publish entity Pt.E as 'Items' (ReadMode: source, InsertMode: not_supported, UpdateMode: not_supported, DeleteMode: not_supported) + expose (Code as 'id' (KEY), Label, Amount, Flag, Stamp); +}; +` + +func pubTypeHits(t *testing.T, src string, open func() backend.FullBackend) []string { + t.Helper() + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse: %v\n%s", errs, src) + } + var out []string + for _, v := range ValidatePublishedAttributeTypes(prog, open) { + if v.RuleID != publishedAttrTypeRule { + t.Fatalf("unexpected rule %s", v.RuleID) + } + out = append(out, v.Message) + } + return out +} + +func TestValidatePublishedAttributeTypes_Script(t *testing.T) { + alter := func(attr, typ string) string { + return "alter entity Pt.E modify attribute " + attr + ": " + typ + ";\n" + } + cases := []struct { + name string + src string + want int + }{ + {"string to integer after publishing", pubTypesHead + pubTypesService + alter("Label", "integer"), 1}, + {"decimal to integer", pubTypesHead + pubTypesService + alter("Amount", "integer"), 1}, + {"boolean to string", pubTypesHead + pubTypesService + alter("Flag", "string(10)"), 1}, + {"datetime to string", pubTypesHead + pubTypesService + alter("Stamp", "string(30)"), 1}, + {"string to decimal", pubTypesHead + pubTypesService + alter("Label", "decimal"), 1}, + {"same type, longer string (control)", pubTypesHead + pubTypesService + alter("Label", "string(80)"), 0}, + {"integer to long is Int64 both (control)", pubTypesHead + pubTypesService + alter("Code", "long"), 0}, + {"alter before publishing (control)", pubTypesHead + alter("Label", "integer") + pubTypesService, 0}, + {"alter then re-publish with create or modify (control, measured clean)", + pubTypesHead + pubTypesService + alter("Label", "integer") + pubTypesService, 0}, + {"re-publish then alter (measured CE5016)", + pubTypesHead + alter("Label", "integer") + pubTypesService + alter("Label", "string(20)"), 1}, + {"an attribute the service does not expose (control)", + pubTypesHead + strings.Replace(pubTypesService, ", Flag, Stamp", "", 1) + alter("Flag", "string(10)"), 0}, + {"a service dropped before the alter (control)", + pubTypesHead + pubTypesService + "drop odata service Pt.Svc;\n" + alter("Label", "integer"), 0}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + hits := pubTypeHits(t, c.src, nil) + if len(hits) != c.want { + t.Fatalf("violations=%v, want %d", hits, c.want) + } + if c.want > 0 && (!strings.Contains(hits[0], "CE5016") || !strings.Contains(hits[0], "Pt.Svc")) { + t.Errorf("message should name CE5016 and the service: %s", hits[0]) + } + }) + } +} + +// The service is stored in the project, not created by the script: its published +// type is what the project says. +func TestValidatePublishedAttributeTypes_ProjectService(t *testing.T) { + mod := &model.Module{BaseElement: model.BaseElement{ID: "mod-pj"}, Name: "Pj"} + ent := &domainmodel.Entity{BaseElement: model.BaseElement{ID: "e1"}, ContainerID: "dm-pj", Name: "E"} + dm := &domainmodel.DomainModel{BaseElement: model.BaseElement{ID: "dm-pj"}, ContainerID: mod.ID, + Entities: []*domainmodel.Entity{ent}} + svc := &model.PublishedODataService{ + BaseElement: model.BaseElement{ID: "svc"}, ContainerID: mod.ID, Name: "Svc", + EntityTypes: []*model.PublishedEntityType{{ + Entity: "Pj.E", + Members: []*model.PublishedMember{ + {Kind: "attribute", Name: "Pj.E.Label", EdmType: "Edm.String"}, + {Kind: "attribute", Name: "Pj.E.Count", EdmType: "Edm.Int64"}, + {Kind: "association", Name: "Pj.E.Link"}, + }, + }}, + } + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListDomainModelsFunc: func() ([]*domainmodel.DomainModel, error) { return []*domainmodel.DomainModel{dm}, nil }, + GetModuleByNameFunc: func(string) (*model.Module, error) { return mod, nil }, + GetDomainModelFunc: func(model.ID) (*domainmodel.DomainModel, error) { return dm, nil }, + ListPublishedODataServicesFunc: func() ([]*model.PublishedODataService, error) { return []*model.PublishedODataService{svc}, nil }, + } + open := func() backend.FullBackend { return mb } + alter := func(a string) string { return "mdl 1;\nalter entity Pj.E modify attribute " + a + ";\n" } + + if hits := pubTypeHits(t, alter("Label: integer"), open); len(hits) != 1 || !strings.Contains(hits[0], "Pj.Svc") { + t.Errorf("String to Integer on a published String: want one violation naming Pj.Svc, got %v", hits) + } + if hits := pubTypeHits(t, alter("Label: string(120)"), open); len(hits) != 0 { + t.Errorf("control: same EDM type reported: %v", hits) + } + if hits := pubTypeHits(t, alter("Count: long"), open); len(hits) != 0 { + t.Errorf("control: Integer and Long are both Edm.Int64: %v", hits) + } + if hits := pubTypeHits(t, alter("Other: integer"), open); len(hits) != 0 { + t.Errorf("control: an attribute no service publishes: %v", hits) + } + // Re-published afterwards: the types are re-derived, nothing stale remains. + republish := alter("Label: integer") + "create or modify published odata service Pj.Svc (path: 'odata/pj/', version: '1.0.0', ODataVersion: OData4, namespace: 'Pj.S', ServiceName: 'Svc', Authentication: (basic))\n{\n publish entity Pj.E as 'Items' (ReadMode: source, InsertMode: not_supported, UpdateMode: not_supported, DeleteMode: not_supported)\n expose (Label);\n};\n" + if hits := pubTypeHits(t, republish, open); len(hits) != 0 { + t.Errorf("control: a later create or modify re-publishes the service: %v", hits) + } + // No project: what only the project could say is not judged. + if hits := pubTypeHits(t, alter("Label: integer"), nil); len(hits) != 0 { + t.Errorf("without a project nothing stored is known: %v", hits) + } +} + +// r2 of the measured pair, over a project entity: the script re-publishes the +// service (the published types are derived from the attributes as they are in the +// project) and THEN alters one. The script never declared the attribute, so its +// type has to come from the project. +func TestValidatePublishedAttributeTypes_RepublishThenAlterProjectEntity(t *testing.T) { + mod := &model.Module{BaseElement: model.BaseElement{ID: "mod-pj"}, Name: "Pj"} + ent := &domainmodel.Entity{BaseElement: model.BaseElement{ID: "e1"}, ContainerID: "dm-pj", Name: "E", + Attributes: []*domainmodel.Attribute{ + {BaseElement: model.BaseElement{ID: "a1"}, Name: "Label", Type: &domainmodel.StringAttributeType{Length: 50}}, + }} + dm := &domainmodel.DomainModel{BaseElement: model.BaseElement{ID: "dm-pj"}, ContainerID: mod.ID, + Entities: []*domainmodel.Entity{ent}} + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListDomainModelsFunc: func() ([]*domainmodel.DomainModel, error) { return []*domainmodel.DomainModel{dm}, nil }, + GetModuleByNameFunc: func(string) (*model.Module, error) { return mod, nil }, + GetDomainModelFunc: func(model.ID) (*domainmodel.DomainModel, error) { return dm, nil }, + ListPublishedODataServicesFunc: func() ([]*model.PublishedODataService, error) { return nil, nil }, + } + open := func() backend.FullBackend { return mb } + svc := "create or modify published odata service Pj.Svc (path: 'odata/pj/', version: '1.0.0', ODataVersion: OData4, namespace: 'Pj.S', ServiceName: 'Svc', Authentication: (basic))\n{\n publish entity Pj.E as 'Items' (ReadMode: source, InsertMode: not_supported, UpdateMode: not_supported, DeleteMode: not_supported)\n expose (Label);\n};\n" + alter := "alter entity Pj.E modify attribute Label: integer;\n" + if hits := pubTypeHits(t, "mdl 1;\n"+svc+alter, open); len(hits) != 1 { + t.Errorf("re-publish then alter: want one violation, got %v", hits) + } + if hits := pubTypeHits(t, "mdl 1;\n"+alter+svc, open); len(hits) != 0 { + t.Errorf("control, alter then re-publish: %v", hits) + } + if hits := pubTypeHits(t, "mdl 1;\n"+svc+"alter entity Pj.E modify attribute Label: string(90);\n", open); len(hits) != 0 { + t.Errorf("control, same EDM type: %v", hits) + } +} From 5d7041a33ea643aaea86b9e7703efcec923c3124 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 06:24:44 +0000 Subject: [PATCH 7/9] feat(businessevents): show the service kind and the channel id MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Second slice of part A1 (#2384 §5.4). Both roles of BusinessEvents$BusinessEventService were listed identically and the channel id could not be read from a publisher at all, although it is the value the two apps have to agree on: the broker derives the topic from it (`businessevents..`). - `list business event services` gains a Kind column, published or consumed. Without it a consumer reads as a contradiction — 0 messages and 1 subscribe — because its messages live in the cached contract and never in a Definition. `mixed` and `empty` are named rather than guessed at: neither is a shape Studio Pro writes. - `list business events` gains a Channel column, and now lists what an app CONSUMES. It only ever walked svc.Definition, so every subscription was silently absent: on the fixture project it reported 1 event where the two services handle 2. A consumer's rows take their channel and payload property count from the cached contract; a contract that is absent or unparseable leaves those blank rather than hiding the subscription. - `show contract channels|messages from ` works. It refused with "no cached AsyncAPI contract … (it may be a publisher, not a consumer)" because it only read Document, while a publisher keeps its channels in Definition. contractFromDefinition projects what is stored — it is not the AsyncAPI renderer (#2384 part E): OperationID is left empty, because a publisher stores none and Studio Pro mints it at export time. A service with neither contract nor definition still refuses, naming which is missing. On the fixture both sides now show the same channel id f2477dc90f714d69bb12df0df1ee9cce, so the publisher/subscriber join is readable from one command. The business-events skill is corrected too: it claimed "there is no MDL property to state a channel name, so two apps defining the same service get different topics". The premise is right and the conclusion wrong — the producing app mints the id and it reaches a consumer inside the exported contract, not by both apps spelling the same string. Controls: each of the three behaviours stubbed in turn, and the matching test fails with the reported symptom (Kind blank, consumer's subscription missing, publisher refused). Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011oK82qQLKUyoXXGmbmX68t --- .../skills/mendix/business-events/SKILL.md | 38 ++- mdl/executor/cmd_businessevents.go | 91 ++++++- .../cmd_businessevents_visibility_test.go | 243 ++++++++++++++++++ mdl/executor/cmd_contract.go | 56 +++- 4 files changed, 406 insertions(+), 22 deletions(-) create mode 100644 mdl/executor/cmd_businessevents_visibility_test.go diff --git a/.claude/skills/mendix/business-events/SKILL.md b/.claude/skills/mendix/business-events/SKILL.md index 708778bde..d300400ae 100644 --- a/.claude/skills/mendix/business-events/SKILL.md +++ b/.claude/skills/mendix/business-events/SKILL.md @@ -11,12 +11,12 @@ description: "Define event-driven APIs over Kafka with Mendix business event ser | lines | section | |---|---| | 24-31 | When to Use This Skill | -| 32-101 | Commands | -| 102-117 | Message Definition Syntax | -| 118-125 | Service Properties | -| 126-132 | Operations | -| 133-179 | Publishing Events from Microflows | -| 180-192 | Checklist | +| 32-117 | Commands | +| 118-133 | Message Definition Syntax | +| 134-141 | Service Properties | +| 142-148 | Operations | +| 149-195 | Publishing Events from Microflows | +| 196-208 | Checklist | # Business Events @@ -49,10 +49,20 @@ list business events; -- Filter messages by module list business events in MyModule; --- Full MDL description (round-trippable) +-- Full MDL description (round-trippable for a PUBLISHED service) describe business event service Module.ServiceName; ``` +`list business event services` has a **Kind** column: `published` (a definition of its +own) or `consumed` (no definition, the producer's AsyncAPI contract cached instead). A +consumed service shows 0 messages because its messages live in the contract, not in a +definition — the kind is what makes that legible. `list business events` lists both +roles' events with the **Channel** the broker derives the topic from. + +A **consumed** service is described as MDL comments — its subscriptions and the contract +verbatim — not as a statement: MDL cannot author that shape yet, and `create or modify` +over one is refused rather than allowed to destroy the contract (ako/mxcli#2381). + ### Create a Business Event Service ```sql @@ -74,10 +84,16 @@ create business event service Module.CustomerEventsApi Preserves the existing UUID so other documents that reference this service remain valid. `OR REPLACE` is accepted as a synonym. Each message keeps its stored channel — the -channel name is the Kafka topic subscribers listen on — matched by message name, so a -re-run of an unchanged script reports `Unchanged`; only a new message gets a new -(random) channel name. Renaming a message is a new channel. There is no MDL property to -state a channel name, so two apps defining "the same" service get different topics. +channel name is what the broker derives the Kafka topic from +(`businessevents..`) — matched by message name, so a re-run of an +unchanged script reports `Unchanged`; only a new message gets a new (random) channel +name. Renaming a message is a new channel. + +There is no MDL property to state a channel name, and that is correct: the producing app +mints the id, and it reaches a consumer **inside the exported AsyncAPI contract**, not by +both apps spelling the same string. Read it with `list business events` (Channel column) +or `show contract channels from Module.Service`, which works on a publisher as well as a +consumer. ```sql mdl 1; diff --git a/mdl/executor/cmd_businessevents.go b/mdl/executor/cmd_businessevents.go index bc1040dd1..9c61e948d 100644 --- a/mdl/executor/cmd_businessevents.go +++ b/mdl/executor/cmd_businessevents.go @@ -48,7 +48,7 @@ func listBusinessEventServices(ctx *ExecContext, inModule string) error { } type row struct { - module, qualifiedName, name string + module, qualifiedName, name, kind string msgCount, publishCount, subscribeCount int } var rows []row @@ -57,7 +57,7 @@ func listBusinessEventServices(ctx *ExecContext, inModule string) error { modID := h.FindModuleID(svc.ContainerID) moduleName := h.GetModuleName(modID) qn := moduleName + "." + svc.Name - r := row{module: moduleName, qualifiedName: qn, name: svc.Name} + r := row{module: moduleName, qualifiedName: qn, name: svc.Name, kind: businessEventServiceKind(svc)} if svc.Definition != nil { for _, ch := range svc.Definition.Channels { @@ -77,15 +77,37 @@ func listBusinessEventServices(ctx *ExecContext, inModule string) error { } result := &TableResult{ - Columns: []string{"Module", "QualifiedName", "Service", "Messages", "Publish", "Subscribe"}, + Columns: []string{"Module", "QualifiedName", "Service", "Kind", "Messages", "Publish", "Subscribe"}, Summary: fmt.Sprintf("(%d business event services)", len(filtered)), } for _, r := range rows { - result.Rows = append(result.Rows, []any{r.module, r.qualifiedName, r.name, r.msgCount, r.publishCount, r.subscribeCount}) + result.Rows = append(result.Rows, []any{r.module, r.qualifiedName, r.name, r.kind, r.msgCount, r.publishCount, r.subscribeCount}) } return writeResult(ctx, result) } +// businessEventServiceKind tells the two roles of +// BusinessEvents$BusinessEventService apart. Studio Pro stores a *published* +// service as a Definition with an empty Document, and a *consumed* one the other +// way round: no Definition, and the producing app's AsyncAPI contract in +// Document. Without this the list reads as a contradiction — a consumer shows 0 +// messages and 1 subscribe, because its messages live in the contract and never +// in a Definition. +func businessEventServiceKind(svc *model.BusinessEventService) string { + hasContract := strings.TrimSpace(svc.Document) != "" + switch { + case hasContract && svc.Definition == nil: + return "consumed" + case !hasContract && svc.Definition != nil: + return "published" + case hasContract: + // Both: not a shape Studio Pro writes. Say so rather than pick one. + return "mixed" + default: + return "empty" + } +} + // listBusinessEventClients displays a table of all business event client documents. func listBusinessEventClients(ctx *ExecContext, inModule string) error { fmt.Fprintln(ctx.Output, "Business event clients are not yet implemented.") @@ -109,8 +131,8 @@ func listBusinessEvents(ctx *ExecContext, inModule string) error { } type row struct { - service, message, operation, entity string - attrs int + service, message, channel, operation, entity string + attrs int } var rows []row @@ -139,14 +161,36 @@ func listBusinessEvents(ctx *ExecContext, inModule string) error { entityStr = op.Entity } rows = append(rows, row{ - service: svcQN, - message: msg.MessageName, + service: svcQN, + message: msg.MessageName, + // The channel name is the topic the broker derives + // (`businessevents..`), so it is what a + // subscriber in another app — or a lake's topic config — + // has to agree on. + channel: ch.ChannelName, operation: opStr, entity: entityStr, attrs: len(msg.Attributes), }) } } + continue + } + + // A consumed service has no Definition: its events are the subscriptions, + // and their channels live in the cached contract. Without this branch the + // list reported only what the app publishes and silently omitted + // everything it consumes. + channels, attrCounts := consumedContractIndex(svc.Document) + for _, op := range svc.OperationImplementations { + rows = append(rows, row{ + service: svcQN, + message: op.MessageName, + channel: channels[op.MessageName], + operation: strings.ToUpper(op.Operation), + entity: op.Entity, + attrs: attrCounts[op.MessageName], + }) } } @@ -160,15 +204,42 @@ func listBusinessEvents(ctx *ExecContext, inModule string) error { } result := &TableResult{ - Columns: []string{"Service", "Message", "Operation", "Entity", "Attributes"}, + Columns: []string{"Service", "Message", "Channel", "Operation", "Entity", "Attributes"}, Summary: fmt.Sprintf("(%d business events)", len(rows)), } for _, r := range rows { - result.Rows = append(result.Rows, []any{r.service, r.message, r.operation, r.entity, r.attrs}) + result.Rows = append(result.Rows, []any{r.service, r.message, r.channel, r.operation, r.entity, r.attrs}) } return writeResult(ctx, result) } +// consumedContractIndex reads a consumed service's cached AsyncAPI contract and +// returns, per message name, its channel id and its payload property count. A +// contract that is absent or does not parse yields empty maps: the caller still +// lists the subscriptions, just without the channel, because an unreadable +// contract is no reason to hide what the app subscribes to. +func consumedContractIndex(contract string) (channels map[string]string, attrCounts map[string]int) { + channels, attrCounts = map[string]string{}, map[string]int{} + if strings.TrimSpace(contract) == "" { + return channels, attrCounts + } + doc, err := types.ParseAsyncAPI(contract) + if err != nil { + return channels, attrCounts + } + for _, ch := range doc.Channels { + if ch.MessageRef != "" { + if _, seen := channels[ch.MessageRef]; !seen { + channels[ch.MessageRef] = ch.Name + } + } + } + for _, m := range doc.Messages { + attrCounts[m.Name] = len(m.Properties) + } + return channels, attrCounts +} + // describeBusinessEventService outputs the full MDL description of a business event service. func describeBusinessEventService(ctx *ExecContext, name ast.QualifiedName) error { if !ctx.Connected() { diff --git a/mdl/executor/cmd_businessevents_visibility_test.go b/mdl/executor/cmd_businessevents_visibility_test.go new file mode 100644 index 000000000..189ba18c2 --- /dev/null +++ b/mdl/executor/cmd_businessevents_visibility_test.go @@ -0,0 +1,243 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" +) + +// ako/mxcli#2384 §5.4. Both roles of BusinessEvents$BusinessEventService were +// listed identically, so a consumer read as a contradiction — 0 messages and 1 +// subscribe — and the channel id, which is the topic the broker derives +// (`businessevents..`) and therefore the one value a +// publisher and a subscriber in another app must agree on, could not be read at +// all: `show contract channels` refused a publisher because it only looked at +// the cached contract, which a publisher does not have. + +// twoRoleProject is a publisher and a consumer of the same event, the shape of +// ako/TestApp: the channel id in the publisher's definition is the same string +// as the channel key in the consumer's cached contract. +const sharedChannelID = "f2477dc90f714d69bb12df0df1ee9cce" + +const consumerContract = `asyncapi: 2.2.0 +info: + title: "OrdersEventService" + version: "1.0.0" +channels: + f2477dc90f714d69bb12df0df1ee9cce: + subscribe: + operationId: receiveOrderPlaceEventEvents + message: + $ref: '#/components/messages/OrderPlaceEvent' +components: + messages: + OrderPlaceEvent: + name: OrderPlaceEvent + contentType: application/json + payload: + $ref: '#/components/schemas/OrderPlaceEventPayload' + schemas: + OrderPlaceEventPayload: + type: object + properties: + OrderId: + type: integer + format: int64 + CustomerId: + type: integer + format: int64 +` + +func twoRoleProject(t *testing.T) (*mock.MockBackend, *ContainerHierarchy, *model.BusinessEventService) { + t.Helper() + pub := mkModule("CdcPublish") + con := mkModule("BusinessEventsClient") + + publisher := &model.BusinessEventService{ + BaseElement: model.BaseElement{ID: "sp-pub"}, + ContainerID: pub.ID, + Name: "OrdersEventService", + Definition: &model.BusinessEventDefinition{ + ServiceName: "OrdersEventService", + Channels: []*model.BusinessEventChannel{{ + ChannelName: sharedChannelID, + Messages: []*model.BusinessEventMessage{{ + MessageName: "OrderPlaceEvent", + // Studio Pro's "Other apps can: Subscribe" — a published + // message is one others subscribe to. + CanSubscribe: true, + Attributes: []*model.BusinessEventAttribute{ + {AttributeName: "OrderId", AttributeType: "Long"}, + {AttributeName: "CustomerId", AttributeType: "Long"}, + }, + }}, + }}, + }, + OperationImplementations: []*model.ServiceOperation{{ + MessageName: "OrderPlaceEvent", Operation: "publish", + Entity: "CdcPublish.PBE_OrderPlaceEvent", + }}, + } + consumer := &model.BusinessEventService{ + BaseElement: model.BaseElement{ID: "sp-con"}, + ContainerID: con.ID, + Name: "OrdersEventClient", + Document: consumerContract, + OperationImplementations: []*model.ServiceOperation{{ + MessageName: "OrderPlaceEvent", Operation: "subscribe", + Entity: "BusinessEventsClient.CBE_OrderPlaceEvent", + Microflow: "BusinessEventsClient.HandleBE_OrderPlaceEvent", + }}, + } + + h := mkHierarchy(pub, con) + withContainer(h, publisher.ContainerID, pub.ID) + withContainer(h, consumer.ContainerID, con.ID) + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListBusinessEventServicesFunc: func() ([]*model.BusinessEventService, error) { + return []*model.BusinessEventService{publisher, consumer}, nil + }, + } + return mb, h, publisher +} + +func TestListBusinessEventServices_KindColumn(t *testing.T) { + mb, h, _ := twoRoleProject(t) + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, listBusinessEventServices(ctx, "")) + + out := buf.String() + if !strings.Contains(out, "Kind") { + t.Fatalf("no Kind column:\n%s", out) + } + for _, want := range []string{"published", "consumed"} { + if !strings.Contains(out, want) { + t.Errorf("Kind %q missing:\n%s", want, out) + } + } + // The kind is what makes the counts legible: a consumer's 0 messages is not + // a defect, its messages live in the contract. + for _, line := range strings.Split(out, "\n") { + if strings.Contains(line, "OrdersEventClient") && !strings.Contains(line, "consumed") { + t.Errorf("consumer row not marked consumed: %s", line) + } + if strings.Contains(line, "OrdersEventService") && !strings.Contains(line, "published") { + t.Errorf("publisher row not marked published: %s", line) + } + } +} + +func TestBusinessEventServiceKind_Discriminator(t *testing.T) { + doc := "asyncapi: 2.2.0\n" + for _, tc := range []struct { + name string + svc *model.BusinessEventService + want string + }{ + {"publisher", &model.BusinessEventService{Definition: &model.BusinessEventDefinition{}}, "published"}, + {"consumer", &model.BusinessEventService{Document: doc}, "consumed"}, + // Not a shape Studio Pro writes; naming it is better than picking a side. + {"both", &model.BusinessEventService{Definition: &model.BusinessEventDefinition{}, Document: doc}, "mixed"}, + {"neither", &model.BusinessEventService{}, "empty"}, + // Whitespace is not a contract. + {"blank document", &model.BusinessEventService{Definition: &model.BusinessEventDefinition{}, Document: " \n"}, "published"}, + } { + if got := businessEventServiceKind(tc.svc); got != tc.want { + t.Errorf("%s: kind = %q, want %q", tc.name, got, tc.want) + } + } +} + +// The channel id is the join between the two apps, so both rows must show the +// same string — and the consumer's subscription must appear at all, which it +// did not: listBusinessEvents only walked Definition, so everything an app +// consumes was silently missing from the list. +func TestListBusinessEvents_ChannelIDAndConsumedEvents(t *testing.T) { + mb, h, _ := twoRoleProject(t) + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, listBusinessEvents(ctx, "")) + + out := buf.String() + var pubLine, conLine string + for _, line := range strings.Split(out, "\n") { + switch { + case strings.Contains(line, "OrdersEventService") && strings.Contains(line, "PUBLISH"): + pubLine = line + case strings.Contains(line, "OrdersEventClient") && strings.Contains(line, "SUBSCRIBE"): + conLine = line + } + } + if pubLine == "" { + t.Fatalf("publisher's event missing:\n%s", out) + } + if conLine == "" { + t.Fatalf("consumer's subscription missing — an app's consumed events were invisible:\n%s", out) + } + for _, line := range []string{pubLine, conLine} { + if !strings.Contains(line, sharedChannelID) { + t.Errorf("row does not carry the channel id %s: %s", sharedChannelID, line) + } + } + // The consumer's attribute count comes from the contract's payload schema. + if !strings.Contains(conLine, "2") { + t.Errorf("consumer row lost its payload property count: %s", conLine) + } +} + +// `show contract channels from ` errored with "no cached AsyncAPI +// contract … (it may be a publisher, not a consumer)", although the channel id +// was stored right there in Definition.Channels. +func TestShowContractChannels_WorksOnAPublisher(t *testing.T) { + mb, h, _ := twoRoleProject(t) + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + name := ast.QualifiedName{Module: "CdcPublish", Name: "OrdersEventService"} + if err := listContractChannels(ctx, &name); err != nil { + t.Fatalf("show contract channels on a publisher: %v", err) + } + out := buf.String() + if !strings.Contains(out, sharedChannelID) { + t.Errorf("publisher's channel id not shown:\n%s", out) + } + // AsyncAPI 2's direction is the other app's: a published message is one + // others subscribe to, which is what Studio Pro's export writes. + if !strings.Contains(out, "subscribe") { + t.Errorf("operation direction wrong — a published message is `subscribe` in AsyncAPI 2:\n%s", out) + } +} + +// A service with neither a contract nor a definition still has to refuse, and +// say which of the two is missing. Without this control the projection could +// return an empty document and the reader would print a blank table. +func TestShowContractChannels_RefusesWhenNothingStored(t *testing.T) { + mod := mkModule("M") + empty := &model.BusinessEventService{ + BaseElement: model.BaseElement{ID: "sp-empty"}, + ContainerID: mod.ID, + Name: "Nothing", + } + h := mkHierarchy(mod) + withContainer(h, empty.ContainerID, mod.ID) + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListBusinessEventServicesFunc: func() ([]*model.BusinessEventService, error) { + return []*model.BusinessEventService{empty}, nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + name := ast.QualifiedName{Module: "M", Name: "Nothing"} + err := listContractChannels(ctx, &name) + if err == nil { + t.Fatal("a service with neither contract nor definition was accepted") + } + if !strings.Contains(err.Error(), "neither") { + t.Errorf("refusal does not say what is missing: %v", err) + } +} diff --git a/mdl/executor/cmd_contract.go b/mdl/executor/cmd_contract.go index ab87412ec..449a4cf41 100644 --- a/mdl/executor/cmd_contract.go +++ b/mdl/executor/cmd_contract.go @@ -1835,7 +1835,15 @@ func parseAsyncAPIContract(ctx *ExecContext, name ast.QualifiedName) (*types.Asy svcQN := modName + "." + svc.Name if svc.Document == "" { - return nil, svcQN, mdlerrors.NewValidationf("no cached AsyncAPI contract for %s. This service has no Document field (it may be a publisher, not a consumer)", svcQN) + // A publisher caches no contract — Studio Pro renders one on export + // and stores nothing. Its channels and messages are in the stored + // Definition, though, and the channel id there is the topic other + // apps bind to, so project the definition rather than refusing the + // read (ako/mxcli#2381, #2384 §5.4). + if doc := contractFromDefinition(svc); doc != nil { + return doc, svcQN, nil + } + return nil, svcQN, mdlerrors.NewValidationf("no AsyncAPI contract for %s: it has neither a cached contract (a consumed service) nor a definition (a published one)", svcQN) } doc, err := types.ParseAsyncAPI(svc.Document) @@ -1849,6 +1857,52 @@ func parseAsyncAPIContract(ctx *ExecContext, name ast.QualifiedName) (*types.Asy return nil, "", mdlerrors.NewNotFound("business event service", name.Module+"."+name.Name) } +// contractFromDefinition projects a published business event service's stored +// definition into the shape the contract readers expect. It is a view of what is +// stored, not a rendering of the AsyncAPI document Studio Pro exports: only the +// values the model actually holds are filled in, and OperationID is left empty +// because a publisher stores none (Studio Pro mints `receiveEvents` at +// export time, which is the renderer's business — #2384 part E). +// +// OperationType follows AsyncAPI 2's direction, which is the *other* app's point +// of view: a message this service publishes is one others `subscribe` to, which +// is what Studio Pro writes and what CanSubscribe records. +func contractFromDefinition(svc *model.BusinessEventService) *types.AsyncAPIDocument { + if svc == nil || svc.Definition == nil { + return nil + } + doc := &types.AsyncAPIDocument{ + Title: svc.Definition.ServiceName, + DocVersion: "1.0.0", + } + for _, ch := range svc.Definition.Channels { + for _, msg := range ch.Messages { + op := "publish" + if msg.CanSubscribe { + op = "subscribe" + } + doc.Channels = append(doc.Channels, &types.AsyncAPIChannel{ + Name: ch.ChannelName, + OperationType: op, + MessageRef: msg.MessageName, + }) + m := &types.AsyncAPIMessage{ + Name: msg.MessageName, + Description: msg.Description, + ContentType: "application/json", + } + for _, a := range msg.Attributes { + m.Properties = append(m.Properties, &types.AsyncAPIProperty{ + Name: a.AttributeName, + Type: a.AttributeType, + }) + } + doc.Messages = append(doc.Messages, m) + } + } + return doc +} + // asyncTypeString formats an AsyncAPI property type for display. func asyncTypeString(p *types.AsyncAPIProperty) string { if p.Format != "" { From 47081e5d81673cdfae8e425aaba7e290bd2c93c6 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 07:30:48 +0000 Subject: [PATCH 8/9] check: refuse a REST call that returns a list from an object-rooted mapping (CE0243) Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_011oK82tETa46YQmMS58pQjL --- ...st-mapping-list-of-object-root-ce0243.json | 1 + CHANGELOG.md | 1 + cmd/mxcli/syntax/features_integration.go | 3 + .../rest-mapping-list-of-object-root.fail.mdl | 22 +++++ .../rest-mapping-list-of-object-root.mdl | 28 ++++++ mdl/executor/validate_program.go | 2 + mdl/executor/validate_rest_mapping_result.go | 95 +++++++++++++++++++ .../validate_rest_mapping_result_test.go | 76 +++++++++++++++ 8 files changed, 228 insertions(+) create mode 100644 .claude/skills/fix-issue/findings/mdl-executor/2026-11-18-rest-mapping-list-of-object-root-ce0243.json create mode 100644 mdl-examples/bug-tests/rest-mapping-list-of-object-root.fail.mdl create mode 100644 mdl-examples/bug-tests/rest-mapping-list-of-object-root.mdl create mode 100644 mdl/executor/validate_rest_mapping_result.go create mode 100644 mdl/executor/validate_rest_mapping_result_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-rest-mapping-list-of-object-root-ce0243.json b/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-rest-mapping-list-of-object-root-ce0243.json new file mode 100644 index 000000000..754b286d1 --- /dev/null +++ b/.claude/skills/fix-issue/findings/mdl-executor/2026-11-18-rest-mapping-list-of-object-root-ce0243.json @@ -0,0 +1 @@ +{"area": "mdl/executor", "date": "2026-11-18", "refs": ["ako/mxcli#1258"], "ce": ["CE0243"], "symptom": "`$R = call rest service get $Url (…) returns mapping M.IMM as list of M.E` over an import mapping whose root is one object passed `check` and `exec`; `mx check` reported CE0243 \"The mapping used to return a value of type 'List of M.E', but now returns a value of type 'M.E'\". The other five combinations built clean.", "file": "mdl/executor/validate_rest_mapping_result.go ValidateRestMappingResult (MDL103), called from validate_program.go next to ValidateImportMappingRange; shape from newMappingShapeResolver. Tests: validate_rest_mapping_result_test.go; fixtures mdl-examples/bug-tests/rest-mapping-list-of-object-root(.fail).mdl", "cause": "The REST call is the one place the author states the result's cardinality themselves (`import from mapping` infers it from the mapping), so it is the one place it can disagree with the mapping's root. The four earlier CE0243 findings were all the builder INFERRING that cardinality wrongly, so the hand-written disagreement was never recorded.", "insight": "**When the field-hit records for a code are all the writer's own inference going wrong, look for the one construct where the author states the value instead of the tool inferring it: that is where a script can disagree.** Measured on MxBuild 11.16.1, one microflow per case over 3 mapping roots x 2 declared results: object-rooted `as E` clean, object-rooted `as list of E` CE0243, array-rooted either clean, a root below an array either clean. Binding ONE object out of a list is a real Studio Pro pattern (PrivateCloudData.REST_GetEnvironmentByUUID), so `as E` is never judged. The shape resolver and its conservative unknowns (root path, XML, message definition, no project) already existed for MDL-MAP04, so this rule is a second consumer, not a second resolver. A first end-to-end run against a project that already held my own flows reported a reference error on every case; a project holding only the mappings gave the clean comparison, so build the comparison project from the dependencies only. Sweep: 1,322 repo .mdl files, 0 MDL103 hits. Control: stubbing the rule fails the 4 sub-tests that expect a violation; the 6-flow comparison agrees with mx check on all six."} diff --git a/CHANGELOG.md b/CHANGELOG.md index fc58cb594..8fae108c4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Fixed +- **`check` rejects `rest call … returns mapping M as list of E` over an object-rooted mapping** (ako/mxcli#1258) — an import mapping whose root is one object returns one object, and declaring the call's result as a list passed `check` and `exec` and the build reported CE0243 "The mapping used to return a value of type 'List of …', but now returns a value of type '…'". New MDL103, which reads the mapping's shape from the script or the project (the same resolver as MDL-MAP04); a mapping whose shape cannot be established is not judged. `as Module.E` is fine on either root — over an array it binds one object out of the list. Measured on MxBuild 11.16.1. - **`check` rejects an `alter entity … modify attribute` that leaves a published OData attribute published as the wrong type** (ako/mxcli#1258) — changing the type of an attribute a published service exposes (String to Integer, Decimal to Integer, Boolean or DateTime to String, String to Decimal) passed `check` and `exec` and the build reported CE5016 "Attribute '…' has type …, but is published as …", one per attribute. New MDL102, which reads the service from the script and from the project. A change inside one EDM type (String(50) to String(80); Integer to Long) is fine, and so is a `create or modify published odata service` AFTER the alter, which re-derives the types — before the alter it does not help. Measured on MxBuild 11.16.1. - **`check` rejects a primitive where a list operation needs an object or a list** (ako/mxcli#1258) — `$R = UNION $L WITH $S`, and the same for `INTERSECT`, `SUBTRACT`, `EQUALS` and `CONTAINS $S IN $L`, with a String, Integer, Long, Decimal, Boolean or DateTime variable as the operand that is not the list, passed `check` and `exec` and the build reported CE0023 "Selected variable '…' must be of type Object or List." New MDL101, in microflows and nanoflows. An Object or a List there is fine, `contains($Hay, $Needle)` over two Strings is still the string function, and an enumeration parameter is not judged (it reads as an entity type until the project says otherwise). Measured on MxBuild 11.16.1. - **`check` rejects a mapping element whose association does not connect it to its parent** (ako/mxcli#1258) — in an import or export mapping, `Assoc/Entity` has to connect the entity the element sits under to its own entity. An association between other entities, or one that touches only the parent or only the element's entity, passed `check` and `exec` and the build reported CE0295 "Association '...' is not allowed". New MDL100, which reads the association from the script or the project and follows generalizations of either entity (measured clean); an association it cannot find is never refused. A JSON object element over a to-many association is also CE0295 and is not covered. Measured on MxBuild 11.16.1, import and export. diff --git a/cmd/mxcli/syntax/features_integration.go b/cmd/mxcli/syntax/features_integration.go index 0dfaaf087..adafbd074 100644 --- a/cmd/mxcli/syntax/features_integration.go +++ b/cmd/mxcli/syntax/features_integration.go @@ -287,6 +287,9 @@ func init() { "RETURNS MAPPING Module.IMM AS Module.E -- apply an import mapping (single object)\n" + "RETURNS MAPPING Module.IMM AS LIST OF Module.E\n" + "RETURNS NOTHING -- ignore the response\n\n" + + "-- AS LIST OF needs a mapping whose root is an array: over an object-rooted\n" + + "-- mapping the build reports CE0243 (mxcli check: MDL103). AS Module.E is fine\n" + + "-- on either root (on an array it binds one object out of the list).\n\n" + "-- The file document form takes a SPECIALIZATION of System.FileDocument.\n" + "-- Mendix rejects the base type as a return type (CE0362), and MDL064\n" + "-- reports that before the write. There is no matching form for an\n" + diff --git a/mdl-examples/bug-tests/rest-mapping-list-of-object-root.fail.mdl b/mdl-examples/bug-tests/rest-mapping-list-of-object-root.fail.mdl new file mode 100644 index 000000000..51b819c0b --- /dev/null +++ b/mdl-examples/bug-tests/rest-mapping-list-of-object-root.fail.mdl @@ -0,0 +1,22 @@ +mdl 1; +-- NEGATIVE TEST — `mxcli check` must REFUSE this file (MDL103), with no project. +-- +-- `rest call … returns mapping M as list of E` over an import mapping whose root +-- is ONE object. `check` and `exec` accepted it, and the build reported +-- +-- [CE0243] "The mapping used to return a value of type 'List of E', but now +-- returns a value of type 'E'." +-- +-- Measured on MxBuild 11.16.1: an object-rooted mapping is clean `as E` and +-- CE0243 `as list of E`; an array-rooted mapping is clean either way (binding one +-- object out of a list is a real pattern). See rest-mapping-list-of-object-root.mdl. +create non-persistent entity MyFirstModule.RmFailItem (Rid: integer, Name: string(50)); +create json structure MyFirstModule.JS_RmFailObj sample '{"id": 1, "name": "a"}'; +create import mapping MyFirstModule.IMM_RmFailObj with json structure MyFirstModule.JS_RmFailObj { + create MyFirstModule.RmFailItem { Rid = id, Name = name } +}; + +create microflow MyFirstModule.RmFailList ($Url: string) returns List of MyFirstModule.RmFailItem begin + $R = call rest service get $Url (Timeout: 30) returns mapping MyFirstModule.IMM_RmFailObj as list of MyFirstModule.RmFailItem on error rollback; + return $R; +end; diff --git a/mdl-examples/bug-tests/rest-mapping-list-of-object-root.mdl b/mdl-examples/bug-tests/rest-mapping-list-of-object-root.mdl new file mode 100644 index 000000000..bf5236932 --- /dev/null +++ b/mdl-examples/bug-tests/rest-mapping-list-of-object-root.mdl @@ -0,0 +1,28 @@ +mdl 1; +-- Control for rest-mapping-list-of-object-root.fail.mdl: every other combination of +-- mapping root and declared result. Builds with 0 errors on MxBuild 11.16.1. +create non-persistent entity MyFirstModule.RmOkItem (Rid: integer, Name: string(50)); +create json structure MyFirstModule.JS_RmOkObj sample '{"id": 1, "name": "a"}'; +create json structure MyFirstModule.JS_RmOkArr sample '[{"id": 1, "name": "a"}]'; +create import mapping MyFirstModule.IMM_RmOkObj with json structure MyFirstModule.JS_RmOkObj { + create MyFirstModule.RmOkItem { Rid = id, Name = name } +}; +create import mapping MyFirstModule.IMM_RmOkArr with json structure MyFirstModule.JS_RmOkArr { + create MyFirstModule.RmOkItem { Rid = id, Name = name } +}; + +create microflow MyFirstModule.RmOkObjOne ($Url: string) returns MyFirstModule.RmOkItem begin + $R = call rest service get $Url (Timeout: 30) returns mapping MyFirstModule.IMM_RmOkObj as MyFirstModule.RmOkItem on error rollback; + return $R; +end; + +create microflow MyFirstModule.RmOkArrList ($Url: string) returns List of MyFirstModule.RmOkItem begin + $R = call rest service get $Url (Timeout: 30) returns mapping MyFirstModule.IMM_RmOkArr as list of MyFirstModule.RmOkItem on error rollback; + return $R; +end; + +-- Binding ONE object out of a list-rooted mapping is a real pattern. +create microflow MyFirstModule.RmOkArrOne ($Url: string) returns MyFirstModule.RmOkItem begin + $R = call rest service get $Url (Timeout: 30) returns mapping MyFirstModule.IMM_RmOkArr as MyFirstModule.RmOkItem on error rollback; + return $R; +end; diff --git a/mdl/executor/validate_program.go b/mdl/executor/validate_program.go index e8d9dd56c..33ca539a2 100644 --- a/mdl/executor/validate_program.go +++ b/mdl/executor/validate_program.go @@ -421,6 +421,8 @@ func ValidateProgram(prog *ast.Program, projectPath string) []linter.Violation { // mapping's shape comes from the script, or the project when given // (ako/mxcli#570). violations = append(violations, ValidateImportMappingRange(prog, projectPath)...) + // A REST call declared to return a list from a mapping that returns one object (CE0243). + violations = append(violations, ValidateRestMappingResult(prog, projectPath)...) // Flag a REST client operation whose Body/Response mapping clause has no // `{ ... }` body — Mendix cannot reference a mapping document from an diff --git a/mdl/executor/validate_rest_mapping_result.go b/mdl/executor/validate_rest_mapping_result.go new file mode 100644 index 000000000..29509ee98 --- /dev/null +++ b/mdl/executor/validate_rest_mapping_result.go @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// restMappingResultRule is the rule ID for "a REST call declared to return a +// list from a mapping that returns one object". +const restMappingResultRule = "MDL103" + +// CE0243 "The mapping used to return a value of type 'List of X', but now +// returns a value of type 'X'." — a `rest call … returns mapping M as list of E` +// over an import mapping whose root is one object. Measured on MxBuild 11.16.1, +// one microflow per case: +// +// object-rooted mapping as E clean +// object-rooted mapping as list of E CE0243 <- check and exec accepted it +// array-rooted mapping as E clean (binds one out of a list) +// array-rooted mapping as list of E clean +// mapping rooted below an array, either form clean +// +// Only the first row of the second kind is wrong. Binding ONE object out of a +// list-rooted mapping is a real pattern (Studio Pro's own +// PrivateCloudData.REST_GetEnvironmentByUUID), so `as E` is never judged. +// +// The mapping's shape comes from the same resolver MDL-MAP04 uses: the script +// when it creates the mapping and its JSON structure, the project otherwise. A +// shape it cannot establish — a root path, an XML or message-definition mapping, +// no project — leaves the statement alone. +func ValidateRestMappingResult(prog *ast.Program, projectPath string) []linter.Violation { + if prog == nil { + return nil + } + type use struct { + kind string + flow ast.QualifiedName + call *ast.RestCallStmt + } + var uses []use + collect := func(kind string, name ast.QualifiedName, body []ast.MicroflowStatement) { + var walk func([]ast.MicroflowStatement) + walk = func(stmts []ast.MicroflowStatement) { + forEachMicroflowStatement(stmts, func(s ast.MicroflowStatement) { + if c, ok := s.(*ast.RestCallStmt); ok && c.Result.Type == ast.RestResultMapping && c.Result.IsList { + uses = append(uses, use{kind, name, c}) + } + if h := getErrorHandlerBody(s); len(h) > 0 { + walk(h) + } + }) + } + walk(body) + } + for _, stmt := range prog.Statements { + switch s := stmt.(type) { + case *ast.CreateMicroflowStmt: + collect("microflow", s.Name, s.Body) + case *ast.CreateNanoflowStmt: + collect("nanoflow", s.Name, s.Body) + } + } + if len(uses) == 0 { + return nil + } + + shapes := newMappingShapeResolver(prog, projectPath) + defer shapes.close() + + var out []linter.Violation + for _, u := range uses { + mapping := u.call.Result.MappingName + if shapes.shape(mapping) != shapeObject { + continue + } + out = append(out, linter.Violation{ + RuleID: restMappingResultRule, + Severity: linter.SeverityError, + Message: fmt.Sprintf( + "`rest call … returns mapping %s as list of %s`: %s is object-rooted, so it returns ONE object, "+ + "not a list — exec and the check accept it and mxbuild reports CE0243 \"The mapping used to return "+ + "a value of type 'List of %s', but now returns a value of type '%s'\"", + mapping, u.call.Result.ResultEntity, mapping, u.call.Result.ResultEntity, u.call.Result.ResultEntity), + Location: linter.Location{Module: u.flow.Module, DocumentType: u.kind, DocumentName: u.flow.Name}, + Suggestion: fmt.Sprintf( + "Declare the result as the entity — `returns mapping %s as %s` — or map a JSON structure whose root is an array.", + mapping, u.call.Result.ResultEntity), + }) + } + return out +} diff --git a/mdl/executor/validate_rest_mapping_result_test.go b/mdl/executor/validate_rest_mapping_result_test.go new file mode 100644 index 000000000..88560166f --- /dev/null +++ b/mdl/executor/validate_rest_mapping_result_test.go @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" +) + +// CE0243 "The mapping used to return a value of type 'List of X', but now +// returns a value of type 'X'." Measured on MxBuild 11.16.1, one microflow per +// case, `rest call … returns mapping M as …`: +// +// object-rooted mapping as Entity clean +// object-rooted mapping as list of Entity CE0243 <- accepted by check and exec +// array-rooted mapping as Entity clean (binds one out of a list) +// array-rooted mapping as list of Entity clean +// mapping rooted below an array, either form clean +// +// So only an object-rooted mapping declared as a list is wrong; binding one +// object out of a list-rooted mapping is a legitimate pattern. +const restMapScript = `mdl 1; +create non-persistent entity Rm.Item (Rid: integer, Name: string(50)); +create json structure Rm.JS_Obj sample '{"id": 1, "name": "a"}'; +create json structure Rm.JS_List sample '[{"id": 1, "name": "a"}]'; +create import mapping Rm.IMM_Obj with json structure Rm.JS_Obj { + create Rm.Item { Rid = id, Name = name } +}; +create import mapping Rm.IMM_List with json structure Rm.JS_List { + create Rm.Item { Rid = id, Name = name } +}; +create import mapping Rm.IMM_Below with json structure Rm.JS_Obj root name { + create Rm.Item { Rid = id, Name = name } +}; +` + +func restMapFlow(kind, call string) string { + return restMapScript + "create " + kind + " Rm.F ($Url: string)\nbegin\n" + call + "\nend;\n" +} + +func restCall(mapping, as string) string { + return " $R = call rest service get $Url (Timeout: 30) returns mapping " + mapping + " as " + as + " on error rollback;" +} + +func TestValidateRestMappingResult(t *testing.T) { + cases := []struct { + name, call string + want int + }{ + {"object-rooted mapping as list of", restCall("Rm.IMM_Obj", "list of Rm.Item"), 1}, + {"object-rooted mapping as the entity (control)", restCall("Rm.IMM_Obj", "Rm.Item"), 0}, + {"list-rooted mapping as list of (control)", restCall("Rm.IMM_List", "list of Rm.Item"), 0}, + {"list-rooted mapping as the entity: binds one out of a list (control, measured clean)", + restCall("Rm.IMM_List", "Rm.Item"), 0}, + {"a mapping with a root path is not judged (control)", restCall("Rm.IMM_Below", "list of Rm.Item"), 0}, + {"a mapping that is neither the script's nor known (control)", restCall("Other.IMM_Elsewhere", "list of Rm.Item"), 0}, + {"inside an if branch", " if $Url != empty then\n" + restCall("Rm.IMM_Obj", "list of Rm.Item") + "\n end if;", 1}, + } + for _, kind := range []string{"microflow", "nanoflow"} { + for _, c := range cases { + t.Run(kind+": "+c.name, func(t *testing.T) { + got := ValidateRestMappingResult(mustBuild(t, restMapFlow(kind, c.call)), "") + if len(got) != c.want { + t.Fatalf("violations=%v, want %d", rulesOf(got), c.want) + } + if c.want > 0 { + v := got[0] + if v.RuleID != restMappingResultRule || !strings.Contains(v.Message, "CE0243") || + !strings.Contains(v.Message, "Rm.IMM_Obj") || v.Location.DocumentName != "F" { + t.Errorf("rule, CE code, mapping or location wrong: %+v", v) + } + } + }) + } + } +} From 2e4bf5a5ac5a8143174744c95ffa0efc0560f1d1 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 07:41:28 +0000 Subject: [PATCH 9/9] feat(businessevents): author a consumed service from its AsyncAPI contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Part B of #2384, first slice. A consumed business event service — the same BusinessEvents$BusinessEventService document as a published one, but storing the producer's contract instead of a definition of its own — can now be written from MDL: create or modify business event service BusinessEventsClient.OrdersEventClient ( ContractFile: 'contract.yaml' ) -- or Contract: $$$$ { consume OrderPlaceEvent entity BusinessEventsClient.CBE_OrderPlaceEvent microflow BusinessEventsClient.HandleBE_OrderPlaceEvent; }; Run against the Studio Pro-authored consumer in ako/TestApp 619864b that reports **Unchanged**: the statement reproduces the stored document byte for byte, so ADR-0008 elision fires and nothing is written. That is the evidence the write path emits Studio Pro's shape rather than merely something `check` accepts. describe -> exec -> describe is a fixed point. - Grammar: a CONSUME token distinct from the existing CONSUMED (longest match keeps `consumed` on the latter); a `consume entity … microflow …` body item alongside `message …`; DOLLAR_STRING accepted as an odata property value, gated per key by the strict-property schema rather than by the parser. - AST/visitor: Contract, ContractFile, ContractSet, Consumes, and the R11 exclusivity rules — the two shapes write different documents, so a mixed statement is refused by name rather than half-applied. - Write: Document carries a value (canon completes the key on both roles, but only an explicit write carries the text); a consumer has no Definition, which is what tells the roles apart in storage; a shared write tail gives it the same folder and Created/Modified handling as a publisher. - ContractFile is read at exec time and stored verbatim, so nothing later depends on the file. An empty or missing one is an error: mxbuild throws "business events service should have definitions" on a contractless consumer. - describe emits the statement, with the contract inline. Two cases fall back to the comment form rather than emit something that cannot re-execute: no contract stored, and a contract containing `$$`, which ends the block early (`'$$' .*? '$$'` is non-greedy) with no escape available. The #2381 guard changes with it, as the bug-patterns page says a guard must when its construct becomes authorable: a contract-bearing statement is a restatement and passes, while a published-shaped statement over a stored contract still refuses. Both arms have a test — without the second, #2381 returns in silence. One defect found by running the round trip rather than assuming it: describe first printed `Contract: $$\n\n$$` for looks, and since unquoteDollarString returns everything between the delimiters, the re-parsed contract carried a leading newline the stored one did not. describe -> exec reported Modified on an untouched service until the text was emitted byte-exact. Controls: the consumer branch, ContractFile validation, the guard's lifecycle change and R11 each stubbed in turn, with the matching tests failing on their own symptoms. Still open in part B: ContractFrom (needs the B2 renderer, shared with part E) and the B4 check rules. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011oK82qQLKUyoXXGmbmX68t --- .../skills/mendix/business-events/SKILL.md | 47 +++- cmd/mxcli/lsp_completions_gen.go | 1 + cmd/mxcli/syntax/features_integration.go | 15 +- .../13b-business-events-consumed.mdl | 58 +++++ mdl/ast/ast_businessevents.go | 44 +++- mdl/backend/modelsdk/business_event_write.go | 4 + mdl/executor/cmd_businessevents.go | 163 ++++++++++++-- .../cmd_businessevents_authoring_test.go | 212 ++++++++++++++++++ .../cmd_businessevents_consumer_test.go | 94 ++++++-- mdl/grammar/MDLLexer.g4 | 4 + mdl/grammar/domains/MDLService.g4 | 25 ++- mdl/grammar/domains/MDLSettings.g4 | 2 +- mdl/visitor/visitor_businessevents.go | 56 ++++- mdl/visitor/visitor_businessevents_test.go | 79 +++++++ mdl/visitor/visitor_odata.go | 5 + mdl/visitor/visitor_strict_properties.go | 7 + 16 files changed, 754 insertions(+), 62 deletions(-) create mode 100644 mdl-examples/doctype-tests/13b-business-events-consumed.mdl create mode 100644 mdl/executor/cmd_businessevents_authoring_test.go diff --git a/.claude/skills/mendix/business-events/SKILL.md b/.claude/skills/mendix/business-events/SKILL.md index d300400ae..c0310a4d3 100644 --- a/.claude/skills/mendix/business-events/SKILL.md +++ b/.claude/skills/mendix/business-events/SKILL.md @@ -11,12 +11,12 @@ description: "Define event-driven APIs over Kafka with Mendix business event ser | lines | section | |---|---| | 24-31 | When to Use This Skill | -| 32-117 | Commands | -| 118-133 | Message Definition Syntax | -| 134-141 | Service Properties | -| 142-148 | Operations | -| 149-195 | Publishing Events from Microflows | -| 196-208 | Checklist | +| 32-146 | Commands | +| 147-162 | Message Definition Syntax | +| 163-170 | Service Properties | +| 171-177 | Operations | +| 178-224 | Publishing Events from Microflows | +| 225-237 | Checklist | # Business Events @@ -59,9 +59,38 @@ consumed service shows 0 messages because its messages live in the contract, not definition — the kind is what makes that legible. `list business events` lists both roles' events with the **Channel** the broker derives the topic from. -A **consumed** service is described as MDL comments — its subscriptions and the contract -verbatim — not as a statement: MDL cannot author that shape yet, and `create or modify` -over one is refused rather than allowed to destroy the contract (ako/mxcli#2381). +### Author a Consumed Service + +A **consumed** service stores the producing app's AsyncAPI contract instead of a +definition of its own, and binds events from it: + +```sql +mdl 1; +create or modify business event service Module.OrdersEventClient +( + ContractFile: 'contracts/OrdersEventService.yaml' -- or Contract: $$$$ +) +{ + consume OrderPlaceEvent + entity Module.CBE_OrderPlaceEvent + microflow Module.HandleBE_OrderPlaceEvent; +}; +``` + +- Exactly one of `Contract:` / `ContractFile:`, and neither may be mixed with + `ServiceName`, `EventNamePrefix` or `message` — those are the published shape. +- `ContractFile` is read when the script runs and stored **verbatim**, so nothing later + depends on the file. `describe` prints the `Contract:` form, which re-executes anywhere. +- It is `consume`, not `subscribe`: `message X (…) subscribe` already means "this app + consumes X of its *own* definition". The stored `Operation` is `"subscribe"` either + way; only the keyword differs. +- The consumed entity (`CBE_…`, extending `BusinessEvents.ConsumedBusinessEvent`) and the + handler microflow are ordinary MDL, declared before the service. +- A service whose contract contains `$$` cannot be described inline — `$$…$$` has no + escape — so `describe` falls back to comments and tells you to use `ContractFile`. + +A published-shaped statement over a stored contract is still refused rather than allowed +to destroy it (ako/mxcli#2381). ### Create a Business Event Service diff --git a/cmd/mxcli/lsp_completions_gen.go b/cmd/mxcli/lsp_completions_gen.go index 555fc1c94..f2541c63f 100644 --- a/cmd/mxcli/lsp_completions_gen.go +++ b/cmd/mxcli/lsp_completions_gen.go @@ -313,6 +313,7 @@ var mdlGeneratedKeywords = []protocol.CompletionItem{ {Label: "KNOWLEDGE", Kind: protocol.CompletionItemKindKeyword, Detail: "Widget keyword"}, {Label: "BASES", Kind: protocol.CompletionItemKindKeyword, Detail: "Widget keyword"}, {Label: "CONSUMED", Kind: protocol.CompletionItemKindKeyword, Detail: "Widget keyword"}, + {Label: "CONSUME", Kind: protocol.CompletionItemKindKeyword, Detail: "Widget keyword"}, {Label: "MCP", Kind: protocol.CompletionItemKindKeyword, Detail: "Widget keyword"}, {Label: "STATICIMAGE", Kind: protocol.CompletionItemKindKeyword, Detail: "Widget keyword"}, {Label: "DYNAMICIMAGE", Kind: protocol.CompletionItemKindKeyword, Detail: "Widget keyword"}, diff --git a/cmd/mxcli/syntax/features_integration.go b/cmd/mxcli/syntax/features_integration.go index 0dfaaf087..870949a8c 100644 --- a/cmd/mxcli/syntax/features_integration.go +++ b/cmd/mxcli/syntax/features_integration.go @@ -642,14 +642,23 @@ DESCRIBE DATABASE CONNECTION Ops.Erp;`, Register(SyntaxFeature{ Path: "business-events.create", - Summary: "Create and drop business event service definitions with messages", + Summary: "Create and drop business event services — published (messages) and consumed (a contract plus subscriptions)", Keywords: []string{ "create business event", "drop business event", "message", "publish", "subscribe", "entity", "event name prefix", + "consume", "contract", "contractfile", "asyncapi", "consumed", }, - Syntax: "CREATE [OR MODIFY] BUSINESS EVENT SERVICE Module.Name\n(\n ServiceName: 'Name',\n EventNamePrefix: ''\n)\n{\n MESSAGE EventName (Attr: Type, ...) PUBLISH|SUBSCRIBE\n ENTITY Module.PBE_Entity;\n};\n\nDROP BUSINESS EVENT SERVICE Module.Name;", - Example: "mdl 1;\nCREATE BUSINESS EVENT SERVICE Module.CustomerEventsApi\n(\n ServiceName: 'CustomerEventsApi',\n EventNamePrefix: ''\n)\n{\n MESSAGE CustomerChangedEvent (CustomerId: Long) PUBLISH\n ENTITY Module.PBE_CustomerChangedEvent;\n};\n\nDROP BUSINESS EVENT SERVICE Module.CustomerEventsApi;", + Syntax: "-- PUBLISHED: declares its own messages.\n" + + "CREATE [OR MODIFY] BUSINESS EVENT SERVICE Module.Name\n(\n ServiceName: 'Name',\n EventNamePrefix: ''\n)\n{\n MESSAGE EventName (Attr: Type, ...) PUBLISH|SUBSCRIBE\n ENTITY Module.PBE_Entity [MICROFLOW Module.Handler];\n};\n\n" + + "-- CONSUMED: names the producer's AsyncAPI contract and binds events from it.\n" + + "-- Exactly one of Contract / ContractFile; neither may be mixed with\n" + + "-- ServiceName, EventNamePrefix or MESSAGE.\n" + + "CREATE [OR MODIFY] BUSINESS EVENT SERVICE Module.Name\n(\n Contract: $$$$ | ContractFile: 'path/to/contract.yaml'\n)\n{\n CONSUME EventName\n ENTITY Module.CBE_Entity\n MICROFLOW Module.HandleBE_Event;\n};\n\n" + + "DROP BUSINESS EVENT SERVICE Module.Name;", + Example: "mdl 1;\nCREATE BUSINESS EVENT SERVICE Module.CustomerEventsApi\n(\n ServiceName: 'CustomerEventsApi',\n EventNamePrefix: ''\n)\n{\n MESSAGE CustomerChangedEvent (CustomerId: Long) PUBLISH\n ENTITY Module.PBE_CustomerChangedEvent;\n};\n\n" + + "-- The consuming app, from the contract the producer exported:\n" + + "CREATE BUSINESS EVENT SERVICE Module.CustomerEventsClient\n(\n ContractFile: 'contracts/CustomerEventsApi.yaml'\n)\n{\n CONSUME CustomerChangedEvent\n ENTITY Module.CBE_CustomerChangedEvent\n MICROFLOW Module.HandleBE_CustomerChangedEvent;\n};\n\nDROP BUSINESS EVENT SERVICE Module.CustomerEventsApi;", SeeAlso: []string{"business-events"}, }) diff --git a/mdl-examples/doctype-tests/13b-business-events-consumed.mdl b/mdl-examples/doctype-tests/13b-business-events-consumed.mdl new file mode 100644 index 000000000..a26f2acf3 --- /dev/null +++ b/mdl-examples/doctype-tests/13b-business-events-consumed.mdl @@ -0,0 +1,58 @@ +mdl 1; +-- ============================================================================ +-- Consumed business event service (ako/mxcli#2384 part B). +-- +-- In Mendix 11 both roles are the same BusinessEvents$BusinessEventService +-- document. A PUBLISHED one declares its own messages; a CONSUMED one stores +-- the producing app's AsyncAPI contract and binds events from it. The two are +-- mutually exclusive — see 13-business-events-examples.mdl for the published +-- shape. +-- +-- The contract travels with the statement: `Contract:` holds it inline as a +-- $$…$$ block (it is YAML, so its own quoting is content), and `ContractFile:` +-- reads a path relative to the script and stores the text verbatim, so nothing +-- afterwards depends on the file still being there. +-- +-- `describe` prints the Contract form, which re-executes anywhere and is a +-- fixed point: describe -> exec reports Unchanged on an untouched service. +-- ============================================================================ + +create or modify business event service MyFirstModule.OrdersEventClient +( + Contract: $$asyncapi: 2.2.0 +info: + title: "OrdersEventService" + version: "1.0.0" + x-mendix-generated: true +channels: + f2477dc90f714d69bb12df0df1ee9cce: + subscribe: + operationId: receiveOrderPlaceEventEvents + message: + $ref: '#/components/messages/OrderPlaceEvent' +components: + messages: + OrderPlaceEvent: + name: OrderPlaceEvent + contentType: application/json + payload: + $ref: '#/components/schemas/OrderPlaceEventPayload' + schemas: + OrderPlaceEventPayload: + type: object + properties: + OrderId: + type: integer + format: int64 +$$ +) +{ + -- CONSUME, not SUBSCRIBE: `message X (…) subscribe` already means "this app + -- consumes X of its own definition". The stored Operation is "subscribe" + -- either way; only the MDL keyword differs. + consume OrderPlaceEvent + entity MyFirstModule.CBE_OrderPlaceEvent + microflow MyFirstModule.HandleBE_OrderPlaceEvent; +}; + +drop business event service MyFirstModule.OrdersEventClient; diff --git a/mdl/ast/ast_businessevents.go b/mdl/ast/ast_businessevents.go index 8ca33ff1b..a3f4d7c59 100644 --- a/mdl/ast/ast_businessevents.go +++ b/mdl/ast/ast_businessevents.go @@ -2,19 +2,51 @@ package ast -// CreateBusinessEventServiceStmt represents CREATE BUSINESS EVENT SERVICE. +// CreateBusinessEventServiceStmt represents CREATE BUSINESS EVENT SERVICE, in +// either of the document's two shapes. +// +// A PUBLISHED service names ServiceName/EventNamePrefix and declares Messages. +// A CONSUMED one names a contract — Contract (the AsyncAPI text) or +// ContractFile (a path to it) — and binds events from it with Consumes. The two +// are mutually exclusive; the visitor refuses a statement that mixes them +// (#2384 §5.1). type CreateBusinessEventServiceStmt struct { - CreateGuard // `create … if not exists` (ako/mxcli#731) - Name QualifiedName - ServiceName string - EventNamePrefix string - Messages []*BusinessEventMessageDef + CreateGuard // `create … if not exists` (ako/mxcli#731) + Name QualifiedName + ServiceName string + EventNamePrefix string + Messages []*BusinessEventMessageDef + + // Contract is the AsyncAPI document itself, from `Contract: $$…$$`. + Contract string + // ContractFile is a path to it, resolved relative to the script like an + // image's `File:`. Its text is stored verbatim, so nothing afterwards + // depends on the file still being there. + ContractFile string + // ContractSet records that one of them was written at all, so an empty + // contract can be told from an absent one. + ContractSet bool + Consumes []*BusinessEventConsumeDef + CreateOrModify bool Folder string Documentation string DocumentationSet bool // see mendixlabs/mxcli#1018: absent preserves, empty clears } +// IsConsumer reports whether the statement describes a consumed service. +func (s *CreateBusinessEventServiceStmt) IsConsumer() bool { + return s.ContractSet || len(s.Consumes) > 0 +} + +// BusinessEventConsumeDef binds one event of the contract to the entity it is +// mapped onto and the microflow that handles it: `consume X entity … microflow …`. +type BusinessEventConsumeDef struct { + MessageName string + Entity string // Qualified name of the CBE_ entity + Microflow string // Qualified name of the handler +} + func (s *CreateBusinessEventServiceStmt) isStatement() {} // BusinessEventMessageDef defines a message within a business event service. diff --git a/mdl/backend/modelsdk/business_event_write.go b/mdl/backend/modelsdk/business_event_write.go index 5ccc90196..f9138489a 100644 --- a/mdl/backend/modelsdk/business_event_write.go +++ b/mdl/backend/modelsdk/business_event_write.go @@ -85,6 +85,10 @@ func (b *Backend) DeleteBusinessEventService(id model.ID) error { func businessEventServiceToGen(svc *model.BusinessEventService) element.Element { g := newElem("BusinessEvents$BusinessEventService", string(svc.ID)) addStr(g, "Name", svc.Name) + // The producer's AsyncAPI contract, on a consumed service. Studio Pro writes + // the key on both roles — empty on a publisher — and canon completes it, but + // only an explicit write carries the text (ako/mxcli#2384 part B). + addStr(g, "Document", svc.Document) addStr(g, "Documentation", svc.Documentation) addBool(g, "Excluded", svc.Excluded) addStr(g, "ExportLevel", orDefault(svc.ExportLevel, "Hidden")) diff --git a/mdl/executor/cmd_businessevents.go b/mdl/executor/cmd_businessevents.go index 9c61e948d..6cf7e0f29 100644 --- a/mdl/executor/cmd_businessevents.go +++ b/mdl/executor/cmd_businessevents.go @@ -4,6 +4,7 @@ package executor import ( "fmt" + "os" "strings" "github.com/mendixlabs/mxcli/mdl/ast" @@ -341,12 +342,64 @@ func describeBusinessEventService(ctx *ExecContext, name ast.QualifiedName) erro return nil } -// describeConsumedBusinessEventService writes a consumed service as MDL comments: -// the subscriptions, and the stored AsyncAPI contract verbatim. The contract is -// the irreplaceable part — its channel keys are the topics a subscriber binds to, -// and nothing else in the model records them — so it is printed in full rather -// than summarised. +// describeConsumedBusinessEventService writes a consumed service. +// +// Since MDL can author this shape (ako/mxcli#2384 part B) the output is the +// statement, with the contract inline as a $$…$$ block so it needs no side file +// and re-executes anywhere. Two cases have no statement to write, and both fall +// back to comments rather than emit something that would not round-trip: +// +// - no contract stored — `consume` has nothing to take its event from, and +// mxbuild throws on a contractless consumer anyway; +// - a contract containing `$$` — DOLLAR_STRING is `'$$' .*? '$$'`, so the +// block would end early, and there is no escape for it. func describeConsumedBusinessEventService(ctx *ExecContext, moduleName string, svc *model.BusinessEventService) { + contract := svc.Document + switch { + case strings.TrimSpace(contract) == "": + describeConsumedAsComments(ctx, moduleName, svc, + "It stores no AsyncAPI contract, so there is no statement to write: a", + "consumed service without one makes mxbuild throw \"business events", + "service should have definitions\".") + return + case strings.Contains(contract, "$$"): + describeConsumedAsComments(ctx, moduleName, svc, + "Its AsyncAPI contract contains `$$`, which ends a $$…$$ block and has no", + "escape, so the contract cannot be written inline. Author this service with", + "ContractFile: '' instead, using the contract printed below.") + return + } + + if svc.Documentation != "" { + outputJavadoc(ctx.Output, svc.Documentation) + } + fmt.Fprintf(ctx.Output, "create or modify business event service %s.%s\n", moduleName, svc.Name) + // The stored text goes between the delimiters EXACTLY: unquoteDollarString + // returns everything between them, so a newline added for looks is a + // character the re-executed contract has and the stored one does not, and + // the round trip stops being a fixed point — describe → exec reported + // Modified on an unchanged service until this was byte-exact. The contract + // normally ends in a newline, which puts the closing $$ on its own line. + fmt.Fprintf(ctx.Output, "(\n Contract: $$%s$$\n)\n{\n", svc.Document) + for _, op := range svc.OperationImplementations { + fmt.Fprintf(ctx.Output, " consume %s", op.MessageName) + if op.Entity != "" { + fmt.Fprintf(ctx.Output, "\n entity %s", op.Entity) + } + if op.Microflow != "" { + fmt.Fprintf(ctx.Output, "\n microflow %s", op.Microflow) + } + fmt.Fprintf(ctx.Output, ";\n") + } + fmt.Fprintf(ctx.Output, "};\n") +} + +// describeConsumedAsComments states a consumed service that cannot be written +// as a statement: why not, its subscriptions, and the contract verbatim. The +// contract is the irreplaceable part — its channel keys are the topics a +// subscriber binds to, and nothing else in the model records them — so it is +// printed in full rather than summarised. +func describeConsumedAsComments(ctx *ExecContext, moduleName string, svc *model.BusinessEventService, why ...string) { comment := func(format string, args ...any) { line := fmt.Sprintf(format, args...) if line == "" { @@ -358,10 +411,9 @@ func describeConsumedBusinessEventService(ctx *ExecContext, moduleName string, s comment("%s.%s is a CONSUMED business event service.", moduleName, svc.Name) comment("") - comment("It stores no definition of its own: the producing app's AsyncAPI contract") - comment("and one subscription per event it handles. MDL cannot author this shape") - comment("yet, so what follows is a description and not a statement — there is") - comment("nothing here to re-run, and `create or modify` over it is refused.") + for _, line := range why { + comment("%s", line) + } if svc.Documentation != "" { comment("") @@ -385,8 +437,7 @@ func describeConsumedBusinessEventService(ctx *ExecContext, moduleName string, s comment("") if strings.TrimSpace(svc.Document) == "" { - comment("Contract: none stored. mxbuild refuses a consumed service without one") - comment("(\"business events service should have definitions\").") + comment("Contract: none stored.") return } comment("Contract (AsyncAPI, verbatim). Each key under `channels:` is the channel") @@ -444,11 +495,14 @@ func createBusinessEventService(ctx *ExecContext, stmt *ast.CreateBusinessEventS return mdlerrors.NewAlreadyExistsMsg("business event service", moduleName+"."+stmt.Name.Name, fmt.Sprintf("business event service already exists: %s.%s (use create or modify to update)", moduleName, stmt.Name.Name)) } // Guard, don't drop (ADR-0005): a stored AsyncAPI contract marks a - // *consumed* service, and this statement can only build a published one. - // Rewriting would null the contract and every handler microflow with it — - // and the contract is the only record of the channel id a subscriber binds - // to, so nothing in the model could put it back (ako/mxcli#2381). - if strings.TrimSpace(existing.Document) != "" { + // *consumed* service, and a published-shaped statement would null it and + // every handler microflow with it — the contract being the only record of + // the channel id a subscriber binds to (ako/mxcli#2381). + // + // A guard has a lifecycle: now that MDL can express a consumer, a + // statement that names a contract is restating the document, not + // destroying it, and must pass. Only the published shape still refuses. + if strings.TrimSpace(existing.Document) != "" && !stmt.IsConsumer() { return mdlerrors.NewValidationf( "%s.%s stores an AsyncAPI contract (%d bytes) — the shape of a consumed "+ "business event service, which MDL cannot author yet. That contract holds the "+ @@ -499,6 +553,38 @@ func createBusinessEventService(ctx *ExecContext, stmt *ast.CreateBusinessEventS // subscribers listen on, so minting a new one per run moved every topic of a // deployed app and made an unchanged re-run a write. carry := newBusinessEventIdentityCarry(existingSvc) + + // A consumed service: the contract plus one subscription per `consume`. + // It has no Definition of its own — that is what tells the two roles apart + // in storage — so it is built and written here rather than below. + if stmt.IsConsumer() { + contract, err := resolveBusinessEventContract(ctx, stmt) + if err != nil { + return err + } + svc.Document = contract + svc.Definition = nil + for _, cons := range stmt.Consumes { + op := &model.ServiceOperation{ + MessageName: cons.MessageName, + Operation: "subscribe", + Entity: cons.Entity, + Microflow: cons.Microflow, + } + if storedOp := carry.operationFor(cons.MessageName); storedOp != nil { + op.ID = storedOp.ID + if op.Microflow == "" { + op.Microflow = storedOp.Microflow + } + if op.Entity == "" { + op.Entity = storedOp.Entity + } + } + op.TypeName = "BusinessEvents$ServiceOperation" + svc.OperationImplementations = append(svc.OperationImplementations, op) + } + return writeBusinessEventService(ctx, svc, stmt, moduleName, existingID, existingContainerID) + } def := &model.BusinessEventDefinition{ ServiceName: stmt.ServiceName, EventNamePrefix: stmt.EventNamePrefix, @@ -560,7 +646,13 @@ func createBusinessEventService(ctx *ExecContext, stmt *ast.CreateBusinessEventS svc.Definition = def - // Write to project + return writeBusinessEventService(ctx, svc, stmt, moduleName, existingID, existingContainerID) +} + +// writeBusinessEventService stores a built service, created or rewritten. Both +// roles share it, so a consumer gets the same folder handling and the same +// Created/Modified reporting as a publisher. +func writeBusinessEventService(ctx *ExecContext, svc *model.BusinessEventService, stmt *ast.CreateBusinessEventServiceStmt, moduleName string, existingID, existingContainerID model.ID) error { if existingID != "" { if stmt.Folder == "" { svc.ContainerID = existingContainerID @@ -572,15 +664,42 @@ func createBusinessEventService(ctx *ExecContext, stmt *ast.CreateBusinessEventS return err } ctx.ReportMutation("Modified", "business event service: %s.%s", moduleName, stmt.Name.Name) - } else { - if err := ctx.Backend.CreateBusinessEventService(svc); err != nil { - return mdlerrors.NewBackend("create business event service", err) - } - fmt.Fprintf(ctx.Output, "Created business event service: %s.%s\n", moduleName, stmt.Name.Name) + return nil } + if err := ctx.Backend.CreateBusinessEventService(svc); err != nil { + return mdlerrors.NewBackend("create business event service", err) + } + fmt.Fprintf(ctx.Output, "Created business event service: %s.%s\n", moduleName, stmt.Name.Name) return nil } +// resolveBusinessEventContract returns the AsyncAPI text the statement names, +// whether written inline or pointed at by a path. +// +// The text is what gets stored, so a ContractFile is read now and nothing +// afterwards depends on the file: the same service re-executed on another +// machine, or described and re-run, carries its own contract. An unreadable +// file is an error rather than an empty contract — mxbuild throws +// "business events service should have definitions" on one of those, which is a +// far worse place to find out. +func resolveBusinessEventContract(ctx *ExecContext, stmt *ast.CreateBusinessEventServiceStmt) (string, error) { + if stmt.ContractFile == "" { + return stmt.Contract, nil + } + path, err := ctx.ResolveScriptRelative(stmt.ContractFile) + if err != nil { + return "", mdlerrors.NewValidationf("ContractFile %q: %v", stmt.ContractFile, err) + } + data, err := os.ReadFile(path) + if err != nil { + return "", mdlerrors.NewValidationf("ContractFile %q: %v", stmt.ContractFile, err) + } + if strings.TrimSpace(string(data)) == "" { + return "", mdlerrors.NewValidationf("ContractFile %q is empty: a consumed business event service without a contract makes mxbuild throw \"business events service should have definitions\"", stmt.ContractFile) + } + return string(data), nil +} + // dropBusinessEventService deletes a business event service. func dropBusinessEventService(ctx *ExecContext, stmt *ast.DropBusinessEventServiceStmt) error { if !ctx.ConnectedForWrite() { diff --git a/mdl/executor/cmd_businessevents_authoring_test.go b/mdl/executor/cmd_businessevents_authoring_test.go new file mode 100644 index 000000000..8fbd8753a --- /dev/null +++ b/mdl/executor/cmd_businessevents_authoring_test.go @@ -0,0 +1,212 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" +) + +// ako/mxcli#2384 part B: a consumed business event service is authorable. It is +// the contract plus one subscription per event, and no Definition — that +// absence is what tells the two roles apart in storage. + +// consumerCtx returns a context over a project holding `stored` (nil for an +// empty project), and a pointer to whatever gets written. +func consumerCtx(t *testing.T, stored *model.BusinessEventService) (*ExecContext, **model.BusinessEventService) { + t.Helper() + mod := mkModule("M") + h := mkHierarchy(mod) + services := []*model.BusinessEventService{} + if stored != nil { + stored.ContainerID = mod.ID + withContainer(h, stored.ContainerID, mod.ID) + services = append(services, stored) + } + written := new(*model.BusinessEventService) + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListBusinessEventServicesFunc: func() ([]*model.BusinessEventService, error) { + return services, nil + }, + CreateBusinessEventServiceFunc: func(svc *model.BusinessEventService) error { *written = svc; return nil }, + UpdateBusinessEventServiceFunc: func(svc *model.BusinessEventService) error { *written = svc; return nil }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + return ctx, written +} + +func consumerStmt(contract, contractFile string) *ast.CreateBusinessEventServiceStmt { + return &ast.CreateBusinessEventServiceStmt{ + Name: ast.QualifiedName{Module: "M", Name: "OrdersEventClient"}, + Contract: contract, + ContractFile: contractFile, + ContractSet: true, + Consumes: []*ast.BusinessEventConsumeDef{{ + MessageName: "OrderPlaceEvent", + Entity: "M.CBE_OrderPlaceEvent", + Microflow: "M.HandleBE_OrderPlaceEvent", + }}, + } +} + +func TestCreateBusinessEventService_FromInlineContract(t *testing.T) { + ctx, written := consumerCtx(t, nil) + assertNoError(t, createBusinessEventService(ctx, consumerStmt(storedContract, ""))) + + svc := *written + if svc == nil { + t.Fatal("nothing was written") + } + if svc.Document != storedContract { + t.Errorf("contract not stored verbatim:\n got %q\nwant %q", svc.Document, storedContract) + } + // The discriminator. A consumer with a Definition is a shape Studio Pro + // never writes, and it is what the pre-part-B rebuild produced. + if svc.Definition != nil { + t.Errorf("a consumed service must have no Definition, got %+v", svc.Definition) + } + if len(svc.OperationImplementations) != 1 { + t.Fatalf("wrote %d operations, want 1", len(svc.OperationImplementations)) + } + op := svc.OperationImplementations[0] + // `consume` is the MDL keyword; the stored Operation stays "subscribe", + // which is what Studio Pro writes and what the runtime reads. + if op.Operation != "subscribe" { + t.Errorf("Operation = %q, want subscribe", op.Operation) + } + if op.MessageName != "OrderPlaceEvent" || op.Entity != "M.CBE_OrderPlaceEvent" || op.Microflow != "M.HandleBE_OrderPlaceEvent" { + t.Errorf("subscription not written: %+v", op) + } +} + +func TestCreateBusinessEventService_FromContractFile(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "contract.yaml"), []byte(storedContract), 0o600); err != nil { + t.Fatal(err) + } + ctx, written := consumerCtx(t, nil) + ctx.ScriptDir = dir + + assertNoError(t, createBusinessEventService(ctx, consumerStmt("", "contract.yaml"))) + svc := *written + if svc == nil { + t.Fatal("nothing was written") + } + // The text is what gets stored, so nothing afterwards depends on the file: + // the same service described and re-run elsewhere carries its own contract. + if svc.Document != storedContract { + t.Errorf("ContractFile not stored verbatim:\n got %q\nwant %q", svc.Document, storedContract) + } +} + +// An empty or missing file must be an error, not an empty contract: mxbuild +// throws "business events service should have definitions" on a contractless +// consumer, which is a far worse place to find out. +func TestCreateBusinessEventService_UnusableContractFileRefused(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "empty.yaml"), []byte(" \n"), 0o600); err != nil { + t.Fatal(err) + } + for _, tc := range []struct{ name, file string }{ + {"empty file", "empty.yaml"}, + {"missing file", "nope.yaml"}, + } { + t.Run(tc.name, func(t *testing.T) { + ctx, written := consumerCtx(t, nil) + ctx.ScriptDir = dir + err := createBusinessEventService(ctx, consumerStmt("", tc.file)) + if err == nil { + t.Fatalf("accepted an unusable ContractFile; wrote %+v", *written) + } + if *written != nil { + t.Errorf("wrote a service anyway: %+v", *written) + } + if !strings.Contains(err.Error(), tc.file) { + t.Errorf("error does not name the file: %v", err) + } + }) + } +} + +// The #2381 guard has a lifecycle (docs-wiki/bug-patterns/rewrite-drops- +// unauthored-state.md): while MDL could not express a consumer it refused every +// rewrite of a stored contract. Now that it can, a contract-bearing statement is +// a restatement and must pass — otherwise the feature's own scripts are refused +// by the backstop that exists to protect them. +func TestCreateOrModifyBusinessEventService_ContractStatementIsNotRefused(t *testing.T) { + stored := storedConsumer("") + ctx, written := consumerCtx(t, stored) + + stmt := consumerStmt(storedContract, "") + stmt.CreateOrModify = true + assertNoError(t, createBusinessEventService(ctx, stmt)) + + svc := *written + if svc == nil { + t.Fatal("the rewrite was refused; the guard did not change with the feature") + } + if svc.Document != storedContract { + t.Errorf("contract not preserved through the rewrite: %d chars", len(svc.Document)) + } + // The operation $ID is carried by message name, so an unchanged rewrite + // equals the stored unit and is elided (ADR-0008). + if len(svc.OperationImplementations) != 1 || svc.OperationImplementations[0].ID != "sp-op" { + t.Errorf("operation $ID not carried: %+v", svc.OperationImplementations) + } +} + +// The other arm. A published-shaped statement over a stored contract still +// destroys it, so it must still refuse — a guard that opened up for part B and +// forgot this would reinstate #2381 in silence. +func TestCreateOrModifyBusinessEventService_PublishedShapeStillRefusedOverAContract(t *testing.T) { + stored := storedConsumer("") + ctx, written := consumerCtx(t, stored) + + err := createBusinessEventService(ctx, &ast.CreateBusinessEventServiceStmt{ + Name: ast.QualifiedName{Module: "M", Name: "OrdersEventClient"}, + ServiceName: "OrdersEventService", + CreateOrModify: true, + Messages: []*ast.BusinessEventMessageDef{{ + MessageName: "OrderPlaceEvent", Operation: "subscribe", Entity: "M.CBE_OrderPlaceEvent", + Attributes: []*ast.BusinessEventAttributeDef{{Name: "OrderId", TypeName: "Long"}}, + }}, + }) + if err == nil { + t.Fatalf("a published-shaped rewrite over a contract was accepted; #2381 is back.\nwritten = %+v", *written) + } + if *written != nil { + t.Errorf("the write was attempted anyway: %+v", *written) + } +} + +// A statement silent about the entity or the handler keeps the stored ones: the +// same partial-statement rule the published shape follows (#1018, #2381). +func TestCreateOrModifyBusinessEventService_ConsumeCarriesStoredBindings(t *testing.T) { + stored := storedConsumer("") + ctx, written := consumerCtx(t, stored) + + stmt := &ast.CreateBusinessEventServiceStmt{ + Name: ast.QualifiedName{Module: "M", Name: "OrdersEventClient"}, + Contract: storedContract, + ContractSet: true, + CreateOrModify: true, + Consumes: []*ast.BusinessEventConsumeDef{{MessageName: "OrderPlaceEvent"}}, + } + assertNoError(t, createBusinessEventService(ctx, stmt)) + + op := (*written).OperationImplementations[0] + if op.Entity != "M.CBE_OrderPlaceEvent" { + t.Errorf("stored entity dropped: %q", op.Entity) + } + if op.Microflow != "M.HandleBE_OrderPlaceEvent" { + t.Errorf("stored handler dropped: %q", op.Microflow) + } +} diff --git a/mdl/executor/cmd_businessevents_consumer_test.go b/mdl/executor/cmd_businessevents_consumer_test.go index f9d837929..fffbfbaa3 100644 --- a/mdl/executor/cmd_businessevents_consumer_test.go +++ b/mdl/executor/cmd_businessevents_consumer_test.go @@ -263,12 +263,13 @@ func TestCreateOrModifyBusinessEventService_CarriesStoredHandlerMicroflow(t *tes // describe printed only `create or modify business event service M.X` for a // consumer — no property list, no body, no `;` — because the whole statement is // written inside `if found.Definition != nil`. `mxcli exec` of that output is a -// parse error, so the round trip is broken and an LLM reading it learns nothing. +// parse error, so the round trip was broken. // -// Until the consumer form has a grammar (proposal part B), describe states the -// contract and the subscriptions as comments: output that parses, and that says -// why there is no statement. -func TestDescribeBusinessEventService_ConsumerDescribesAsParseableComments(t *testing.T) { +// Part B gave the shape a grammar, so describe now writes the statement. What +// this test pins is the property the comment form was standing in for: the +// output parses, and re-executing it restates the same service rather than +// destroying its contract. +func TestDescribeBusinessEventService_ConsumerDescribesAsParseableStatement(t *testing.T) { for _, lang := range []langver.Version{langver.V0, langver.V1} { mod := mkModule("M") stored := storedConsumer(mod.ID) @@ -292,27 +293,82 @@ func TestDescribeBusinessEventService_ConsumerDescribesAsParseableComments(t *te if len(errs) > 0 { t.Fatalf("mdl %d: consumer describe output does not parse: %v\n%s", lang, errs, src) } + var stmt *ast.CreateBusinessEventServiceStmt for _, s := range prog.Statements { - if _, ok := s.(*ast.CreateBusinessEventServiceStmt); ok { - t.Fatalf("mdl %d: consumer describe emitted a create statement; "+ - "re-running it would destroy the contract (#2381)\n%s", lang, src) + if c, ok := s.(*ast.CreateBusinessEventServiceStmt); ok { + stmt = c } } - // What the reader needs is in there: the channel id (the topic), the - // subscription, its handler, and why no statement was written. - for _, want := range []string{ - "f2477dc90f714d69bb12df0df1ee9cce", - "OrderPlaceEvent", - "M.CBE_OrderPlaceEvent", - "M.HandleBE_OrderPlaceEvent", - } { - if !strings.Contains(src, want) { - t.Errorf("mdl %d: consumer describe omits %q:\n%s", lang, want, src) - } + if stmt == nil { + t.Fatalf("mdl %d: no statement in consumer describe:\n%s", lang, src) + } + // The contract must come back byte-identical. A newline added inside the + // $$…$$ block for looks is a character the re-executed contract has and + // the stored one does not, and describe → exec then reports Modified on + // an unchanged service. + if stmt.Contract != storedContract { + t.Errorf("mdl %d: contract not byte-identical through the round trip\n got %q\nwant %q", + lang, stmt.Contract, storedContract) + } + if len(stmt.Consumes) != 1 { + t.Fatalf("mdl %d: re-parsed %d consume items, want 1\n%s", lang, len(stmt.Consumes), src) + } + got := stmt.Consumes[0] + if got.MessageName != "OrderPlaceEvent" || + got.Entity != "M.CBE_OrderPlaceEvent" || + got.Microflow != "M.HandleBE_OrderPlaceEvent" { + t.Errorf("mdl %d: subscription lost through the round trip: %+v", lang, got) } } } +// Two consumers have no statement to write, and both have to fall back to +// comments rather than emit something that cannot be re-executed. +func TestDescribeBusinessEventService_ConsumerWithoutAStatementFallsBackToComments(t *testing.T) { + for _, tc := range []struct { + name, document, want string + }{ + // DOLLAR_STRING is `'$$' .*? '$$'`, so an embedded $$ ends the block + // early and there is no escape for it. + {"contract containing $$", "asyncapi: 2.2.0\nsummary: a $$ b\n", "ContractFile"}, + {"no contract at all", "", "no AsyncAPI contract"}, + } { + t.Run(tc.name, func(t *testing.T) { + mod := mkModule("M") + stored := storedConsumer(mod.ID) + stored.Document = tc.document + h := mkHierarchy(mod) + withContainer(h, stored.ContainerID, mod.ID) + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListBusinessEventServicesFunc: func() ([]*model.BusinessEventService, error) { + return []*model.BusinessEventService{stored}, nil + }, + } + ctx, buf := newMockCtx(t, withBackend(mb), withHierarchy(h)) + assertNoError(t, describeBusinessEventService(ctx, ast.QualifiedName{Module: "M", Name: "OrdersEventClient"})) + + src := langver.HeaderFor(langver.V1) + "\n" + buf.String() + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("fallback output does not parse: %v\n%s", errs, src) + } + for _, s := range prog.Statements { + if _, ok := s.(*ast.CreateBusinessEventServiceStmt); ok { + t.Fatalf("emitted a statement that cannot round-trip:\n%s", src) + } + } + if !strings.Contains(buf.String(), tc.want) { + t.Errorf("fallback does not say why there is no statement (want %q):\n%s", tc.want, buf.String()) + } + // The subscriptions still have to be stated, whichever form is used. + if !strings.Contains(buf.String(), "M.HandleBE_OrderPlaceEvent") { + t.Errorf("fallback omits the handler:\n%s", buf.String()) + } + }) + } +} + // A handler microflow is authorable (`MICROFLOW qualifiedName` on the message // rule) but describe never printed one, so describe → exec dropped it on a // publisher-shaped service too — a loss independent of the consumer case. diff --git a/mdl/grammar/MDLLexer.g4 b/mdl/grammar/MDLLexer.g4 index 8d06a6098..7ba473421 100644 --- a/mdl/grammar/MDLLexer.g4 +++ b/mdl/grammar/MDLLexer.g4 @@ -552,6 +552,10 @@ TOOL: T O O L; KNOWLEDGE: K N O W L E D G E; BASES: B A S E S; CONSUMED: C O N S U M E D; +// CONSUME must stay distinct from CONSUMED above: ANTLR's longest match keeps +// `consumed` on CONSUMED whichever order they appear in, so this only adds the +// shorter word. Body item of a consumed business event service (#2384 §5.1). +CONSUME: C O N S U M E; MCP: M C P; STATICIMAGE: S T A T I C I M A G E; DYNAMICIMAGE: D Y N A M I C I M A G E; diff --git a/mdl/grammar/domains/MDLService.g4 b/mdl/grammar/domains/MDLService.g4 index 11812ae88..4775c16c8 100644 --- a/mdl/grammar/domains/MDLService.g4 +++ b/mdl/grammar/domains/MDLService.g4 @@ -306,6 +306,12 @@ createODataServiceStatement odataPropertyValue : STRING_LITERAL + // A $$…$$ block, for a value too long or too quote-laden for one literal: + // a consumed business event service's AsyncAPI `Contract:` (ako/mxcli#2384 + // §5.1). The grammar accepts it on any key here; which keys may actually + // take it is the strict-property schema's business (textShapes), so an + // OData key still reports "expected a string". + | DOLLAR_STRING | NUMBER_LITERAL | TRUE | FALSE @@ -417,10 +423,27 @@ odataHeaderEntry // BUSINESS EVENT SERVICE // ============================================================================= +// One statement, two documents. A PUBLISHED service declares its own messages +// (`message … publish|subscribe`); a CONSUMED one names a contract in its +// property list and binds events from it (`consume …`). Both shapes are +// accepted here and told apart in the visitor, which refuses a statement that +// mixes them — a parse error could not say which half to drop (#2384 §5.1). createBusinessEventServiceStatement : BUSINESS EVENT SERVICE ifNotExists? qualifiedName LPAREN odataPropertyAssignment (COMMA odataPropertyAssignment)* RPAREN - LBRACE businessEventMessageDef+ RBRACE + LBRACE (businessEventMessageDef | businessEventConsumeDef)+ RBRACE + ; + +// `consume entity … microflow …` binds one event of the contract to +// the entity it is mapped onto and the microflow that handles it. It is spelled +// CONSUME, not SUBSCRIBE: `message X (…) subscribe` already means "this app +// consumes X of its *own* definition", and two spellings writing the same +// ServiceOperation would be unreadable even though R11 keeps them unambiguous. +businessEventConsumeDef + : CONSUME identifierOrKeyword + (ENTITY qualifiedName)? + (MICROFLOW qualifiedName)? + SEMICOLON ; businessEventMessageDef diff --git a/mdl/grammar/domains/MDLSettings.g4 b/mdl/grammar/domains/MDLSettings.g4 index b2603873f..1f22e50d5 100644 --- a/mdl/grammar/domains/MDLSettings.g4 +++ b/mdl/grammar/domains/MDLSettings.g4 @@ -664,7 +664,7 @@ keyword | STORE | STRUCTURE | STRUCTURES | VIEW // Agent editor - | AGENT | AGENTS | KNOWLEDGE | BASES | CONSUMED | MCP | TOOL + | AGENT | AGENTS | KNOWLEDGE | BASES | CONSUMED | CONSUME | MCP | TOOL // Workflow user task settings — words, so still usable as names | DECIDE | CONSENSUS | MAJORITY | MORE_KW | THAN | HALF | MOST | CHOSEN | THRESHOLD diff --git a/mdl/visitor/visitor_businessevents.go b/mdl/visitor/visitor_businessevents.go index e9ba2ffd1..9d56aa775 100644 --- a/mdl/visitor/visitor_businessevents.go +++ b/mdl/visitor/visitor_businessevents.go @@ -3,6 +3,7 @@ package visitor import ( + "fmt" "strings" "github.com/mendixlabs/mxcli/mdl/ast" @@ -15,7 +16,13 @@ func (b *Builder) ExitCreateBusinessEventServiceStatement(ctx *parser.CreateBusi Name: buildQualifiedName(ctx.QualifiedName()), } - // Parse property assignments (ServiceName, EventNamePrefix, Folder) + // R11: a statement is either the published shape (ServiceName / + // EventNamePrefix / `message`) or the consumed one (Contract* / `consume`), + // never both. Tracked while reading so the error can name what collided. + var contractKeys []string + sawPublisherKey := false + + // Parse property assignments (ServiceName, EventNamePrefix, Folder, Contract*) for _, propCtx := range ctx.AllOdataPropertyAssignment() { prop := propCtx.(*parser.OdataPropertyAssignmentContext) name := identifierOrKeywordText(prop.IdentifierOrKeyword()) @@ -25,10 +32,20 @@ func (b *Builder) ExitCreateBusinessEventServiceStatement(ctx *parser.CreateBusi switch strings.ToLower(name) { case "servicename": stmt.ServiceName = value + sawPublisherKey = true case "eventnameprefix": stmt.EventNamePrefix = value + sawPublisherKey = true case "folder": stmt.Folder = value + case "contract": + stmt.Contract = value + stmt.ContractSet = true + contractKeys = append(contractKeys, name) + case "contractfile": + stmt.ContractFile = value + stmt.ContractSet = true + contractKeys = append(contractKeys, name) } } @@ -81,6 +98,43 @@ func (b *Builder) ExitCreateBusinessEventServiceStatement(ctx *parser.CreateBusi stmt.Messages = append(stmt.Messages, msg) } + // Parse `consume entity … microflow …` body items. + for _, consCtx := range ctx.AllBusinessEventConsumeDef() { + consDef, ok := consCtx.(*parser.BusinessEventConsumeDefContext) + if !ok || consDef == nil { + continue + } + cons := &ast.BusinessEventConsumeDef{} + if iok := consDef.IdentifierOrKeyword(); iok != nil { + cons.MessageName = identifierOrKeywordText(iok) + } + qualNames := consDef.AllQualifiedName() + idx := 0 + if consDef.ENTITY() != nil && idx < len(qualNames) { + cons.Entity = getQualifiedNameText(qualNames[idx]) + idx++ + } + if consDef.MICROFLOW() != nil && idx < len(qualNames) { + cons.Microflow = getQualifiedNameText(qualNames[idx]) + } + stmt.Consumes = append(stmt.Consumes, cons) + } + + // R11, reported once with the specific collision rather than a generic + // "invalid statement": the two shapes write different documents, and + // guessing which half the author meant is how a contract gets dropped. + switch { + case len(contractKeys) > 1: + b.addError(fmt.Errorf("a business event service takes one of Contract or ContractFile, not both (%s)", + strings.Join(contractKeys, ", "))) + case len(contractKeys) > 0 && sawPublisherKey: + b.addError(fmt.Errorf("%s describes a consumed business event service, so it cannot also set ServiceName or EventNamePrefix — those belong to a published one", contractKeys[0])) + case len(contractKeys) > 0 && len(stmt.Messages) > 0: + b.addError(fmt.Errorf("%s describes a consumed business event service, whose events come from the contract: use `consume entity … microflow …`, not `message …`", contractKeys[0])) + case len(contractKeys) == 0 && len(stmt.Consumes) > 0: + b.addError(fmt.Errorf("`consume` needs a contract to take the event from: add Contract: $$…$$ or ContractFile: '…' to the property list")) + } + // Check for CREATE OR MODIFY (or OR REPLACE, treated identically) and doc comment createStmt := findParentCreateStatement(ctx) if createStmt != nil { diff --git a/mdl/visitor/visitor_businessevents_test.go b/mdl/visitor/visitor_businessevents_test.go index cd3ca2933..2e4372fe3 100644 --- a/mdl/visitor/visitor_businessevents_test.go +++ b/mdl/visitor/visitor_businessevents_test.go @@ -3,6 +3,7 @@ package visitor import ( + "strings" "testing" "github.com/mendixlabs/mxcli/mdl/ast" @@ -42,3 +43,81 @@ func TestCreateBusinessEventService(t *testing.T) { t.Errorf("Got Operation %q", stmt.Messages[0].Operation) } } + +// ako/mxcli#2384 part B: the consumed shape. `Contract:` takes a $$…$$ block +// because the contract is YAML, and `consume` binds one of its events. +func TestCreateBusinessEventService_ConsumedShape(t *testing.T) { + input := "mdl 1;\n" + `create business event service M.OrdersClient ( + Contract: $$asyncapi: 2.2.0 +info: + title: "OrdersEventService" +$$ + ) { + consume OrderPlaceEvent entity M.CBE_OrderPlaceEvent microflow M.HandleBE_OrderPlaceEvent; + };` + prog, errs := Build(input) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + stmt, ok := prog.Statements[0].(*ast.CreateBusinessEventServiceStmt) + if !ok { + t.Fatalf("got %T", prog.Statements[0]) + } + if !stmt.IsConsumer() { + t.Error("statement does not read as a consumer") + } + // Verbatim: the contract's own quoting and newlines are content. + const wantContract = "asyncapi: 2.2.0\ninfo:\n title: \"OrdersEventService\"\n" + if stmt.Contract != wantContract { + t.Errorf("contract = %q, want %q", stmt.Contract, wantContract) + } + if len(stmt.Consumes) != 1 { + t.Fatalf("got %d consume items, want 1", len(stmt.Consumes)) + } + c := stmt.Consumes[0] + if c.MessageName != "OrderPlaceEvent" || c.Entity != "M.CBE_OrderPlaceEvent" || c.Microflow != "M.HandleBE_OrderPlaceEvent" { + t.Errorf("consume item = %+v", c) + } +} + +// R11. The two shapes write different documents, so a statement that mixes them +// is refused by name rather than half-applied — guessing which half the author +// meant is how a contract gets dropped. +func TestCreateBusinessEventService_ShapesAreExclusive(t *testing.T) { + for _, tc := range []struct{ name, body, want string }{ + { + "contract plus ServiceName", + `( Contract: $$a: b$$, ServiceName: 'X' ) { consume E entity M.E; }`, + "cannot also set ServiceName", + }, + { + "contract plus message", + `( Contract: $$a: b$$ ) { message E (I: Long) publish entity M.E; }`, + "use `consume", + }, + { + "both contract keys", + `( Contract: $$a: b$$, ContractFile: 'c.yaml' ) { consume E entity M.E; }`, + "not both", + }, + { + "consume without a contract", + `( ServiceName: 'X' ) { consume E entity M.E; }`, + "needs a contract", + }, + } { + t.Run(tc.name, func(t *testing.T) { + _, errs := Build("mdl 1;\ncreate business event service M.S " + tc.body + ";") + if len(errs) == 0 { + t.Fatalf("mixed shape accepted: %s", tc.body) + } + var joined string + for _, e := range errs { + joined += e.Error() + "\n" + } + if !strings.Contains(joined, tc.want) { + t.Errorf("error does not say what collided (want %q):\n%s", tc.want, joined) + } + }) + } +} diff --git a/mdl/visitor/visitor_odata.go b/mdl/visitor/visitor_odata.go index afe50ffc9..2bd2e5db8 100644 --- a/mdl/visitor/visitor_odata.go +++ b/mdl/visitor/visitor_odata.go @@ -313,6 +313,11 @@ func odataValueText(val *parser.OdataPropertyValueContext) string { if sl := val.STRING_LITERAL(); sl != nil { return unquoteStringLit(sl) } + // A $$…$$ block: a consumed business event service's AsyncAPI Contract + // (#2384 §5.1). Taken verbatim — it is YAML, and its own quoting is content. + if ds := val.DOLLAR_STRING(); ds != nil { + return unquoteDollarString(ds.GetText()) + } if nl := val.NUMBER_LITERAL(); nl != nil { return nl.GetText() } diff --git a/mdl/visitor/visitor_strict_properties.go b/mdl/visitor/visitor_strict_properties.go index 52070577a..fd2ccc9ad 100644 --- a/mdl/visitor/visitor_strict_properties.go +++ b/mdl/visitor/visitor_strict_properties.go @@ -205,6 +205,11 @@ var businessEventServiceSchema = propSchema{on: "a business event service", keys {"ServiceName", []propShape{shapeString}}, {"EventNamePrefix", []propShape{shapeString}}, {"Folder", []propShape{shapeString}}, + // A consumed service names the producer's AsyncAPI contract instead + // (#2384 §5.1). Contract carries the document, usually as a $$…$$ block + // since it is YAML; ContractFile a path to it. + {"Contract", textShapes}, + {"ContractFile", []propShape{shapeString}}, }} // The model visitor reads each value by one alternative only: a Provider @@ -439,6 +444,8 @@ func odataPropertyShape(pc *parser.OdataPropertyAssignmentContext) propShape { switch { case v.STRING_LITERAL() != nil: return shapeString + case v.DOLLAR_STRING() != nil: + return shapeDollar case v.NUMBER_LITERAL() != nil: return shapeNumber case v.TRUE() != nil || v.FALSE() != nil: