Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .claude/skills/fix-issue.md
Original file line number Diff line number Diff line change
Expand Up @@ -320,6 +320,7 @@ cases for these three BSON types — they fell to `default: return nil`.
| `describe` (and `context` / `diff-local`) renders a Retrieve's XPath with only its **first** predicate group — `where A/B[EndDate = $X];` when the BSON holds `[A/B[EndDate = $X]][Status != 'Completed'][CompletionDate = empty]`. No warning; the output reads as a complete but materially *less restrictive* query, so correct defensive code looks buggy | The grammar's `xpathConstraint` rule matches ONE bracket group, and Mendix concatenates siblings. `ParseXPathConstraint` removes the error listeners, so ANTLR parsed group 1, left the rest on the token stream, and **still returned ok=true**; `enrichXPathConstraintForDescribe` treated that as a full parse and re-rendered only what came back. The `if !ok { return original }` fallback never fired | `mdl/visitor/visitor_xpath_public.go` (`ParseXPathConstraint`), `mdl/visitor/xpath_groups.go` (`SplitXPathPredicateGroups`), `mdl/executor/cmd_microflows_format_action.go` (`enrichXPathGroups`, and the render-path split) | Two layers. (1) Reject a partial parse — after the rule, require `stream.LA(1) == antlr.TokenEOF`; that alone stops the loss, since the caller then falls back to the stored string. (2) Split into top-level groups and enrich each, so enrichment still reaches groups after the first. The splitter must track **nesting depth and quoting**: a naive `][` split mangles a nested `[A/B[x = 1]]` and a literal containing `]`. **Generalisable**: a parser that silently accepts a prefix is worse than one that fails — any `ok` returned by a rule that can match less than its input must be checked against EOF before callers treat it as lossless. Repro `mdl-examples/bug-tests/772-xpath-constraint-groups.mdl`; A/B against a pre-fix binary on the same project shows the two dropped groups. Issue #772 |
| An import/export mapping over an entity created with `EXTENDS` maps only its **own** attributes; every inherited field shows unmapped in Studio Pro, and `mx check` reports CE1613 "The selected attribute 'Mod.Child.Attr' no longer exists". An inherited Boolean/DateTime element also gets `DataType=String` | The mapping builder prefixed the entity being mapped unconditionally (`attr = parentEntity + "." + attr`), but a member reference is qualified against the entity that **declares** it — the same rule as entity access rules (#758) and the change-object writer (#451). Separately `resolveAttributeType` scanned only the entity's own attributes and fell through to its `"String"` default | `mdl/executor/cmd_import_mappings.go` and `cmd_export_mappings.go` (both carry the same two lines), `mdl/executor/entity_hierarchy.go` (`ResolveMemberRef`, `ResolveMemberType`) | Route both sites through the generalization walk added for #758: `ResolveMemberRef` returns the declaring-entity reference and `ResolveMemberType` finds the type up the chain, each falling back to the old behaviour when the member cannot be resolved. **Watch for the sibling defect**: the old `resolveAttributeType` matched entities **by name across every domain model**, so a same-named entity in another module could win — resolve the module by name instead. **Generalisable**: when one rule has several call sites, a fix at one of them proves nothing about the others; grep for the *pattern* (`range entity.Attributes`, `parentEntity + "."`) rather than the reported symptom. Repro `mdl-examples/bug-tests/703-mapping-inherited-attributes.mdl`; A/B on the same project shows `Map703.Contract.DocName` (CE1613) become `Map703.DocumentBase.DocName`. Issue #703, umbrella #765 |
| `alter settings model JavaVersion = 'Java21'` on Mendix 11.12+ produces a project mxbuild refuses to **load**: `mx check` reports `System.ArgumentOutOfRangeException ... (Parameter 'majorVersion is an unsupported value: Java21')` at `JavaVersionExtensions.fromString`. Every check downstream of the settings unit is lost with it | Mendix renamed the property between 11.6 (`JavaVersion` = `"Java21"`) and 11.12 (`JavaMajorVersion` = `"21"`) — and the rename changed the **value format** as well as the key. The #759 fix followed only the key, writing the caller's value through verbatim, so the 11.6 spelling landed in the 11.12 key | `mdl/settingsoverlay/settingsoverlay.go` (`JavaVersionValue`, `SetJavaVersion`) — shared by both engines; the dead third copy in `modelsdk/mpr/serialize_services.go` carried it too | Render the value in the dialect the stored key expects: strip/add the `Java` prefix per key, and pass an unrecognisable value through untouched so a typo surfaces as a Mendix error instead of a mangled setting. **Generalisable**: a renamed property is not only a renamed key — check whether the value encoding moved with it, and cover *both* directions (either spelling in, document's dialect out). Note the sharper failure mode: the original #759 shape was an unknown property, which mxbuild **tolerates**, so only Studio Pro broke; a wrong *value* for a known enum is a hard build failure, which is why this one surfaced as a red nightly rather than a user report. Repro `mdl-examples/bug-tests/759-java-version-value-dialect.mdl`. Issue #759 (follow-up) |
| On **Mendix 11.13 only**, every microflow using `EXECUTE DATABASE QUERY` fails `mx check` with **CE5277** "Please re-run and save the query to fix the error", once per activity. The queries themselves report nothing — the error lands on the *activities* pointing at them, so it reads like a microflow defect. Both engines. 11.12 and below are clean | 11.13 replaced the integer `QueryType` (1 = custom SQL) on `DatabaseConnector$DatabaseQuery` with a `Type` **string enum** (`Select` / `NonSelect` / `Unknown`), shipping a one-time conversion (`ExternalDatabaseConnectionQueryTypeConversion`) for old documents. mxcli wrote the legacy integer unconditionally, so on 11.13 the new property was simply **absent** — and an absent `Type` reads as Unknown, which is exactly what CE5277 reports | `mdl/dbconnector/querytype.go` (new, shared by both engines), `sdk/mpr/writer_dbconnection.go` + `parser_dbconnection.go`, `mdl/backend/modelsdk/db_write.go` + `integration_read.go`, `model/types.go` (`DatabaseQuery.QueryTypeName`) | Branch on the project's Mendix version (`ProjectVersion().IsAtLeast(11, 13)`) and write **exactly one** spelling. Writing both is not a safe hedge — a property the target's metamodel does not define is the #759 Studio-Pro-won't-open shape. Read side must accept either, or the next ALTER of an 11.13 project writes Unknown straight back. mxcli can't derive the type the way Studio Pro does (running the query and inspecting the result set), so it reads the leading SQL keyword — still better than Mendix's own converter, which marks every migrated query `Select` regardless of statement. **Diagnosis method**: `mx convert -p -s <old-project>` with the NEW mxbuild runs the version's own migration, then diff the BSON — that is what showed `QueryType: 1` → `Type: "Select"` without guessing. **Generalisable**: onboarding a new Mendix minor is not just adding it to the nightly matrix — run the doctype corpus against it first (`MX_BINARY=~/.mxcli/mxbuild/<ver>/modeler/mx go test -tags integration -run TestMxCheck_DoctypeScripts`), because a renamed property surfaces as a red matrix job, not a compile error. Repro `mdl-examples/bug-tests/1113-database-query-type-enum.mdl`. Sibling drift found in the same sweep and deliberately NOT fixed: **CE5278** ("The <db> JDBC driver is missing from the module settings"), a new 11.13 check about the module's Java dependencies, which mxcli has no way to author |

**Key insight:** `microflows$ListRange` stores offset/limit inside a nested
`CustomRange` map — must cast `raw["CustomRange"].(map[string]any)` before
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
mendix-version: ['10.24.19.104498', '11.6.6', '11.12.0']
mendix-version: ['10.24.24.119349', '11.6.8', '11.12.2', '11.13.0']
fail-fast: false
name: test (Mendix ${{ matrix.mendix-version }})
steps:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/push-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
- name: Check docs-site MDL blocks
run: ./scripts/check-skill-mdl.sh ./bin/mxcli docs-site/src
- name: Setup mxbuild
run: ./bin/mxcli setup mxbuild --version 11.9.0
run: ./bin/mxcli setup mxbuild --version 11.12.2
- name: Integration tests
run: make test-integration
timeout-minutes: 30
Expand Down
9 changes: 9 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,15 @@ Enum-valued properties are the sibling trap: validate against
`generated/metamodel` (e.g. `SettingsDatabaseType` is `Hsqldb`, never `HSQLDB`)
rather than passing a user string through.

**On a CREATE there is no stored document to read the key off.** Rule 1 then
becomes: branch on the project's Mendix version and write exactly one spelling —
never both as a hedge. `mdl/dbconnector` does this for the 11.13 rename of
`DatabaseQuery.QueryType` (int) to `Type` (string enum), which mxbuild *does*
catch, as CE5277 on every activity using the query. To learn the target shape
without guessing, run the new mxbuild's own migration over an old project
(`mx convert -p -s <project>`) and diff the BSON: Mendix ships a one-time
conversion per renamed property, so the converted document is authoritative.

### Association Parent/Child Pointer Semantics (Counter-Intuitive)

**CRITICAL**: Mendix BSON uses inverted naming for association pointers:
Expand Down
40 changes: 40 additions & 0 deletions docs/03-development/WIDGET_BSON_VERSION_COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,46 @@ manual patching after CE0463 reports come in.
In the meantime, this doc + the `.claude/skills/debug-bson.md` methodology
keep the patch cadence manageable.

## 11.13 onboarding: the drift was outside the widget layer

Adding a Mendix minor to the nightly matrix is not a one-line change to
`.github/workflows/nightly.yml` — run the doctype corpus against it first:

```bash
mxcli setup mxbuild --version 11.13.0
MX_BINARY=~/.mxcli/mxbuild/11.13.0/modeler/mx \
go test -tags integration -count=1 -run TestMxCheck_DoctypeScripts ./mdl/executor/
```

11.13 produced **no** widget-envelope drift. The one failure came from a
different layer entirely — the Database Connector — and shows the shape of
problem to expect from any new minor: a renamed, re-encoded property.

| Construct | ≤ 11.12 | 11.13 | Cause |
|---|---|---|---|
| any `EXECUTE DATABASE QUERY` activity | accepted | **CE5277** "Please re-run and save the query to fix the error" | `DatabaseConnector$DatabaseQuery.QueryType` (int, 1 = custom SQL) became `Type` (string enum `Select`/`NonSelect`/`Unknown`). mxcli wrote only the legacy key, so `Type` was absent — and absent reads as Unknown. Fixed in `mdl/dbconnector` |
| a `PostgreSQL`/`MSSQL` connection in a bare project | accepted | **CE5278** "The … JDBC driver is missing from the module settings" | A new check on the module's Java dependencies, not on anything mxcli writes. Not fixed: mxcli has no way to author module settings. Invisible in the doctype harness, which imports the connector mpk |

**The method that settled it without guessing**: run the *new* mxbuild's own
migration over an old project and diff the result.

```bash
mx convert -p -s /path/to/11.12-project # with the 11.13 mx binary
```

Mendix ships a one-time conversion per renamed property
(`ExternalDatabaseConnectionQueryTypeConversion` here), so the converted
document *is* the authoritative target shape. This is the non-widget counterpart
of the "Studio Pro Update Widget" diff above, and it needs no Studio Pro.

Two rules carried over from the settings rename (#759) apply verbatim:

1. **Write exactly one spelling.** Writing both is not a safe hedge — a property
the target version's metamodel does not define is what Studio Pro fails to
resolve on open. mxbuild tolerates it, so `mx check` is not the gate here.
2. **The read side must accept either**, or the next ALTER of a new-version
project writes the old key's default straight back over the new one.

## References

- [`.claude/skills/debug-bson.md`](../../.claude/skills/debug-bson.md) — investigation procedure for CE0463 and related widget BSON errors
Expand Down
68 changes: 68 additions & 0 deletions mdl-examples/bug-tests/1113-database-query-type-enum.mdl
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
-- Mendix 11.13: DatabaseQuery.QueryType became the `Type` string enum
--
-- Symptom (on an 11.13 project only): every microflow using EXECUTE DATABASE
-- QUERY fails `mx check`, once per activity:
--
-- [error] [CE5277] "Please re-run and save the query to fix the error"
-- at Execute database query action activity 'Query external database'
--
-- The queries themselves report nothing — the error lands on the activities that
-- point at them, which is why it reads like a microflow problem.
--
-- Cause: Mendix 11.13 replaced the integer `QueryType` (1 = custom SQL) on
-- DatabaseConnector$DatabaseQuery with a `Type` string enum ("Select" /
-- "NonSelect" / "Unknown"), and ships a one-time conversion
-- (ExternalDatabaseConnectionQueryTypeConversion) that rewrites old documents.
-- mxcli wrote the legacy integer unconditionally, so on 11.13 the new property
-- was simply absent — and an absent `Type` reads as Unknown, which is what
-- CE5277 reports.
--
-- Fix: mdl/dbconnector decides the spelling from the project's Mendix version
-- and writes exactly one of them. Writing both is not a safe hedge: a property
-- the target version's metamodel does not define is what Studio Pro fails to
-- resolve on open (the #759 failure shape).
--
-- Because mxcli never connects to the database, it cannot derive the type the
-- way Studio Pro does (by running the query and inspecting the result set). It
-- reads the leading SQL keyword instead — better than Mendix's own converter,
-- which marks every migrated query "Select" regardless of statement.
--
-- Verify: run against an 11.13+ project, then `mx check` — no CE5277. On 11.12
-- and below the same script stores the legacy integer and also checks clean.
--
-- On a bare 11.13 project `mx check` still reports CE5278 ("The PostgreSQL JDBC
-- driver ... is missing from the module settings") — a SEPARATE check 11.13 added,
-- about the module's Java dependencies rather than anything mxcli writes. Add the
-- driver in Studio Pro; the doctype harness does not hit it because it imports the
-- connector mpk, which brings the dependency with it.

create module DbType;

create constant DbType.Dsn type string
default 'jdbc:postgresql://localhost:5432/demo';
create constant DbType.User type string default 'demo';
create constant DbType.Pass type string default '';

create database connection DbType.Warehouse
type 'PostgreSQL'
connection string @DbType.Dsn
username @DbType.User
password @DbType.Pass
begin
-- Reads: stored as Type = 'Select' on 11.13+.
query ListOrders
sql 'SELECT orderId, customer, total FROM orders ORDER BY orderId';

-- Writes: stored as Type = 'NonSelect'. The legacy integer could not express
-- this distinction at all.
query ArchiveOrders
sql 'UPDATE orders SET archived = true WHERE total = 0';
end;

-- The activity is where CE5277 surfaced, so the repro needs one.
create microflow DbType.MF_ListOrders()
begin
$Orders = execute database query DbType.Warehouse.ListOrders;
end;

describe database connection DbType.Warehouse;
29 changes: 23 additions & 6 deletions mdl/backend/modelsdk/db_write.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ package modelsdkbackend
import (
"fmt"

"github.com/mendixlabs/mxcli/mdl/dbconnector"
"github.com/mendixlabs/mxcli/model"
"github.com/mendixlabs/mxcli/modelsdk/codec"
"github.com/mendixlabs/mxcli/modelsdk/element"
Expand Down Expand Up @@ -49,7 +50,7 @@ func (b *Backend) CreateDatabaseConnection(conn *model.DatabaseConnection) error
if conn.ID == "" {
conn.ID = model.ID(mmpr.GenerateID())
}
g := databaseConnectionToGen(conn)
g := databaseConnectionToGen(conn, b.storesQueryTypeEnum())
contents, err := (&codec.Encoder{}).Encode(g)
if err != nil {
return fmt.Errorf("CreateDatabaseConnection: encode: %w", err)
Expand All @@ -67,7 +68,7 @@ func (b *Backend) UpdateDatabaseConnection(conn *model.DatabaseConnection) error
if b.writer == nil {
return fmt.Errorf("UpdateDatabaseConnection: not connected for writing")
}
g := databaseConnectionToGen(conn)
g := databaseConnectionToGen(conn, b.storesQueryTypeEnum())
contents, err := (&codec.Encoder{}).Encode(g)
if err != nil {
return fmt.Errorf("UpdateDatabaseConnection: encode: %w", err)
Expand All @@ -86,10 +87,20 @@ func (b *Backend) DeleteDatabaseConnection(id model.ID) error {
return b.writer.DeleteUnit(string(id))
}

// storesQueryTypeEnum reports whether this project stores a query's type under the
// Mendix 11.13+ `Type` key. An unreadable version falls back to the legacy key.
func (b *Backend) storesQueryTypeEnum() bool {
pv := b.ProjectVersion()
if pv == nil {
return false
}
return dbconnector.StoresTypeEnum(pv.MajorVersion, pv.MinorVersion)
}

// databaseConnectionToGen builds the DatabaseConnection element tree directly with
// the verified storage keys. The gen/databaseconnector setters bind different
// property keys, so this mirrors sdk/mpr.serializeDatabaseConnection field-for-field.
func databaseConnectionToGen(conn *model.DatabaseConnection) element.Element {
func databaseConnectionToGen(conn *model.DatabaseConnection, typeEnum bool) element.Element {
e := newElem("DatabaseConnector$DatabaseConnection", string(conn.ID))
addStr(e, "Name", conn.Name)
addStr(e, "DatabaseType", conn.DatabaseType)
Expand All @@ -107,7 +118,7 @@ func databaseConnectionToGen(conn *model.DatabaseConnection) element.Element {

queries := make([]element.Element, 0, len(conn.Queries))
for _, q := range conn.Queries {
queries = append(queries, databaseQueryToGen(q))
queries = append(queries, databaseQueryToGen(q, typeEnum))
}
addPartList(e, "Queries", queries)

Expand All @@ -118,11 +129,17 @@ func databaseConnectionToGen(conn *model.DatabaseConnection) element.Element {
}

// databaseQueryToGen builds a DatabaseConnector$DatabaseQuery sub-element.
func databaseQueryToGen(q *model.DatabaseQuery) element.Element {
func databaseQueryToGen(q *model.DatabaseQuery, typeEnum bool) element.Element {
e := newElem("DatabaseConnector$DatabaseQuery", string(q.ID))
addStr(e, "Name", q.Name)
addStr(e, "Query", q.SQL)
addInt64(e, "QueryType", int64(q.QueryType))
// Exactly one of the two spellings — writing the other invents a property the
// target version's metamodel does not define. See mdl/dbconnector.
if typeEnum {
addStr(e, dbconnector.TypeKey, dbconnector.TypeToWrite(q.QueryTypeName, q.SQL))
} else {
addInt64(e, dbconnector.QueryTypeKey, int64(q.QueryType))
}

mappings := make([]element.Element, 0, len(q.TableMappings))
for _, m := range q.TableMappings {
Expand Down
Loading
Loading