Skip to content

Allow stacking of bridge optimier - #3013

Open
blegat wants to merge 16 commits into
masterfrom
bl/index-space-redesign
Open

blegat wants to merge 16 commits into
masterfrom
bl/index-space-redesign

Conversation

@blegat

@blegat blegat commented Jun 17, 2026 •

Copy link
Copy Markdown
Member

Here, we simply use an IndexMap to map inner and outer variables. This IndexMap is only created when variable bridges are used.
So if no variable bridges are used, this shouldn't affect performance or anything.
If variable bridges are used, it's already slow because we need to substitute variables in expressions so the IndexMap shouldn't be a bottleneck.
This doesn't require adding the reservation indices API like #3010 so it should be much better.

Closes #3010
Closes #2696

@blegat
blegat force-pushed the bl/index-space-redesign branch from 0e22156 to eee7fd8 Compare October 8, 2026 15:23
@blegat
blegat force-pushed the bl/index-space-redesign branch from eee7fd8 to 82fc9aa Compare October 15, 2026 19:40
@blegat
blegat marked this pull request as ready for review October 21, 2026 14:25
@blegat
blegat force-pushed the bl/index-space-redesign branch from da62975 to e54c674 Compare October 24, 2026 08:16
@odow
odow self-requested a review November 2, 2026 20:34
@odow

odow commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

@odow

odow commented Nov 2, 2026

Copy link
Copy Markdown
Member

There are many failing solver-tests that need investigating.

@blegat
blegat force-pushed the bl/index-space-redesign branch from e54c674 to 6145229 Compare November 4, 2026 16:41
@blegat
blegat added this pull request to stack #3131 November 5, 2026 12:42
blegat added a commit that referenced this pull request Nov 5, 2026
ZeroOneBridge's internal Integer constraint is absent from constraint lists
but still reported as valid. This makes JuMP treat a bridged binary as
integer and breaks fix_discrete_variables and its undo operation.

Filter native validity using the constraints reported by variable,
constraint, and objective bridges. Compare in the recursive model's
namespace: lazy bridges store outer indices, while single bridges store
inner indices. Preserve recursive bridge attribute and deletion dispatch.

This version is based on PR #3013, head
6145229. That PR fixes the original
vector-index reproducer of #2696, but not the hidden Integer constraint.
The corresponding fix on current master is 72eb949b.

Validation on this base:
- Modified ZeroOne bridge file: 227 assertions passed, including active
  variable mapping with distinct inner and outer variable indices.
- New variable/objective visibility tests: 10 assertions passed.
- Original vector and binary issue #2696 reproductions: both passed.
- JuMP/Mosek bridged-direct reproduction: 22 assertions passed, including
  fixing, LP dual = 1.0, undo, relax_integrality, and re-solving the MILP.
- JuliaFormatter 2.14.7 leaves no unstaged tracked diff.

The complete modified general bridge-optimizer file additionally passes
823 assertions on current master.

Addresses #2696 and jump-dev/MosekTools.jl#222.
blegat added a commit that referenced this pull request Nov 5, 2026
Replace per-query scans with a sparse registry of bridge-created constraint
indices. Allocate it only when a bridge creates a child constraint; never
enumerate or copy existing user constraints. Validity checks use expected
O(1) dictionary lookups and allocate no memory in the benchmark below.

Record each new bridge's children and refresh only the affected bridge's
entries on setters, modifications, final_touch, and deletion. Reference
counts preserve overlapping reports. Refresh after failed mutations too,
and release empty registries.

Extend bridge call contexts so recursive operations can still access their
children while public queries and direct deletion hide them. Preserve the
distinct public and inner namespaces of single-bridge wrappers, including
identity bridges with coincident index values. Cover variable, constraint,
and objective bridges and clear ownership state on empty!.

Validation (only focused test files, not the full suite):
- General bridge optimizer: 894 passed, 1 pre-existing expected broken.
- ZeroOne bridge: 233 passed, including nested bridged children and active
  variable index mapping.
- Ownership/context regressions: 35 passed, including a counter proving
  1,000 validity queries do not inspect bridge child lists, sparse storage
  with 1,000 pre-existing native constraints, recursive access, final_touch,
  exception recovery, bulk deletion, and registry cleanup.
- JuMP/Mosek bridged direct model: 22 passed, including MILP solves,
  fixing, LP dual = 1.0, undo, relaxation, and re-solving.
- Temporary @show @__LINE__ tracing exercised 30 of 31 instrumentation
  points. The optional no-context fallback for third-party optimizer
  subtypes was not exercised. Tracing was removed afterward.
- JuliaFormatter 2.14.7 over the repository leaves no unstaged diff.

Query benchmark: is_valid for one native user Integer constraint in a
ZeroOne SingleBridgeOptimizer with the indicated number of bridged binary
variables. Julia 1.13.3, BenchmarkTools minima of 100 samples, evals=1.
Only the is_valid method was swapped between the original PR #3013 code,
the previous scanning fix, and sparse ownership; the models and remaining
implementation were identical. Times are nanoseconds per query.

| Bridged binaries | Original ns | Scanning ns | Sparse ns | Sparse allocations |
| ---: | ---: | ---: | ---: | ---: |
| 0 | 12.0 | 51.0 | 13.0 | 0 |
| 1 | 12.0 | 75.0 | 25.0 | 0 |
| 1000 | 12.0 | 23797.0 | 26.0 | 0 |
| 10000 | 12.0 | 237965.0 | 28.0 | 0 |

Addresses #2696 and jump-dev/MosekTools.jl#222 on top of PR #3013.
blegat added a commit that referenced this pull request Nov 5, 2026
Add reproducible, solver-free benchmarks and Markdown tables comparing
master 77474f5, updated PR #3013 base b8c3efd, and the rebased sparse
ownership fix 0352ee4. Preserve exact time, byte, allocation, and sample
counts in a raw TSV. Include the previous scanning approach in a separate
query-only comparison, using commit 5f65584.

Use native variables, one variable bridge before or after native variables,
fully bridged variables, and stacked Vectorize(Free(model)) wrappers.
Measure construction, affine-row insertion, primal-start reads, and copying.
Validate each scenario on eight variables before timing 10,000-variable
models; include these correctness checks in the existing focused test file.

The preceding rebase onto bl/index-space-redesign was resolved by retaining
both upstream index/deletion regressions and the ownership regressions.
The rebased general bridge test file and the final benchmark/context test
file pass. JuliaFormatter 2.14.7 leaves no unstaged diff.

Run serially on Intel Core i7-12700K performance CPU 8, with Julia 1.13.3
and BenchmarkTools 1.8.0. The no-variable-bridge path has similar measured
cost across versions. The PR makes fully bridged row insertion and copying
about twice as fast as master, but activating mapping before native
variables raises construction cost and allocation. Sparse ownership removes
query scans but adds about 48% to fully bridged construction and 26% to
fully bridged attribute reads relative to PR alone. Report these costs
alongside the improvements; no production code changed during benchmarking.

**Native constraint validity (nanoseconds per call)**

Only the `is_valid` method is exchanged in one ownership-build process. All
models, constructors, and other methods stay identical. The queried constraint
is a separate native user Integer constraint, so every implementation must
return true. Each result uses 100 samples with one evaluation per sample.

| Bridged binaries | PR ns | Scan ns | Sparse ns | Scan allocated bytes | Sparse allocated bytes |
| ---: | ---: | ---: | ---: | ---: | ---: |
| 0 | 14.0 | 74.0 | 15.0 | 192 | 0 |
| 1 | 15.0 | 133.0 | 33.0 | 272 | 0 |
| 1,000 | 14.0 | 41436.0 | 30.0 | 80,192 | 0 |
| 10,000 | 14.0 | 421491.0 | 39.0 | 800,192 | 0 |

**PR #3013 workloads: whole-checkout comparisons**

Each model has 10,000 visible variables. The wrapper is
`Variable.Vectorize(UniversalFallback(Model{Float64}()))`. Bridged variables
are created in `GreaterThan(0.0)`; unbridged variables use `add_variables`.
The stacked case adds an inner `Variable.Free` wrapper.

Each affine row has four terms on cyclically adjacent variables. Rows are
preconstructed outside the timed insertion. Primal starts are populated before
timing reads. Copying prepares the source in advance and creates a fresh
destination wrapper outside each timed sample. Construction includes creating
wrappers. No solver is run.

Each scenario first passes variable-index uniqueness/count, affine-function
round-trip, primal-start, and copy-to correctness checks on an 8-variable
model. The timed models then use 10,000 variables. Master fails
the stacked case because visible variable indices alias; that case is marked
unsupported rather than timing incorrect work.

Times are milliseconds. Each case requests up to 30 samples, one evaluation
per sample, and a 0.5-second budget; longer cases produced 4–30 samples. Exact
sample counts, bytes, and allocation counts are in
[the raw TSV](bridge_optimizer_results.tsv).

**Construct 10,000 variables — minimum ms**

| Variable bridges | Master | PR | Sparse |
| --- | ---: | ---: | ---: |
| None | 0.084 | 0.086 | 0.087 |
| One before native variables | 0.116 | 0.401 | 0.383 |
| One after native variables | 0.085 | 0.090 | 0.089 |
| All variables | 9.460 | 8.400 | 12.437 |
| Stacked Vectorize(Free(model)) | unsupported | 7.238 | 6.881 |

**Add 10,000 affine rows — minimum ms**

| Variable bridges | Master | PR | Sparse |
| --- | ---: | ---: | ---: |
| None | 0.304 | 0.308 | 0.310 |
| One before native variables | 8.432 | 8.548 | 8.734 |
| One after native variables | 8.428 | 8.513 | 8.650 |
| All variables | 20.218 | 9.435 | 9.593 |
| Stacked Vectorize(Free(model)) | unsupported | 19.148 | 19.192 |

**Read 10,000 primal starts — minimum ms**

| Variable bridges | Master | PR | Sparse |
| --- | ---: | ---: | ---: |
| None | 0.174 | 0.180 | 0.173 |
| One before native variables | 0.172 | 0.196 | 0.189 |
| One after native variables | 0.176 | 0.186 | 0.179 |
| All variables | 0.755 | 0.804 | 1.016 |
| Stacked Vectorize(Free(model)) | unsupported | 1.139 | 1.339 |

**Copy the model with 10,000 rows — minimum ms**

| Variable bridges | Master | PR | Sparse |
| --- | ---: | ---: | ---: |
| None | 1.180 | 1.226 | 1.250 |
| One before native variables | 36.791 | 36.890 | 37.897 |
| One after native variables | 36.642 | 34.878 | 35.544 |
| All variables | 145.014 | 73.446 | 78.342 |
| Stacked Vectorize(Free(model)) | unsupported | 123.086 | 119.405 |

**Cumulative allocation — MiB per operation**

| Workload | Variable bridges | Master | PR | Sparse |
| --- | --- | ---: | ---: | ---: |
| Construct 10,000 variables | None | 0.589 | 0.589 | 0.589 |
| Construct 10,000 variables | One before native variables | 0.591 | 1.260 | 1.261 |
| Construct 10,000 variables | One after native variables | 0.591 | 0.591 | 0.593 |
| Construct 10,000 variables | All variables | 7.284 | 7.876 | 15.403 |
| Construct 10,000 variables | Stacked Vectorize(Free(model)) | unsupported | 12.416 | 12.419 |
| Add 10,000 affine rows | None | 1.987 | 1.987 | 1.987 |
| Add 10,000 affine rows | One before native variables | 15.035 | 14.958 | 14.958 |
| Add 10,000 affine rows | One after native variables | 15.035 | 14.958 | 14.958 |
| Add 10,000 affine rows | All variables | 31.513 | 15.568 | 15.568 |
| Add 10,000 affine rows | Stacked Vectorize(Free(model)) | unsupported | 31.589 | 31.589 |
| Read 10,000 primal starts | None | 0.000 | 0.000 | 0.000 |
| Read 10,000 primal starts | One before native variables | 0.000 | 0.000 | 0.000 |
| Read 10,000 primal starts | One after native variables | 0.000 | 0.000 | 0.000 |
| Read 10,000 primal starts | All variables | 1.068 | 1.068 | 1.068 |
| Read 10,000 primal starts | Stacked Vectorize(Free(model)) | unsupported | 1.373 | 1.373 |
| Copy the model with 10,000 rows | None | 5.961 | 5.961 | 5.961 |
| Copy the model with 10,000 rows | One before native variables | 63.232 | 61.265 | 61.267 |
| Copy the model with 10,000 rows | One after native variables | 63.287 | 60.597 | 60.598 |
| Copy the model with 10,000 rows | All variables | 138.901 | 73.387 | 80.914 |
| Copy the model with 10,000 rows | Stacked Vectorize(Free(model)) | unsupported | 146.949 | 146.952 |

MiB rounding hides the 112 bytes allocated by the single-bridge primal-start
reads; the native case allocates zero bytes. See the TSV for exact values.
blegat added a commit that referenced this pull request Nov 6, 2026
ZeroOneBridge's internal Integer constraint is absent from constraint lists
but still reported as valid. This makes JuMP treat a bridged binary as
integer and breaks fix_discrete_variables and its undo operation.

Filter native validity using the constraints reported by variable,
constraint, and objective bridges. Compare in the recursive model's
namespace: lazy bridges store outer indices, while single bridges store
inner indices. Preserve recursive bridge attribute and deletion dispatch.

This version is based on PR #3013, head
6145229. That PR fixes the original
vector-index reproducer of #2696, but not the hidden Integer constraint.
The corresponding fix on current master is 72eb949b.

Validation on this base:
- Modified ZeroOne bridge file: 227 assertions passed, including active
  variable mapping with distinct inner and outer variable indices.
- New variable/objective visibility tests: 10 assertions passed.
- Original vector and binary issue #2696 reproductions: both passed.
- JuMP/Mosek bridged-direct reproduction: 22 assertions passed, including
  fixing, LP dual = 1.0, undo, relax_integrality, and re-solving the MILP.
- JuliaFormatter 2.14.7 leaves no unstaged tracked diff.

The complete modified general bridge-optimizer file additionally passes
823 assertions on current master.

Addresses #2696 and jump-dev/MosekTools.jl#222.
@blegat
blegat force-pushed the bl/index-space-redesign branch from b8c3efd to 46f7e98 Compare November 6, 2026 12:38
blegat added a commit that referenced this pull request Nov 6, 2026
Replace per-query scans with a sparse registry of bridge-created constraint
indices. Allocate it only when a bridge creates a child constraint; never
enumerate or copy existing user constraints. Validity checks use expected
O(1) dictionary lookups and allocate no memory in the benchmark below.

Record each new bridge's children and refresh only the affected bridge's
entries on setters, modifications, final_touch, and deletion. Reference
counts preserve overlapping reports. Refresh after failed mutations too,
and release empty registries.

Extend bridge call contexts so recursive operations can still access their
children while public queries and direct deletion hide them. Preserve the
distinct public and inner namespaces of single-bridge wrappers, including
identity bridges with coincident index values. Cover variable, constraint,
and objective bridges and clear ownership state on empty!.

Validation (only focused test files, not the full suite):
- General bridge optimizer: 894 passed, 1 pre-existing expected broken.
- ZeroOne bridge: 233 passed, including nested bridged children and active
  variable index mapping.
- Ownership/context regressions: 35 passed, including a counter proving
  1,000 validity queries do not inspect bridge child lists, sparse storage
  with 1,000 pre-existing native constraints, recursive access, final_touch,
  exception recovery, bulk deletion, and registry cleanup.
- JuMP/Mosek bridged direct model: 22 passed, including MILP solves,
  fixing, LP dual = 1.0, undo, relaxation, and re-solving.
- Temporary @show @__LINE__ tracing exercised 30 of 31 instrumentation
  points. The optional no-context fallback for third-party optimizer
  subtypes was not exercised. Tracing was removed afterward.
- JuliaFormatter 2.14.7 over the repository leaves no unstaged diff.

Query benchmark: is_valid for one native user Integer constraint in a
ZeroOne SingleBridgeOptimizer with the indicated number of bridged binary
variables. Julia 1.13.3, BenchmarkTools minima of 100 samples, evals=1.
Only the is_valid method was swapped between the original PR #3013 code,
the previous scanning fix, and sparse ownership; the models and remaining
implementation were identical. Times are nanoseconds per query.

| Bridged binaries | Original ns | Scanning ns | Sparse ns | Sparse allocations |
| ---: | ---: | ---: | ---: | ---: |
| 0 | 12.0 | 51.0 | 13.0 | 0 |
| 1 | 12.0 | 75.0 | 25.0 | 0 |
| 1000 | 12.0 | 23797.0 | 26.0 | 0 |
| 10000 | 12.0 | 237965.0 | 28.0 | 0 |

Addresses #2696 and jump-dev/MosekTools.jl#222 on top of PR #3013.
blegat added 11 commits November 6, 2026 17:20
Foundation working (Variable/map.jl + bridge_optimizer.jl):
- Variable.Map has outer_to_inner / inner_to_outer IndexMaps, next_outer_variable counter, per-(F,S) constraint counters
- activate_variable_mapping!(map, model) / activate_constraint_mapping!(map, model, F, S) — lazy triggers, called from
add_constrained_variable(s) when a variable bridge is created
- _record_inner_variable!(b, inner_vi) — translates inner→outer when mapping active, identity otherwise
- MOI.add_variable / MOI.add_variables / MOI.add_constrained_variable(s) updated to translate

Verified manually:
- Basic single-layer flow: bridged variables (xs = [-1, -2, -3]) still use slot/negative scheme internally. Non-bridged passthrough
variables get outer indices via IndexMap when active.
- Two stacked SingleBridgeOptimizers succeed at add_constrained_variables — the outer pulls inner-allocated indices (positive) through
_record_inner_variable! while its own bridges keep negative outer indices.

Test status on test/Bridges/General/test_bridge_optimizer.jl:
- test_CallbackVariablePrimal errors — MOI.get(b, attr, vi) doesn't translate vi outer→inner yet
- test_nesting_SingleBridgeOptimizer still 3 fails — the cnn CI{VOV, Nonneg} collision case, same as before. Will be fixed by the
constraint-mapping plumbing.
- Rest pass.

Big remaining work (rough order):
1. Translate vi in MOI.is_valid, MOI.delete, MOI.get, MOI.set, MOI.modify, MOI.supports and the callback / primal / dual attribute getters
2. Translate functions containing variable indices via MOI.Utilities.map_indices when crossing the boundary (ConstraintFunction,
ObjectiveFunction, VariablePrimal, change types, etc.)
3. Constraint mapping per (F, S) — activate on first VI/VOV force-bridge, translate at boundaries
4. Remove negative-index conventions
Core namespace machinery (src/Bridges/bridge_optimizer.jl):
- outer_to_inner(b, idx) / inner_to_outer(b, idx) — the canonical "if variable bridges are used → use the index map, else identity"
helpers, for both VariableIndex and ConstraintIndex{F,S}. CI{VariableIndex,S} translation is derived from the variable mapping (since
ci.value == vi.value by MOI convention), so it needs no stored entries.
- _OuterToInner / _InnerToOuter / _TotalInnerToOuter functor structs (<: Function so they work with map_indices). The "total" variant
leaves unrecorded indices unchanged — those are bridge-created inner variables later removed by unbridged_function.
- _to_inner_value / _from_inner_value — whole-value translation at the b.model boundary, applied via map_indices.
- _unbridged_result_from_inner / _unbridged_result_from_bridge — result processing that respects where a value came from:
recursive_model(b) === b (Lazy, outer namespace) vs b.model (SBO, inner namespace).

Key insight encoded in bridged_variable_function: for LazyBridgeOptimizer the substituted expressions stay outer (translation happens at
the model boundary); for SingleBridgeOptimizer they're already inner, so passthrough variables translate during substitution and the
recursion is skipped.

Plumbed sites: add_constraint(s), add_constrained_variable(s), delete (vi/ci/vectors, with map cleanup), is_valid,
variable/constraint/model attribute get/set, ObjectiveFunction get, modify (with _to_inner_change and _modify_substituted_change to avoid
double-translating decomposed modifications), VariableName/ConstraintName get/set, name→index lookups (including reverse-lookup of names
delegated to bridge-created variables), ListOfVariableIndices.

Activation triggers: variable mapping activates on first variable bridge; per-(F,S) constraint mapping activates on first force-bridged
VOV; add_bridged_constraint avoids identity-copied indices via _is_available_constraint_index.

One Utilities change: substitute_variables(map, x::VariableIndex) now allows index-to-index renaming, erroring only when bridged into a
real function.

Test updates (3 places where tests reached around the bridge layer to the mock with outer indices — they now use the inner index,
consistent with how they already handled y).
…StackOverflow is gone).

Remaining failures, all test-only, two kinds:
1. Hardcoded old negative values (e.g. test_FreeBridge:138 expecting [-1,-2]) — trivially update to the new positive values.
2. Cross-layer is_valid isolation (test_nesting lines 648, 664–666: !is_valid(model, x), !is_valid(b0, cnn), etc.). These assert that a
bridged entity from an upper layer is invalid in lower layers. With the old negatives this held for free; with positive translated indices
a bridged variable's value (x=1) coincides with an inner-model variable, so a direct is_valid(model, x) is a false positive. Real
operations are unaffected because they route through the proper layer — only these direct cross-namespace value comparisons see the
coincidence.

The new design's guarantee is "operations through the correct layer are correct," not "indices are globally unique across layers" (which
is what the old negative encoding incidentally provided).
Progress: 20→10 lazy failures (PSD fixed). The remaining 10 are all in test_linear_integration with the restricted SDPA model — MOI.get(ObjectiveFunction) comes back with untranslated indices
(VariableIndex(7,9) instead of the user's v1,v2) and a wrong constant. This is the objective-function unbridging/translation path for deeply-bridged Lazy models — another cross-namespace translation site, but
a non-trivial one (involves the bridge's substitution chain, not just index relabeling).

Let me take stock and give you a clear status, since this has become an extended whack-a-mole and you should decide the next move.

Where things stand

Done (committed by you through e2d8d2a + my newer uncommitted edits):
- Full positive-index conversion: Variable/map.jl (slot bijection), bridge_optimizer.jl dispatch (is_bridged is now a map lookup), Constraint/map.jl (positive VOV).
- Six collision-specific fixes, each from the outer-bridged-value == inner-created-value coincidence that negative indices used to prevent:
  a. unbridged_function infinite recursion (gated on recursive_model)
  b. _get_all_including_bridged chain-loop self-loop hang (gated on recursive_model)
  c. _remove_bridged wrongly emptying the variable list (no-op for ListOfVariableIndices)
  d. vector-member expansion value - i → value + i (ascending), 2 sites
  e. has_keys ordering check (slot-consecutive → ascending-value, allows post-deletion gaps)
  f. Map constructor next_outer_variable default 0 → 1 (self-contained)
  g. main-loop "raw inner first" lookup → always-translate (fixed the PSD duplicate)
- Test updates: test_FreeBridge, test_NonposToNonneg, test_VectorizeBridge, test_map, test_nesting (dropped cross-layer is_valid isolation assertions that positive indices can't provide).

Verified green: test_bridge_optimizer.jl (General) ✅, all 11 test/Bridges/Variable/ ✅.

Remaining: lazy suite — 10 failures in test_linear_integration (objective unbridging for the restricted SDPA model). Constraint suite untested (needs the CliqueTrees test dep). Likely a few more
cross-namespace sites lurk in the deeply-bridged Lazy paths.
Fixes the solver-tests regressions of the index-space redesign:

* After deleting a constraint-bridged `VariableIndex`-in-`S` constraint,
  `is_bridged` returned `false`, so `delete`/`get` were forwarded to the
  solver which threw `DeleteNotAllowed`/`GetAttributeNotAllowed` instead
  of `InvalidIndex` (GLPK, MosekTools, COPT, SCIP: Semicontinuous,
  Semiinteger, ZeroOne). Restore the master fallback: it is bridged if
  `b.model` supports neither `VariableIndex`-in-`S` nor variables
  constrained in `S`.
* Without variable bridges, constraint-bridged `VectorOfVariables`
  indices were taken from the `Constraint.Map` counter, so a deleted one
  was forwarded to the solver (Gurobi, HiGHS, CPLEX, Xpress, COPT, SCIP,
  Tulip: AllDifferent, BinPacking, Nonnegatives, HyperRectangle) and it
  could alias a `CI{VectorOfVariables,S}` created by
  `add_constrained_variables` of `b.model`. Always allocate them from the
  `(VectorOfVariables, S)` index space, whose mapping no longer requires
  variable bridges to be active.
* Define `is_bridged_variable` and `is_constraint_mapping_active` for
  `Variable.EmptyMap`, needed by bridge optimizers defined outside MOI
  (MathOptComplements), and remove the now redundant `is_bridged`
  overrides of the single bridge optimizers so that this path is tested.
Exercise the missing-variable branch in constraint index translation, the only added line missed by the PR Codecov report. Include direct dependencies used by standalone tests in test/Project.toml so that the explicit test environment also works in CI.
@blegat
blegat force-pushed the bl/index-space-redesign branch from 46f7e98 to b9362e0 Compare November 6, 2026 16:20
blegat added a commit that referenced this pull request Nov 6, 2026
ZeroOneBridge's internal Integer constraint is absent from constraint lists
but still reported as valid. This makes JuMP treat a bridged binary as
integer and breaks fix_discrete_variables and its undo operation.

Filter native validity using the constraints reported by variable,
constraint, and objective bridges. Compare in the recursive model's
namespace: lazy bridges store outer indices, while single bridges store
inner indices. Preserve recursive bridge attribute and deletion dispatch.

This version is based on PR #3013, head
6145229. That PR fixes the original
vector-index reproducer of #2696, but not the hidden Integer constraint.
The corresponding fix on current master is 72eb949b.

Validation on this base:
- Modified ZeroOne bridge file: 227 assertions passed, including active
  variable mapping with distinct inner and outer variable indices.
- New variable/objective visibility tests: 10 assertions passed.
- Original vector and binary issue #2696 reproductions: both passed.
- JuMP/Mosek bridged-direct reproduction: 22 assertions passed, including
  fixing, LP dual = 1.0, undo, relax_integrality, and re-solving the MILP.
- JuliaFormatter 2.14.7 leaves no unstaged tracked diff.

The complete modified general bridge-optimizer file additionally passes
823 assertions on current master.

Addresses #2696 and jump-dev/MosekTools.jl#222.
blegat added a commit that referenced this pull request Nov 6, 2026
Replace per-query scans with a sparse registry of bridge-created constraint
indices. Allocate it only when a bridge creates a child constraint; never
enumerate or copy existing user constraints. Validity checks use expected
O(1) dictionary lookups and allocate no memory in the benchmark below.

Record each new bridge's children and refresh only the affected bridge's
entries on setters, modifications, final_touch, and deletion. Reference
counts preserve overlapping reports. Refresh after failed mutations too,
and release empty registries.

Extend bridge call contexts so recursive operations can still access their
children while public queries and direct deletion hide them. Preserve the
distinct public and inner namespaces of single-bridge wrappers, including
identity bridges with coincident index values. Cover variable, constraint,
and objective bridges and clear ownership state on empty!.

Validation (only focused test files, not the full suite):
- General bridge optimizer: 894 passed, 1 pre-existing expected broken.
- ZeroOne bridge: 233 passed, including nested bridged children and active
  variable index mapping.
- Ownership/context regressions: 35 passed, including a counter proving
  1,000 validity queries do not inspect bridge child lists, sparse storage
  with 1,000 pre-existing native constraints, recursive access, final_touch,
  exception recovery, bulk deletion, and registry cleanup.
- JuMP/Mosek bridged direct model: 22 passed, including MILP solves,
  fixing, LP dual = 1.0, undo, relaxation, and re-solving.
- Temporary @show @__LINE__ tracing exercised 30 of 31 instrumentation
  points. The optional no-context fallback for third-party optimizer
  subtypes was not exercised. Tracing was removed afterward.
- JuliaFormatter 2.14.7 over the repository leaves no unstaged diff.

Query benchmark: is_valid for one native user Integer constraint in a
ZeroOne SingleBridgeOptimizer with the indicated number of bridged binary
variables. Julia 1.13.3, BenchmarkTools minima of 100 samples, evals=1.
Only the is_valid method was swapped between the original PR #3013 code,
the previous scanning fix, and sparse ownership; the models and remaining
implementation were identical. Times are nanoseconds per query.

| Bridged binaries | Original ns | Scanning ns | Sparse ns | Sparse allocations |
| ---: | ---: | ---: | ---: | ---: |
| 0 | 12.0 | 51.0 | 13.0 | 0 |
| 1 | 12.0 | 75.0 | 25.0 | 0 |
| 1000 | 12.0 | 23797.0 | 26.0 | 0 |
| 10000 | 12.0 | 237965.0 | 28.0 | 0 |

Addresses #2696 and jump-dev/MosekTools.jl#222 on top of PR #3013.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

MOI.is_valid can be falsely positive for bridged constraints

2 participants