Repository navigation
fix: straight-line calibration fallback for references with few distinct peptidoforms - #138
Merged
Merged
Conversation
…tidoforms MultiHeadRidgeCalibration combined 80 heads with per-head splines and a ridge even when the reference held only a handful of distinct peptidoforms. Repeated PSMs of one peptidoform were counted as separate points, so a reference of 1,125 PSMs from 15 peptidoforms (or 169 PSMs from 2) passed every size check. With so few distinct points the head ranking, splines and ridge are underdetermined and rounding differences in the predictions decide the fit, giving predictions of +-1e7 s that change with the thread count (#136). Distinct peptidoforms are now counted from the predictions themselves (tolerance grouping over the five best heads). Below min_distinct=200 the calibration fits a straight line on the best-correlating head and logs a warning; below three distinct peptidoforms fit raises CalibrationError. The head cap now uses the distinct count. Threshold from a sweep over 19 held-out setups (k = 5..2,000 distinct references, 10 draws each). Tests that exercise the head combination on small synthetic references opt in with min_distinct=3. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the thread-dependent predictions of ±1e7 s reported in #136.
MultiHeadRidgeCalibrationcombined up to 80 heads with per-head splines and a RidgeCV even when the reference held only a few distinct peptidoforms. Repeated PSMs of one peptidoform counted as separate points. A reference of 1,125 PSMs from 15 peptidoforms, or 169 PSMs from 2, therefore passed every size check. With so few distinct points, the head ranking, the splines and the ridge are underdetermined. Rounding differences in the predictions then decide the fit. These differences come from the torch thread count (about 2e-3 min) and from slicing theFactoredPredictionMatrix(about 1.5e-5 min).Change
min_distinct=200. Distinct peptidoforms are counted from the source matrix itself, by tolerance grouping (1e-3 min) over the five best-correlating heads, so callers that pass duplicated PSMs (such as MS2Rescore, which callsfitdirectly) are covered.min_distinct: a straight line on the best-correlating head, with a warning (marked unreliable below 10).fitraisesCalibrationError.uses_fallbackandn_distinct;disagreementreturnsNonein fallback mode.Why 200
Sweep over 19 held-out setups (holdout_v5, never in training), k = 5 to 2,000 distinct reference peptidoforms, 10 draws per k, test error relative to the setup's RT span:
From 200 the ridge has no failures and is better than the straight line in at least 91 % of paired draws (median MAE ratio 0.77).
Effect on the #136 PSM list
calibration_set_size≈0.15reference (2 peptidoforms)CalibrationErrorTests
5 new tests: duplicated PSMs count once and trigger the fallback; the fallback is insensitive to thread-sized noise; large references still combine heads; refusal below 3; validation of
min_distinct. Three existing tests that exercise head combination on 150-row synthetic references now passmin_distinct=3. Full suite: 255 passed.Follow-up (not in this PR)
MS2Rescore could densify the prediction matrix once and deduplicate the reference before calling
fit.🤖 Generated with Claude Code