Skip to content

FEAT: TAP takes success from the scorer's verdict and ranks by kept float feedback - #4354

Open
WatchTree-19 wants to merge 19 commits into
microsoft:mainfrom
WatchTree-19:feat/tap-float-feedback
Open

WatchTree-19 wants to merge 19 commits into
microsoft:mainfrom
WatchTree-19:feat/tap-float-feedback

Conversation

@WatchTree-19

Copy link
Copy Markdown
Contributor

Stacked on feat/scorer-pool-selection, opened alongside; only the commits after it are new here.

Description

This is step 2 of the scorer capabilities design (https://gist.github.com/richlundeen/302e4a8cf581dae4d744935a4ce7ad7c). TAP no longer needs a FloatScaleThresholdScorer or reads a threshold: success comes from the objective scorer's true/false verdict, and branches are ranked by the float value that scorer keeps.

  • FloatFeedback in pyrit.models owns the existing original_float_value key and its readers and writers, so stored scores need no migration. Ordinary reads treat missing or unusable data as unset, strict reads reject it
  • FloatScaleThresholdScorer guarantees the value, TrueFalseFallbackScorer does when both of its scorers do, and inverters clear it
  • A scorer that promises the value is held to it at the public, nested, and reused result boundaries
  • TAP ranks a true verdict first and then complete scores by their kept value, so a success at 0.60 is never pruned for a failure at 0.65. Undetermined results keep their outcomes
  • TAPAttackScoringConfig.threshold is deprecated and only works for threshold-backed scorers
  • TAP declares the requirement on the attack class. The matrix builder picks a scorer that keeps the value and passes it in TAP's own config type, the adaptive scenario skips TAP with the reason, and Execute raises, so a scenario scorer without the value is no longer swapped for TAP's default judge
  • Test fixtures that mocked TAP objective scores as raw floats now mock what the threshold scorer returns (the verdict plus the kept value), so the existing assertions run against realistic scores

Scenarios that opt in (RapidResponse, Cyber, Leakage) accept a CompatibleScorers
pool in place of one objective scorer. Each candidate can be bound to an
evaluation key, datasets name the key they need on DatasetSource, and techniques
state algorithm needs such as a float value on true/false results.

The matrix builder selects once per technique and dataset, without calling a
judge model. A candidate qualifies when it accepts the attack's evidence, carries
the required key, meets the technique's needs, and accepts every objective's
conditions. Pool order breaks ties. When nothing qualifies, the existing
incompatible technique policy skips or raises, naming each candidate's reason and
the keys the pool has.

Scorer validation now raises typed errors (UnsupportedConditionError,
MissingConditionError, AmbiguousConditionError) that subclass ValueError and keep
their messages, so selection tells a scorer mismatch apart from malformed input.
Scorer.capabilities exposes what a scorer accepts, derived from its contracts.

Pool runs record each selection in the run plan with the fingerprints of the
prepared expectations. Resume re-selects and refuses to continue on drift. Each
dataset gets its own baseline judged by the same scorer as its attacks. Runs with
one scorer keep their plan shape, group ids, baseline, and identity.
…loat feedback

TAP no longer requires a FloatScaleThresholdScorer or reads a threshold. Any
true/false objective scorer that guarantees a float value on each complete
verdict qualifies, which FloatScaleThresholdScorer does and a
TrueFalseFallbackScorer does when both of its scorers do. Inverters clear the
guarantee.

FloatFeedback in pyrit.models owns the existing original_float_value key and
its readers and writers, so stored scores need no migration. Ordinary reads
treat missing or unusable data as unset; strict reads reject it. A scorer that
promises the value is held to it at the public, nested, and reused result
boundaries.

TAP ranks a true verdict first, then complete scores by their kept value, so a
successful branch at 0.60 is never pruned for a failed one at 0.65.
Undetermined results keep their outcomes. TAPAttackScoringConfig.threshold is
deprecated and works only for threshold-backed scorers.

TAP declares the requirement on the attack class, and scenarios check it
before building TAP: the matrix builder selects a scorer that keeps the value
and passes it in TAP's own config type, the adaptive scenario skips TAP with
the reason, and Execute raises. A scorer without the value is no longer
replaced with TAP's default judge.
Copilot AI balanced review requested due to automatic review settings November 17, 2026 16:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

TAP receives the wrong scoring-config subtype, causing it to use its default judge instead of the supplied scorer for branching and success.

4 open issues
What changed in this PR

This PR builds on scorer-pool selection to make TAP use the objective scorer’s verdict for success and its preserved float feedback for branch ranking.

Changes:

  • Adds float-feedback models and enforces scorer guarantees at scoring and reuse boundaries.
  • Integrates scorer requirements, selection, and saved bindings into scenario construction.
  • Updates TAP ranking, regression tests, and usage documentation.
File Description
tests/​unit/​score/​test_scorer_selection.py Tests scorer-pool compatibility and selection.
tests/​unit/​score/​test_float_feedback_guarantee.py Tests feedback guarantees and reuse validation.
tests/​unit/​scenario/​scenarios/​technique/​test_execute.py Tests rejection of incompatible TAP scorers.
tests/​unit/​scenario/​scenarios/​adaptive/​test_text_adaptive.py Tests skipping techniques requiring float feedback.
tests/​unit/​scenario/​scenarios/​adaptive/​test_image_technique_adaptive.py Makes mock scorer capabilities explicit.
tests/​unit/​scenario/​core/​test_matrix_scorer_pool.py Tests matrix selection, baselines, and TAP configuration.
tests/​unit/​scenario/​airt/​test_rapid_response_scorer_pool.py Tests pool identity, readiness, and resume checks.
tests/​unit/​models/​test_float_feedback.py Tests feedback validation and metadata handling.
tests/​unit/​executor/​attack/​test_attack_parameter_consistency.py Adds preserved float values to TAP fixtures.
tests/​unit/​executor/​attack/​multi_turn/​test_tree_of_attacks.py Tests verdict-first ranking and scorer requirements.
tests/​unit/​executor/​attack/​multi_turn/​test_pair.py Clarifies plain scorer mock capabilities.
pyrit/​score/​true_false/​true_false_inverter_scorer.py Uses the shared feedback-removal helper.
pyrit/​score/​true_false/​float_scale_threshold_scorer.py Declares and writes preserved float feedback.
pyrit/​score/​scorer.py Exposes capabilities and validates feedback guarantees.
pyrit/​score/​scorer_selection.py Adds ordered scorer pools and compatibility checks.
pyrit/​score/​score_utils.py Keeps the legacy metadata-key alias.
pyrit/​score/​message_scorer.py Raises typed expectation-mismatch errors.
pyrit/​score/​fallback_scorer.py Derives feedback guarantees from both children.
pyrit/​score/​expectation_errors.py Defines typed expectation-validation errors.
pyrit/​score/​__init__.py Exports selection APIs and errors lazily.
pyrit/​scenario/​scenarios/​technique/​execute.py Checks technique float-feedback requirements.
pyrit/​scenario/​scenarios/​airt/​rapid_response.py Enables objective scorer pools.
pyrit/​scenario/​scenarios/​airt/​leakage.py Enables objective scorer pools.
pyrit/​scenario/​scenarios/​airt/​cyber.py Enables objective scorer pools.
pyrit/​scenario/​scenarios/​adaptive/​adaptive_scenario.py Handles incompatible scorer configurations.
pyrit/​scenario/​core/​technique_requirements.py Combines attack and technique scorer requirements.
pyrit/​scenario/​core/​scenario.py Tracks pool readiness, identity, and resume bindings.
pyrit/​scenario/​core/​matrix_atomic_attack_builder.py Selects scorers and builds matching configurations and baselines.
pyrit/​scenario/​core/​dataset_configuration.py Adds dataset-level scorer requirements.
pyrit/​scenario/​core/​atomic_attack.py Stores the selected scorer binding.
pyrit/​models/​score/​scorer_capabilities.py Defines the scorer capability view.
pyrit/​models/​score/​float_feedback.py Centralizes float-feedback metadata and validation.
pyrit/​models/​score/​__init__.py Exports feedback and capability models.
pyrit/​models/​scenario_progress.py Adds saved scorer-selection bindings.
pyrit/​models/​__init__.py Exports new shared models.
pyrit/​executor/​attack/​multi_turn/​tree_of_attacks.py Uses verdict-first ranking and deprecates threshold access.
pyrit/​executor/​attack/​multi_turn/​pair.py Updates scorer-contract documentation only.
pyrit/​executor/​attack/​core/​attack_strategy.py Adds attack-level scorer requirements.
doc/​code/​scenarios/​1_common_scenario_parameters.py Documents per-dataset scorer pools.
doc/​code/​scenarios/​1_common_scenario_parameters.ipynb Adds the matching scorer-pool notebook documentation.
doc/​code/​executor/​2_multi_turn.py Documents TAP verdict and feedback semantics.
doc/​code/​executor/​2_multi_turn.ipynb Adds the matching TAP notebook documentation.

Review effort: Balanced

Comment thread pyrit/score/scorer.py Resolved
…d group ids

A technique whose scoring config cannot carry the pool's selected scorer is now
skipped or raised instead of silently keeping its default judge. The single-scorer
baseline leaves out datasets the scorer cannot judge. A technique with no scorer
requirements keeps the identity it had before the field existed. Selection
fingerprints the expectations the candidate prepared, and the plan and
AtomicAttack share one group id. Scenarios that do not check dataset scorer
requirements reject datasets that declare them.
…nested float feedback

Execute passed a plain scoring config, which a technique that narrows its config
type drops, so TAP judged with its own default scorer. It now builds the type the
attack accepts around the scenario's scorer. A scorer that promises float feedback
is also held to it when a wrapper scores an in-hand message.
- Pass over pool candidates a technique's narrowed scoring config cannot carry
  (its declared objective_scorer type, as AdaptiveScenario decides it), so a later
  fitting candidate is chosen; config construction errors now propagate
- Skip the single-scorer baseline when no dataset remains instead of failing on
  an empty seed list
- Count any pool candidate's score as the objective score in conversation output
- Reject a required evaluation no candidate is bound to when the run is configured
- Fold ENFORCES_DATASET_SCORER_REQUIREMENTS into SUPPORTS_OBJECTIVE_SCORER_POOL,
  drop two unread run-plan binding fields, and reuse fingerprint_expectations
- Test resume through _apply_persisted_run_plan, the Cyber and Leakage opt-in,
  compound requirements, and the requirement guards

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ion, and slim the run plan

- Check each dataset once against the scorer, before any technique, and raise when no
  allowed scorer can judge its objectives, whatever the incompatible technique policy;
  technique needs (TAP's threshold scorer, float feedback) keep the policy
- A candidate bound to an evaluation judges only datasets that require it; a dataset that
  requires none needs an unbound candidate, checked at configuration
- Drop expectation fingerprints from the run-plan binding and the resume drift check: group
  ids already key the selection and resume replays the stored seeds
- Build one pool baseline per dataset and judge, named by candidate index
- Record a pool as its own identity child instead of the run's objective scorer, and show
  each attack's recorded judge in conversation output
- Trim the scorer-pool doc section to usage

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ut call sites

- ScorerSelection keeps only the candidate and its pool index; nothing read its
  requirements or skipped reasons, and the narrowed-config pre-filter no longer remaps them
- Docstrings no longer describe recorded inputs or a single scenario objective scorer
- Conversation output and both CLI conversation views are tested with a pool run's
  per-attack judges

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The pool's identity is its ordered candidates and their evaluation keys. Run scorer
requirements no longer feed it: a requirement that changes a selection already changes
the run plan's group ids, and one that keeps the same judge is the same work.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The example now builds real per-harm judges (Azure Content Safety's hate and violence
categories behind threshold scorers) in a code cell instead of a fenced block in markdown.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Integrate the predecessor before fixed main, keeping both sides' behavior:
- attack_strategy.py: combine imports (CapabilityName and AttemptObserver).
- matrix_atomic_attack_builder.py: keep microsoft#4353's dataset guard
  (_check_datasets_can_be_judged), per-candidate baselines, and the
  annotation-based carried scorer filter; keep the PR's module-level
  build_technique_scoring_config (single-scorer narrowing, None on a
  ValueError refusal) and restore _reject for that path. Other config
  construction errors propagate, as microsoft#4353 required.
- test_matrix_scorer_pool.py: keep both sides' tests; point microsoft#4353's
  annotation-filter tests at a config that still narrows its annotation,
  since TAP's config now carries any TrueFalseScorer; keep the PR's
  single-scorer narrowing tests.

Co-authored-by: Copilot App <[email protected]>
Main is the squash of microsoft#4353 and its tree equals microsoft#4353 head 9fb6507,
which the previous merge already integrated. The five conflicted files
(models/__init__.py, technique_requirements.py, score/scorer.py,
matrix_atomic_attack_builder.py, test_matrix_scorer_pool.py) resolve
to that merge result; this merge changes no content.

Co-authored-by: Copilot App <[email protected]>
@richlundeen

Copy link
Copy Markdown
Contributor

review:5942126440

Addressed the four concerns from this review in 80f10d9, including the predecessor's dataset and baseline guards. The specific resolutions are recorded in each original inline thread: Execute config, dataset requirements, baseline populations, and nested scores.

Verified on published commit 80f10d9f16181425697a72ea853d6efcd6e6ebd4.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review completed. No blocking concerns remain. This review records source coverage; merging is handled separately.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants