Repository navigation
MAINT: Make shell command scorer helpers keyword-only - #4322
Open
Aman (its-amann) wants to merge 4 commits into
Open
Aman (its-amann) wants to merge 4 commits into
Aman (its-amann) wants to merge 4 commits into
Conversation
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The review reports no active issues with the keyword-only migration of the six private scorer helpers.
No open issues 🎉
What changed in this PR
This PR advances #4216 by migrating six private pattern-building helpers in ShellCommandOutputScorer to keyword-only arguments, without changing the public scorer API or regex matching behavior.
Changes:
- Makes all six helpers keyword-only and updates their callers.
- Adds parameterized signature coverage alongside existing behavior tests.
- Removes the six migrated entries from the keyword-only baseline.
| File | Description |
|---|---|
tests/unit/score/regex/test_shell_command_output_scorer.py |
Adds signature checks for all six helpers. |
pyrit/score/true_false/regex/shell_command_output_scorer.py |
Converts helper signatures and calls to keyword-only arguments. |
build_scripts/keyword_only_baseline.txt |
Removes the six migrated helper exemptions. |
Review effort: Balanced
Varun Poojari (Varunpoojari)
left a comment
Contributor
There was a problem hiding this comment.
I went through the diff, and the keyword-only changes look consistent:
- All six converted helpers (
_quoted_option,_download_command,_curl_option,_curl_number_option,_curl_mode_then,_shell_options) have their call sites updated in this PR, and the six entries removed fromkeyword_only_baseline.txtare exactly those. - One thing to confirm.
_DOWNLOADwas restructured from a single f-string into+concatenation, and severalrf"..."strings were split. In a module that builds very large regexes, it is easy to change the pattern by accident (a missing|or parenthesis). Could you confirm the compiled_DOWNLOADpattern string is byte-identical to the one onmain(for example by comparing the two strings once locally)? If not already covered, a test pinning the pattern, or a few representative commands, would guard against this kind of refactor.
…shell-scorer-keyword-only-4216 # Conflicts: # build_scripts/keyword_only_baseline.txt
Roman Lutz (romanlutz)
approved these changes
Nov 19, 2026
The narrow-viewport test measured the baseline and attack success-rate tiles immediately after opening the comparison, racing the layout settling into one row (seen as a 54px y offset that passed on retry, failing CI under --fail-on-flaky-tests). Wrap the measurement in expect().toPass() like the other E2E specs so it waits for the settled layout instead of asserting a single snapshot. Co-Authored-By: Claude Fable 5 <[email protected]>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Relates to #4216. Makes the six private pattern-building helpers in
shell_command_output_scorerkeyword-only and updates their callers.Roman Lutz (@romanlutz) this covers the module assigned to me.