From b3053a41c25512d235c1a09a0fa907ab425833dd Mon Sep 17 00:00:00 2001 From: Saim Shafique Date: Fri, 20 Nov 2026 05:22:40 +0000 Subject: [PATCH 1/2] fix(pstack): log Hillclimb attempts in the show-me-your-work format Hillclimb step 3 routed its decision log via show-me-your-work but defined its own file name (decision.tsv) and nine columns, while show-me-your-work owns a six-column format and log.sh exits 1 on any other argument count. Map each attempt onto the canonical columns instead of restating a competing format, and point the Reply line at the decision log path rather than the invented file name. Fixes #1297 --- pstack/skills/poteto-mode/playbooks/hillclimb.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pstack/skills/poteto-mode/playbooks/hillclimb.md b/pstack/skills/poteto-mode/playbooks/hillclimb.md index dbcff8f923..5a94362fab 100644 --- a/pstack/skills/poteto-mode/playbooks/hillclimb.md +++ b/pstack/skills/poteto-mode/playbooks/hillclimb.md @@ -6,7 +6,7 @@ Core discipline: one change, one measurement, keep or revert. Never stack untest 1. Ground the workload and architecture before choosing the metric. Trace the target, name the realistic workload dimensions that can move the result (data size, history, state, concurrency), and select a case that reproduces the user's complaint. If no case reproduces it, fix the repro instead of hillclimbing. Then fix one metric, the direction that counts as better, and a checkable stop predicate that pairs a target with a floor on attempts so a lucky early win can't end the run (the example "at least 50% better than baseline and at least 10 iterations" is this shape). Use the user's numbers when given, otherwise agree them. 2. Build the measurement harness, prove its sensitivity, then freeze it (the **build-the-lever** principle skill). Run contrasting realistic workloads and confirm the target case reproduces the symptom while easier cases separate as expected. If the harness cannot distinguish them, revise the workload or metric. Vet the harness with the **benchmark-checklist** skill before you freeze it, and make it print its error count and a count of the work done. Once frozen, one repeatable command emits the metric, sampled enough to clear the noise (median of N, not a single run). Record the baseline metric and a green run of the regression gate (the tests that must keep passing) before any change. -3. Open the decision log via the **show-me-your-work** skill. A `decision.tsv`, one row per attempt: id, hypothesis, change, before, after, delta, tests, verdict (kept or reverted), note. Read it before each attempt. Keep it out of the tree (gitignored). +3. Open the decision log via the **show-me-your-work** skill, one row per attempt, in its canonical format (it owns the file name and columns). Map each attempt onto it: phase is the attempt id, decision is the change, why is the hypothesis, evidence is the commit plus the before and after numbers and the tests run, result is kept or reverted with the delta. Read it before each attempt. Keep it out of the tree (gitignored). 4. Ground each hypothesis in the architecture model from step 1, so it names a specific mechanism ("defer X off the boot path because it blocks first paint"), not "try memoizing something". For a perf metric, order hypotheses by the performance mantras in step 2 of the Perf issue playbook (`playbooks/perf-issue.md`). Borrow only their order, not that step's stop rule. 5. Loop, one hypothesis per iteration: - Hand the change to a subagent using your configured hillclimb model (default `claude-opus-5-5-xhigh-fast`) with a tight scope. Pick each subagent's reasoning level for the job, per the Subagents section. Supervise and review the diff rather than typing it (the **guard-the-context-window** principle skill). When several independent hypotheses are live, fan them to parallel subagents, each in its own worktree (the **separate-before-serializing-shared-state** principle skill). @@ -18,4 +18,4 @@ Core discipline: one change, one measurement, keep or revert. Never stack untest 7. Stop when the predicate is met, or when the remaining ideas are marginal and not worth their cost. Don't relax the predicate to meet it, and don't quit while cheap untried hypotheses remain. If you are stuck, surface it instead of spinning. 8. Run **Opening a PR** with the accepted commits stacked in the order they landed. -**Reply:** the metric and target, baseline to final with the percent delta, iterations run (kept vs reverted), each accepted fix on one line, the `decision.tsv` path, and the best idea you would try next if pushed further. +**Reply:** the metric and target, baseline to final with the percent delta, iterations run (kept vs reverted), each accepted fix on one line, the decision log path, and the best idea you would try next if pushed further. From 02651d57cb5f0eaafb19b13a4d276732e90e5af2 Mon Sep 17 00:00:00 2001 From: Saim Shafique Date: Fri, 20 Nov 2026 16:21:59 +0000 Subject: [PATCH 2/2] chore(pstack): regenerate harness trees for the hillclimb log fix The Validate plugins workflow checks that the generated harness trees match the pstack sources. Regenerate them after moving the hillclimb decision log onto the show-me-your-work format. --- .../claude-code/skills/poteto-mode/playbooks/hillclimb.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/harness/pstack/claude-code/skills/poteto-mode/playbooks/hillclimb.md b/harness/pstack/claude-code/skills/poteto-mode/playbooks/hillclimb.md index dbcff8f923..5a94362fab 100644 --- a/harness/pstack/claude-code/skills/poteto-mode/playbooks/hillclimb.md +++ b/harness/pstack/claude-code/skills/poteto-mode/playbooks/hillclimb.md @@ -6,7 +6,7 @@ Core discipline: one change, one measurement, keep or revert. Never stack untest 1. Ground the workload and architecture before choosing the metric. Trace the target, name the realistic workload dimensions that can move the result (data size, history, state, concurrency), and select a case that reproduces the user's complaint. If no case reproduces it, fix the repro instead of hillclimbing. Then fix one metric, the direction that counts as better, and a checkable stop predicate that pairs a target with a floor on attempts so a lucky early win can't end the run (the example "at least 50% better than baseline and at least 10 iterations" is this shape). Use the user's numbers when given, otherwise agree them. 2. Build the measurement harness, prove its sensitivity, then freeze it (the **build-the-lever** principle skill). Run contrasting realistic workloads and confirm the target case reproduces the symptom while easier cases separate as expected. If the harness cannot distinguish them, revise the workload or metric. Vet the harness with the **benchmark-checklist** skill before you freeze it, and make it print its error count and a count of the work done. Once frozen, one repeatable command emits the metric, sampled enough to clear the noise (median of N, not a single run). Record the baseline metric and a green run of the regression gate (the tests that must keep passing) before any change. -3. Open the decision log via the **show-me-your-work** skill. A `decision.tsv`, one row per attempt: id, hypothesis, change, before, after, delta, tests, verdict (kept or reverted), note. Read it before each attempt. Keep it out of the tree (gitignored). +3. Open the decision log via the **show-me-your-work** skill, one row per attempt, in its canonical format (it owns the file name and columns). Map each attempt onto it: phase is the attempt id, decision is the change, why is the hypothesis, evidence is the commit plus the before and after numbers and the tests run, result is kept or reverted with the delta. Read it before each attempt. Keep it out of the tree (gitignored). 4. Ground each hypothesis in the architecture model from step 1, so it names a specific mechanism ("defer X off the boot path because it blocks first paint"), not "try memoizing something". For a perf metric, order hypotheses by the performance mantras in step 2 of the Perf issue playbook (`playbooks/perf-issue.md`). Borrow only their order, not that step's stop rule. 5. Loop, one hypothesis per iteration: - Hand the change to a subagent using your configured hillclimb model (default `claude-opus-5-5-xhigh-fast`) with a tight scope. Pick each subagent's reasoning level for the job, per the Subagents section. Supervise and review the diff rather than typing it (the **guard-the-context-window** principle skill). When several independent hypotheses are live, fan them to parallel subagents, each in its own worktree (the **separate-before-serializing-shared-state** principle skill). @@ -18,4 +18,4 @@ Core discipline: one change, one measurement, keep or revert. Never stack untest 7. Stop when the predicate is met, or when the remaining ideas are marginal and not worth their cost. Don't relax the predicate to meet it, and don't quit while cheap untried hypotheses remain. If you are stuck, surface it instead of spinning. 8. Run **Opening a PR** with the accepted commits stacked in the order they landed. -**Reply:** the metric and target, baseline to final with the percent delta, iterations run (kept vs reverted), each accepted fix on one line, the `decision.tsv` path, and the best idea you would try next if pushed further. +**Reply:** the metric and target, baseline to final with the percent delta, iterations run (kept vs reverted), each accepted fix on one line, the decision log path, and the best idea you would try next if pushed further.