Skip to content

Three tests still assert wall-clock upper bounds and can flake under load #206

Closed

Description

@TMHSDigital

Summary

Three tests still fail if the machine is slow, because they assert that something finished within a wall-clock limit. #167 removed this pattern from the pool, lab and SARIF tests; these three remain. A loaded CI runner (-race, shuffled order) can push them over the limit with no bug present.

Where

  • internal/dns/ratelimit_test.go:49 (rate limiter: fails if the run took more than 1.5x the ideal time plus 100 ms)
  • internal/dns/resolver_test.go:85 (cancelled context: fails if ResolveDomainWithRetry took more than 500 ms)
  • internal/dnsserver/server_test.go:74 (Close with a parked stream connection: fails above 2 s)

Details

if elapsed > ideal*15/10+100*time.Millisecond {
	t.Errorf("finished in %s, well below the configured rate (ideal %s)", elapsed, ideal)
}

Lower bounds (for example "the rate limiter cannot be faster than the rate") are fine: load only makes things slower. Upper bounds are what flake.

Impact

Intermittent red CI that has nothing to do with the change under test, which teaches people to re-run instead of reading failures (the problem #167 described).

Suggested fix

  • Rate limiter: keep the lower bound; replace the upper bound with a deterministic check, for example count tokens granted per tick with a fake clock, or drop it and rely on the lower bound plus a generous ceiling (5x) that only catches a hang.
  • Cancelled context: assert the outcome and that no query reached the server (the test server counts queries) instead of timing the return; or raise the limit to a hang-only ceiling such as 5 s.
  • Close: the real property is that Close returns while a stream connection is parked; run it in a goroutine and fail only on a hang-only ceiling (for example 10 s), or check that the handler count is zero after Close returns, which the test already does.

Acceptance criteria

  • No test fails because a passing operation took longer than a tight wall-clock limit
  • go test -race -count=5 -shuffle=on ./internal/dns/ ./internal/dnsserver/ passes with GOMAXPROCS=2

Getting started


Filed by repo-audit/fix-issues at e5e38c3. Severity: low. Effort: S. audit-key: testing|internal/dns/ratelimit_test.go|elapsed upper bound

Activity

  1. TMHSDigital commented on Nov 9, 2026

    @TMHSDigital
    OwnerAuthor

    Fixed by 30d50f9 (#207) and 3a30307. Acceptance run (-race -count=5 -shuffle=on, GOMAXPROCS=2) passes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueGood for newcomersseverity: lowMinor polish and nice-to-havestestingTest coverage and test infrastructure

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions