Skip to content

Link metadata: join concurrent misses for one URL #791

Closed

Description

@HMarzban

Summary

When N readers open one new link at once, the server starts N scrapes of the same page. The laptop bench saw each request reach the origin, up to the burst size. A layer-2 sketch that joins them cut origin hits to 1 per burst. It also halved p99 at a burst of 500.

Measured

See Benchmark. Coalescing is measured on layer 2 only, with k = 1, and it is a cross-group reading against layer 1. Herd did not run on Linux.

Cause

  • runPipeline (modules/link-metadata/domain/pipeline.ts) has no in-flight join. Each miss runs its own stages until the first write lands.
  • The image route calls runPipeline too (modules/link-metadata/http/imageController.ts).
  • The social-card module already has createSingleFlight (modules/social-card/infra/renderGuards.ts:54).

What to change

  1. Wrap the stage run in runPipeline in a singleflight keyed by the cache key: URL, language and pad generation.
  2. Delete the entry when it settles. The image route shares the same entry.

Decisions

  1. Per process or across replicas? Recommended: per process. At 2 replicas, the worst case is 2 scrapes per link. Other: a Redis SET NX lock, which adds round trips and a stale-lock case.
  2. Reuse createSingleFlight from social-card? Recommended: yes, if its contract fits. Other: a 15-line map in the module.

Benchmark

Laptop before, measured 2026-11-14

Condition: the laptop condition in Bench conditions below. Scenario herd: 5 bursts per size on 5 distinct URLs, 300 ms origin delay.

Burst Layer 1 (20261114T020638, 20261114T033808) origin hits per URL (maxPerUrl) Layer 1 p99 ms Layer 2 module p99 ms Layer 2 coalesce (k = 1): origin hits / p99 ms
200 200 (resets 7.2 % in both) 355.44 / 364.16 364.96-438.78 1 / 321.82
500 500 (origin active max 256) 687.2 / 770.44 686.35-691.75 1 / 343.37
  • Layer 1 sent no coalescing: each request that got through reached the origin. At burst 500 in 20261114T033808, the origin total was 2469 against 2448 answered.
  • At burst 500 the uncoalesced burst runs in two waves behind the 256-slot pool.
  • The layer-1 resets at burst 200 did not occur on layer 2. Their cause is not proven, and coalescing may not fix them.
  • Coalescing lowered REST CPU by 14 to 16 points at 200 and 9 to 14 points at 500.

After

Use the after method in Bench conditions below, on layer 1, at bursts 10, 50, 200 and 500. Report resets apart from the origin-hit count.

Condition

The laptop condition in Bench conditions below. A Linux run of herd is owed (#793).

Content-safety checks

  • The key holds URL, language and pad generation. A joined caller never gets another language or an older pad card.
  • The join shares exactly what the Redis cache already shares across readers. It adds no new exposure.

UX checks

Rollback

One setting turns the join off. Otherwise, revert the commit.

Acceptance criteria

  • Origin hits per burst are 1 at every burst size, on one replica.
  • Every answer is a full card, with 0 degraded.
  • At burst 500, p99 sits below 2 x the origin delay plus 100 ms.
  • A Measured comment posts the after beside the before, with the condition.

Related

Bench conditions

The bench is apps/hocuspocus.server/scripts/bench-link-tolerance/ (commit 1a0d21b7f). The headline numbers are in apps/hocuspocus.server/src/modules/link-metadata/BENCHMARK.md, section Load tolerance. REPORT.md, LINUX-RESULTS.md, research-*.md and raw/ are local bench notes (Notes/local-docs/2026-11-13-link-scraper-tolerance/), not in the repo. The Linux runner is local too.

Linux condition, used by every "Linux before": one REST container at --cpus=1 --memory=512m --memory-swap=512m, in the OrbStack VM on one laptop. Kernel 7.2.12-orbstack-00394-gbecdd6130068, booted with cgroup.memory=nokmem,nosocket, so socket and slab memory are not charged. --no-healthcheck. Bun 1.4.4+38afee6c8. Redis 8.8.0 (redis:alpine) in REST's network namespace. --network none. Pooled loopback to one local origin, no DNS, no TLS. benchTree 8774a6275a05005f, srcTree a9b6ca73ca68c2ec, runnerTree 516099a8c9839eac. Measured 2026-11-14. The core speed ratio to a production vCPU is not measured.

Laptop condition, used by every "Laptop before": local, one laptop (Apple M5 Max, 18 cores, 48 GB, macOS), Bun 1.4.4+38afee6c8, native Redis 8.10.3. REST runs under the sandbox-exec fence. Traffic is plain HTTP to one local origin, pooled loopback, no DNS, no TLS. No CPU quota and no cgroup. benchTree 8774a6275a05005f, srcTree a9b6ca73ca68c2ec. Measured 2026-11-14. Never compare it with a Linux number as one series.

"(void session)" marks a value from a valid step inside a void session. On Linux the OOM kill voided those sessions. The kill is the measured break.

After method, used by every "After":

  • Base = the commit before the change. Head = the change.
  • Run ABAB, at least 3 sessions per arm, each with its control step, at n ≥ 2,000 per step.
  • The change moves srcTree, so the before must run again. Do not reuse the 2026-11-14 numbers as the before.
  • The runner takes src/ from the working tree. So it first needs a source-folder option for base (REPORT.md §5.7 item 5).
  • On Linux, start each session in a fresh REST container, and keep the run agent alive until the footer is written.
  • Quote a change only when every head session sits outside the base range. Give σ and the MDE beside it.

Paths are under apps/hocuspocus.server/src/ unless they say otherwise.

Activity

  1. added this to the v2.3.6 milestone on Nov 14, 2026
  2. modified the milestones: v2.3.6, v2.3.8 on Nov 14, 2026
  3. HMarzban commented on Nov 15, 2026

    @HMarzban
    CollaboratorAuthor

    Measured: before and after (slot 3, 2026-11-15)

    Setup. One REST container at --cpus=1 --memory=512m (OrbStack, Linux cgroup v2), on one laptop. Both arms were built from git archive: base 061fee04d, head f62a5e383, with the same bench. Base and head ran in turns (ABAB, BABA, ABAB). 3 valid sessions per arm. A step is void above 250 % other-process CPU (maintainer ruling, 2026-11-14).

    Bursts of requests for one link that is not in the cache:

    Burst Origin fetches, before After Fallback cards, before After
    10 10 1 0 0
    50 50 1 0 0
    200 64 1 680 0
    500 64 1 2180 0

    Status. Shipped in 2.3.6 (f62a5e383). Canonical table: module BENCHMARK.md, "After #787 to #792". Raw data: Notes/local-docs/2026-11-13-link-scraper-tolerance/raw/linux/s3/ (local).

  4. HMarzban commented on Nov 15, 2026

    @HMarzban
    CollaboratorAuthor

    Shipped in 2.3.6 (f62a5e383). The before and after is in the Measured comment: #791 (comment)

    Closing as done. The issue stays in milestone v2.3.8 as the record of the load-tolerance work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    PerformanceQuery cost, load, and capacityenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions