[glours]

Related to: moby/moby#53485

Summary

Add the docker job command group, a client for the jobs extension of the daemon (org.mobyproject.jobs.v0, served as a gRPC service on the daemon socket). Jobs are one-shot or cron-scheduled containers managed by the daemon; every execution is a run with a durable identity that survives container removal.

• ls, inspect [--run RUN] and runs [--limit N] [--before CURSOR] read jobs and runs.
• logs [-f] JOB reads the logs of the most recent run from the container logs API with the run's container ID (the Jobs API does not proxy logs). A latest run that is still pending is waited for until its container starts. When the latest run has no readable container (removed, or failed before its container was created) it falls back to the most recent run that has one and says so on stderr, with the state and the age of both runs; when the latest run succeeded and its container was removed (a job that removes its container on success), the note says that its output is gone. With -f it keeps following across runs for schedule jobs.
• run [--reschedule], cancel, pause, unpause, rm [--runs keep|remove|remove-finished] and prune [--label LABEL] [-f] act on jobs.
• wait [--run RUN] [--condition terminal|running] JOB blocks until a run reaches the condition and exits with the run's exit code; a run stopped by a timeout exits with status 124 and one stopped by cancel with status 130, as docker compose run does for the same run, so that a script can tell them from a job that failed with exit 1.

Requirements

The daemon must serve Engine API 1.57 or newer and have the extension enabled, which is not the default: dockerd --extension-enable org.mobyproject.jobs.v0, or extensions.enable in daemon.json. The group carries the version: 1.57 annotation, so older daemons hide it and refuse it with the usual "requires API version" error. On a recent daemon where the extension is off, commands fail with a message pointing at that flag, and Unavailable (daemon starting or unreachable) is reported as retryable instead of dumping the gRPC status.

The CLI reaches the extension through github.com/moby/extensions/client, dialed lazily on first use because the command tree is built before the CLI is initialized.

Notes for review

• internal/jobsapi/v0 is a copy of extpoints/jobs/api/v0/jobs.go from moby/moby at 402d0a16f1 (it is not published as a standalone Go module). protogen/ and jobs.proto are generated from it with mobyextgen at the vendored github.com/moby/extensions version.
• github.com/moby/extensions is vendored at the pseudo-version v0.0.0-20260826001921-d37867cb107f; google.golang.org/grpc and google.golang.org/protobuf become direct dependencies.
• Vendoring lands with its first importer rather than in a separate vendor: commit, so that go mod tidy && go mod vendor leaves no diff at any commit. The series is: contract and vendoring, ls/logs, inspect/runs, the action commands, wait, then the generated markdown docs.
• The vendored github.com/moby/extensions runtime (client, clientpoint, core, serverpoint) and the generated protogen match the ones docker compose uses (d2819d88bcbe, 2026-10-20) on the wire: the service and method names and the point ID are the same, and only comments, SPDX headers and source paths differ.
• Every commit builds, passes go test -race ./cli/command/job/... and golangci-lint.

Not in this PR

• No command to create a job: jobs are created through the Jobs API.
• ls and runs have no --format, --quiet or --filter, and inspect prints the raw JSON of the API types (field names as in the Go contract).
• ls has no --filter (the API has Names, Labels, States, TriggerKinds, Paused and LatestRunStates), which would help to narrow the daemon-wide list to the jobs of a Compose project (named <project>.<job>, with the project label).
• logs does not read the runs kept for a job that was removed (the contract accepts the ID of such a job for runs and inspect --run); it would fall back to ListRuns when Inspect fails with NotFound.
• No e2e test, as it needs a daemon started with the extension enabled. The unit tests run every command against a fake client.

Release notes (optional)

A picture of a cute animal (not mandatory but encouraged)

[codecov-commenter]

Codecov Report

❌ Patch coverage is 21.25135% with 1460 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/jobsapi/v0/protogen/jobs.pb.go	0.00%	728 Missing ⚠️
internal/jobsapi/v0/protogen/wire.gen.go	0.00%	661 Missing ⚠️
cli/command/job/cmd.go	34.14%	54 Missing ⚠️
cli/command/job/client.go	52.94%	8 Missing ⚠️
cli/command/job/logs.go	95.91%	4 Missing ⚠️
cli/command/job/prune.go	91.66%	2 Missing ⚠️
cli/command/job/inspect.go	96.87%	1 Missing ⚠️
cli/command/job/runs.go	98.07%	1 Missing ⚠️
cli/command/job/wait.go	97.61%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[docker-agent]

Assessment: 🟡 NEEDS ATTENTION

This PR adds a full suite of docker job commands (cancel, inspect, list, logs, pause, prune, resume, rm, run, runs, wait) backed by a new gRPC jobs API and a vendored github.com/moby/extensions framework. The structure follows Docker CLI conventions well. Three medium-severity nil-pointer risks were found in the new code — all straightforward to fix — plus two lower-confidence edge cases in the vendored extensions client.

[docker-agent]

Assessment: 🟢 APPROVE

Lower-confidence findings (not posted inline)

• [low] cli/command/job/list.go:52 — NextFire column can display a negative duration when NextFireAtNano is in the past (unverified; low severity)

  The guard !j.Paused && j.NextFireAtNano != 0 does not check whether the timestamp is in the future. If the daemon hasn't refreshed NextFireAtNano after the last cron fire, time.Unix(0, j.NextFireAtNano).Sub(now) is negative and units.HumanDuration renders it as e.g. in -2 seconds. Adding .After(now) to the condition would prevent confusing output.

[docker-agent]

Assessment: 🟢 APPROVE

The docker job command group is well-structured, follows Docker CLI conventions, and uses a sound lazy-connection pattern. One minor defensive-coding gap is noted inline; a few additional low-severity observations are listed below for the author's awareness.

Lower-confidence findings (not posted inline — unverified low-severity)

• [low] cli/command/job/logs.go:171 — time.After timer goroutine leak in sleepOrCancel: the time.After channel is not stopped on context cancellation, leaving a 2-second timer goroutine per cancelled poll cycle. Use time.NewTimer + defer t.Stop() to fix. Benign in practice given the short interval.
• [low] cli/command/job/cmd.go:33 — gRPC connection not closed when a subcommand's RunE returns an error (cobra skips PersistentPostRunE on error). The in-code comment acknowledges this; the OS reclaims the socket on exit, so it only matters for multi-command-in-process usage.
• [low] cli/command/job/runs.go:127 — Paging hint prints job name and cursor without shell-quoting: docker job runs --before $cursor $job. A job name containing spaces produces a broken suggested command. Using %q for the job name would make the hint copy-pasteable.
• [low] cli/command/job/inspect.go:81 — v = reply.Run on the --run branch assigns a *Run pointer without a nil check; if the daemon returns nil Run with no error, json.MarshalIndent outputs null silently. A nil guard matching the reply.Job pattern would be more consistent.

[ndeloof]

Thanks for this, the command group is clean and the series is easy to follow. I read the hand-written code of cli/command/job (not the vendored moby/extensions, nor the generated protogen), and I compared the behavior with what docker compose does with the same engine API in docker/compose#14155, since Compose will be the main producer of the jobs these commands show. I did not run anything against a daemon with the extension, and the tests I did not read line by line. The nil-guard threads are settled, I am not going back to them.

Compose registers a scheduled job under <project>.<job> with the project label, with RemoveOnSuccess: true (a successful run of a scheduled job drops its container), runs a manual job through CreateAndRun and then Removes it, keeping the run records and containers, and exits 124 / 130 for a run that timed out / was cancelled. Three points come out of that.

To settle

1. docker job logs for a job whose last run succeeded (threads on logs.go). With Compose, that is every healthy scheduled job.
2. The exit status of docker job wait for a timed out or cancelled run is 1, while docker compose run exits 124 / 130 for the same run (thread on wait.go).

For a follow-up, or a word in the description
3. ls has no --filter, and the API has the filters (thread on list.go).
4. The run history of a manual job removed by Compose is kept by the engine, and logs cannot reach it (thread on logs.go).
5. github.com/moby/extensions is vendored at d37867cb107f (2026-08-26) while Compose moved to d2819d88bcbe (2026-10-20), with the contract copy taken at the same moby commit 402d0a16f1. If the contract did not move between them it is only a version to keep in step, but it would be worth checking that both clients speak the same wire.

What I liked: wrapJobsError, which turns the status codes into advice and keeps Canceled recognizable so Ctrl-C stays quiet; the lazy and memoized dial, which keeps the command tree buildable before the CLI is initialized; and run printing the ID of the run only, which composes well with wait and logs (--reschedule defaulting to false is also what Compose sends).

[ndeloof]

Two cases where this gives the wrong answer, both seen from Compose's side.

1. The run was just created. After docker job run J, the latest run is pending and has no container yet, so resolveLogsRun falls back to an older run. For a job that never ran before, the user gets no logs available for job "J" right after having started it. Compose meets the same window and waits for it: runContainerID does a Wait with the running condition on the run it just created. Would it make sense, when the latest run is pending, to wait for it (the Wait call is already in the contract) rather than to look at the history?

2. The latest run succeeded and its container was removed. Compose registers its scheduled jobs with RemoveOnSuccess: true, so for a healthy job the latest run has ContainerGone, and the fallback shows the most recent run that still has a container: an older failed run. The note on stderr says the container of the latest run is not available, but not that the run succeeded, and a user reading stdout only sees the output of a failure and may conclude that the job is failing. Could the note carry the state and the age of the run it shows, and could the case 'latest run succeeded, container removed' say that the run succeeded and that its output is gone, before showing anything else?

[glours]

Done in the wait commit, as that is where Wait lands in the client interface: when the latest run is pending, logs now waits for it to leave pending (by run ID, with the running condition, as runContainerID does), instead of looking at the history. The fallback note now gives the state and the age of both runs, and when the latest run succeeded and its container was removed it says so first ("the latest run R succeeded 2 hours ago and its container was removed, so its output is gone"), before "Showing the logs of run R2 (failed, 3 hours ago)".

[ndeloof]

A smaller point on logs and Inspect: the contract accepts the ID of a removed job whose runs were kept for ListRuns and InspectRun, and docker job runs / inspect --run pass the argument through, so they work. logs starts with Inspect, which only resolves live jobs. Compose removes a manual job after each run (the run records and the run containers stay, for a postmortem of a failed run). So the logs of that failure are one ListRuns away but out of reach of docker job logs JOB_ID. If Inspect fails with NotFound, falling back to ListRuns (no follow across runs then, there is no schedule to follow) would make logs work for them. It also needs a way to learn the ID, which Compose does not print today, so this can wait for a follow-up.

[glours]

Agreed that this can wait. Added to the "Not in this PR" list of the description: logs would fall back to ListRuns when Inspect returns NotFound, once there is a way to learn the ID of a removed job.

[ndeloof]

This is where the CLI and Compose disagree on the same run. For a run that timed out, docker job wait exits 1, and docker compose run of the same job exits 124 (and 130 for a cancelled run, following timeout(1) and the SIGINT convention), each with a message. The reasoning in the help is sound, an exit code of a stopped container is not the outcome of the job, but 1 is the code of a job that failed with exit 1, and a script cannot tell the two apart without parsing stderr, while with Compose it can.

Would you align on 124 / 130, or keep 1 and say in the description that this is a choice and why? Either is fine; the point is that the same engine run should not give two statuses depending on the client, and I would rather settle it now than after scripts depend on it.

[glours]

Aligned: wait now exits 124 for a timed out run and 130 for a cancelled one, as docker compose run does. A run whose container never started still exits 1 with the reason on stderr. Help text and docs updated.

[ndeloof]

ListRequest takes Names, Labels, States, TriggerKinds, Paused and LatestRunStates, and ls sends none of them. It is in the 'not in this PR' list, so this is only to say why it will matter: the jobs registered by Compose are named <project>.<job> and carry the Compose project label, and the registry is daemon-wide, so with several projects docker job ls becomes a long flat list with no way to narrow it to a project. --filter label=com.docker.compose.project=app, and --format for scripts, would be the first things asked for. Not for this PR, a follow-up issue would do.

[glours]

Agreed, and noted in the description with the Compose use case (jobs named <project>.<job> carrying the project label). --filter and --format fit a follow-up PR.

[glours]

Thanks for the review. On the vendored github.com/moby/extensions version: I diffed d37867cb107f (this PR) against d2819d88bcbe (Compose). The runtime packages (client, clientpoint, core, serverpoint) only differ by an SPDX header and one comment, and the generated protogen only by source path and identifier names. The service and method names and the point ID are the same, so both clients speak the same wire. The other points are answered in their threads.

[docker-agent]

Assessment: 🟢 APPROVE

[docker-agent]

Assessment: 🟢 APPROVE

All medium-severity findings overlapped with prior resolved review threads (logs.go:175, client.go:178/186) and were suppressed per the Prior Review Thread History policy. No high-severity issues found. The low-severity findings below are surfaced for awareness but do not block merging.

Lower-confidence findings (not posted inline)

• [low] cli/command/job/logs.go:222 — time.After in sleepOrCancel creates a timer goroutine that leaks until it fires when the context is cancelled. Impact is minor (at most one 2-second stale goroutine per cancelled poll cycle). Prefer t := time.NewTimer(pollInterval); defer t.Stop() to release the timer immediately on cancellation.

• [low] cli/command/job/cmd.go:31 — PersistentPostRunE (which calls jobs.Close()) is skipped by cobra when a subcommand's RunE returns an error. The gRPC connection is not explicitly closed in the error path. The comment at line 62 acknowledges this ("cobra skips PersistentPostRunE on error, and the process exits right after"), so this is by design for a short-lived CLI, but worth noting for future embedding or test harnesses.

• [low] cli/command/job/wait.go:106 — cli.StatusError{StatusCode: int(run.ExitCode.Value)} casts int64 to int. On 32-bit platforms, exit codes greater than math.MaxInt32 would silently truncate. Container exit codes are conventionally 0–255, so in practice this is benign, but a defensive range clamp or int32 cast would be safer.

• [low] cli/command/job/runs.go:127 — The paging hint uses fmt.Fprintf(..., "docker job runs --before %s %s\n", reply.NextCursor, opts.job) without quoting. If the daemon returns a cursor containing spaces or shell metacharacters, the displayed hint will be un-runnable or misleading. Consider wrapping the cursor with %q or single quotes in the format string.

• [low] vendor.mod:35 — github.com/moby/extensions is pinned to a pseudo-version (v0.0.0-20260826001921-d37867cb107f) on an unreleased commit. vendor.sum provides content integrity at fetch time, but if the commit is ever deleted or rebased before the module proxy caches it, go mod download in a fresh environment would fail. Consider requesting a tagged release from the upstream repository when available.

• [low] vendor/github.com/moby/extensions/fanout.go:72 — In Fold, when FailOpen is true and a provider's fn returns an error, the accumulator is reset to its pre-call value. This correctly drops value changes but cannot roll back in-process side effects (e.g., mutations to shared state pointed to by acc). The existing code comments acknowledge this limitation. Worth documenting that fn must be side-effect-free on acc for FailOpen to work correctly.