Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/DIVERGENCES.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@ touches a compared surface, reflected in the parity matrix.
`check` and `setup` (host preflight/remediation), `up --cache-image` (boot from a
prebuilt image, skip build + feature install), `read-configuration --cache-key`
(deterministic content hash; additive — default output is byte-identical to TS),
`build --secrets-file` (BuildKit build secrets; TS `build` has no such flag), `open`
`build --secrets-file` (BuildKit build secrets; TS `build` has no such flag),
`--ca-bundle` on `build`, `up` and `run-user-commands` (extra CAs trusted in Feature
installs, opted-in Dockerfile `RUN`s and the container, mounted and never copied), `open`
(launch VS Code attached to the workspace's dev container via a vscode-remote:// URI),
`stop`/`down` (pause, or stop+remove, a workspace's container — Compose-aware; the
upstream CLI leaves teardown to the editor), and the automatic credential bridge that
Expand Down
55 changes: 55 additions & 0 deletions docs/go-only-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ divergences, each covered by tests.
- [`up --cache-image`](#up---cache-image) — boot from a prebuilt image
- [`read-configuration --cache-key`](#read-configuration---cache-key) — deterministic cache key
- [`build --secrets-file`](#build---secrets-file) — BuildKit build secrets
- [`--ca-bundle`](#--ca-bundle) — trust extra CAs in builds and in the container
- [Automatic credential bridge](#automatic-credential-bridge) — private registries in `docker build`
- [`--override-config` deep-merge](#--override-config-deep-merge) — partial overrides
- [`config.build.cacheFrom`](#configbuildcachefrom) — honored from devcontainer.json
Expand Down Expand Up @@ -197,6 +198,54 @@ A Docker Compose configuration is built by `docker compose build`, as the refere
builds it, which takes no build secrets from the command line: there the flag is ignored
with a warning.

`up` takes `--secrets-file` too: the secrets reach the Dockerfile's and the Features'
`RUN --mount=type=secret` the same way, and the lifecycle commands as environment
variables. Neither leaves the value in a layer, the image config or a container label.

## `--ca-bundle`

Trust extra certificate authorities, such as a TLS-intercepting proxy's, in image builds
and in the running container, without putting them in the image.

```sh
devcontainer up --workspace-folder . --ca-bundle /run/proxy/ca-bundle.crt
```

The file is a full PEM bundle: the system CAs plus the extra ones, since some tools
replace the system pool with the file instead of adding to it. A file with no PEM
certificate is refused. `build`, `up` and `run-user-commands` take the flag.

| Where | What happens |
| --- | --- |
| Feature install (`build`, `up`) | Every Feature's install `RUN` mounts the bundle as the BuildKit secret `devcontainer-ca-bundle` at `/run/devcontainer/ca-bundle.crt`, and runs `install.sh` with `SSL_CERT_FILE`, `NODE_EXTRA_CA_CERTS`, `REQUESTS_CA_BUNDLE`, `CURL_CA_BUNDLE` and `GIT_SSL_CAINFO` pointing at it, for that `RUN` only. |
| Your Dockerfile (`build`, `up`) | The CLI cannot change your `RUN`s, so it passes the same secret to buildx; a `RUN` opts in by mounting it (below). |
| Container (`up`) | The bundle is bind-mounted read-only at `/run/devcontainer/ca-bundle.crt` and the five variables are set in the container's environment, over the configuration's `containerEnv`, so lifecycle commands and every shell trust it. |
| `run-user-commands` | Sets the variables for the lifecycle commands. A running container takes no new mount, so the container must have been created with `up --ca-bundle`; one without the mount is an error. |

```dockerfile
RUN --mount=type=secret,id=devcontainer-ca-bundle,target=/run/devcontainer/ca-bundle.crt \
SSL_CERT_FILE=/run/devcontainer/ca-bundle.crt CURL_CA_BUNDLE=/run/devcontainer/ca-bundle.crt \
./install-things.sh
```

The secret is mounted readable by root (mode 0400); a `RUN` as another user adds
`uid=<uid>` or `mode=0444` to the mount.

The bundle is mounted, never copied: no layer, image config or history holds it, and its
content is not part of BuildKit's cache key, so a different bundle (a rotated proxy CA)
reuses the cache and builds the same image. A build with the flag still differs from one
without it in the Feature `RUN` instructions themselves (the secret mount and the
variables are part of the instruction), so their caches are not shared.

The bind mount holds the file it was created with: replace the bundle's content in place
rather than renaming a new file over it, or recreate the container.

Builds need buildx; with the legacy builder the bundle is left out of the build with a
warning, as `--secrets-file` is. A Docker Compose configuration's images are built by
`docker compose build`, which takes no build secrets, so the bundle does not reach them
(a warning says so); `up` still mounts it into the Compose service's container with the
same variables.

## Automatic credential bridge

Not a flag — a behavior. The upstream CLI's credential chain applies only to its *own*
Expand All @@ -209,6 +258,12 @@ helpers, `GITHUB_TOKEN` — and hands them to the `docker build` subprocess via
self-contained `DOCKER_CONFIG`. If no credentials resolve, nothing changes: a build that
already authenticated via the environment behaves exactly as before.

The temporary `DOCKER_CONFIG` is a directory under `TMPDIR` (Go's `os.TempDir`), removed
when the build ends. A caller that must keep registry credentials off disk points `TMPDIR`
at a tmpfs. The credentials themselves are read from `$DOCKER_CONFIG/config.json` when
`DOCKER_CONFIG` is set, which may live outside `$HOME` and be a symlink (a tmpfs or FUSE
mount), and from `~/.docker/config.json` only when it is not.

## `--override-config` deep-merge

**Documented divergence.** The upstream CLI *replaces* the configuration wholesale with the
Expand Down
14 changes: 14 additions & 0 deletions docs/parity/cli-flags-inventory.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,11 @@ commands:
secrets-file:
type: string
description: "Path to a json file containing secret environment variables as key-value pairs."
# Go-only divergence: TS has no ca-bundle. A PEM bundle given to builds as
# the secret devcontainer-ca-bundle and bound read-only into the container.
ca-bundle:
type: string
description: "PEM bundle of CAs to trust in builds and in the container."
experimental-lockfile:
type: boolean
default: false
Expand Down Expand Up @@ -425,6 +430,11 @@ commands:
secrets-file:
type: string
description: "Path to a JSON file with build secrets, passed to BuildKit as build secrets (requires buildx)."
# Go-only divergence: TS has no ca-bundle. A PEM bundle given to Feature
# installs and Dockerfile RUNs as the build secret devcontainer-ca-bundle.
ca-bundle:
type: string
description: "PEM bundle of CAs to trust in Feature installs and Dockerfile RUNs (build secret devcontainer-ca-bundle)."
omit-syntax-directive:
type: boolean
default: false
Expand Down Expand Up @@ -516,6 +526,10 @@ commands:
type: string
secrets-file:
type: string
# Go-only divergence: TS has no ca-bundle. The bundle's variables for the
# lifecycle commands of a container up created with --ca-bundle.
ca-bundle:
type: string
log-file:
type: string
description: "Log file path."
Expand Down
13 changes: 12 additions & 1 deletion internal/cli/build.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ type buildOpts struct {
noLockfile bool
frozenLockfile bool
secretsFile string
caBundle string
// lockfileExcludeIDs is populated after merging --additional-features; it lists
// userFeatureIds supplied only via that flag, which 0.88 keeps out of the lockfile.
lockfileExcludeIDs map[string]bool
Expand Down Expand Up @@ -93,6 +94,7 @@ func newBuildCmd() *cobra.Command {
f.BoolVar(&opts.noLockfile, "no-lockfile", false, "Disable lockfile generation and verification.")
f.BoolVar(&opts.frozenLockfile, "frozen-lockfile", false, "Ensure lockfile exists and remains unchanged; fail otherwise.")
f.StringVar(&opts.secretsFile, "secrets-file", "", "Path to a JSON file with build secrets ({\"KEY\":\"VALUE\"}); each is passed to BuildKit as a build secret (requires buildx).")
f.StringVar(&opts.caBundle, "ca-bundle", "", caBundleFlagUsage)
f.Bool("omit-syntax-directive", false, "")
cmd.Flags().MarkHidden("skip-feature-auto-mapping")
cmd.Flags().MarkHidden("skip-persisting-customizations-from-features")
Expand Down Expand Up @@ -127,6 +129,10 @@ func runBuild(ctx context.Context, out Output, opts *buildOpts) error {
); err != nil {
return writeValidationError(out, err.Error())
}
var err error
if opts.caBundle, err = resolveCABundle(opts.caBundle); err != nil {
return writeValidationError(out, err.Error())
}

// --no-lockfile is mutually exclusive with every frozen/write lockfile flag
// (TS devContainersSpecCLI check for the build command).
Expand Down Expand Up @@ -397,7 +403,8 @@ func prepareDockerfileBuild(cfg *config.DevContainer) (*dockerfileBuildPrep, err
// build.subfolder-config-success, build.apply-build-options-success,
// build.output-oci-success, build.cache-to-local-success,
// build.platform-supported-dockerfile-success), also run by
// TestE2E_BuildUpExec_Dockerfile and TestE2E_SiblingDockerignore.
// TestE2E_BuildUpExec_Dockerfile, TestE2E_SiblingDockerignore and
// TestE2E_CABundle (an opted-in RUN reads the bundle).
func (r *buildRunner) buildDockerfile(ctx context.Context, cfg *config.DevContainer, loadResult *config.LoadResult, useBuildx bool) ([]string, error) {
logger, dockerClient, engine, opts := r.log, r.docker, r.engine, r.opts

Expand Down Expand Up @@ -456,6 +463,7 @@ func (r *buildRunner) buildDockerfile(ctx context.Context, cfg *config.DevContai
}
buildSecrets = buildxSecrets(secrets, useBuildx, logger)
}
caSecrets := caBundleBuildSecrets(opts.caBundle, useBuildx, logger)

buildOpts := docker.BuildOptions{
Dockerfile: dockerfilePath,
Expand All @@ -474,6 +482,7 @@ func (r *buildRunner) buildDockerfile(ctx context.Context, cfg *config.DevContai
CacheTo: opts.cacheTo,
Env: authEnv,
Secrets: buildSecrets,
FileSecrets: caSecrets,
}

result, err := dockerClient.Build(ctx, buildOpts)
Expand Down Expand Up @@ -871,6 +880,7 @@ func (r *buildRunner) buildCompose(ctx context.Context, cfg *config.DevContainer
experimentalFrozenLockfile: opts.experimentalFrozenLockfile,
noLockfile: opts.noLockfile,
lockfileExcludeIDs: opts.lockfileExcludeIDs,
caBundle: opts.caBundle,
},
substitute: hostMetadataSubstitution(loadResult.HostSub, nil),
configLabel: r.configLabel,
Expand Down Expand Up @@ -951,6 +961,7 @@ func (r *buildRunner) featureBuildOptions(cfg *config.DevContainer, hostSub *con
opts := r.opts
return &FeatureBuildOptions{
Secrets: buildSecretsFromFile(opts.secretsFile),
CABundle: opts.caBundle,
OverrideFeatureInstallOrder: cfg.OverrideFeatureInstallOrder,
NoCache: opts.noCache,
CacheFrom: opts.cacheFrom,
Expand Down
81 changes: 81 additions & 0 deletions internal/cli/ca_bundle.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
package cli

import (
"crypto/x509"
"fmt"
"maps"
"os"
"path/filepath"
"slices"
"strings"

"github.com/devcontainers/cli/internal/imagemeta"
"github.com/devcontainers/cli/internal/log"
"github.com/moby/moby/api/types/container"
dockermount "github.com/moby/moby/api/types/mount"
)

const caBundleFlagUsage = "PEM bundle of CAs to trust: a build secret `" + imagemeta.CABundleSecretID + "` for Feature installs and Dockerfile RUNs, and a read-only mount at " + imagemeta.CABundlePath + " in the container (requires buildx for builds)."

const caBundleComposeBuildWarning = "--ca-bundle does not reach a Docker Compose configuration's build: `docker compose build` takes no build secrets."

func resolveCABundle(path string) (string, error) {
if path == "" {
return "", nil
}
abs, err := filepath.Abs(path)
if err != nil {
return "", fmt.Errorf("--ca-bundle: %w", err)
}
pem, err := os.ReadFile(abs)
if err != nil {
return "", fmt.Errorf("--ca-bundle: %w", err)
}
if !x509.NewCertPool().AppendCertsFromPEM(pem) {
return "", fmt.Errorf("--ca-bundle: %s holds no PEM certificate", abs)
}
return abs, nil
}

func caBundleBuildSecrets(path string, useBuildx bool, logger log.Logger) map[string]string {
if path == "" {
return nil
}
if !useBuildx {
logger.Write("--ca-bundle requires BuildKit (buildx); the bundle is not given to the legacy builder", log.LevelWarning)
return nil
}
return map[string]string{imagemeta.CABundleSecretID: path}
}

func caBundleMount(path string) dockermount.Mount {
return dockermount.Mount{Type: dockermount.TypeBind, Source: path, Target: imagemeta.CABundlePath, ReadOnly: true}
}

func withCABundleEnv(env map[string]string, path string) map[string]string {
if path == "" {
return env
}
out := maps.Clone(env)
if out == nil {
out = map[string]string{}
}
for _, kv := range imagemeta.CABundleEnv() {
k, v, _ := strings.Cut(kv, "=")
out[k] = v
}
return out
}

func caBundleRemoteEnv(path string) []string {
if path == "" {
return nil
}
return imagemeta.CABundleEnv()
}

func containerHasCABundle(mounts []container.MountPoint) bool {
return slices.ContainsFunc(mounts, func(m container.MountPoint) bool {
return m.Destination == imagemeta.CABundlePath
})
}
Loading
Loading