Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@ COST_HOST_RADS=cost.r.secure-rads.org
COST_HOST_SECURE_RADS=cost.sr.secure-rads.org
PDM_HOST=pdm.sr.secure-rads.org
KMS_HOST=kms.sr.secure-rads.org
# rads baseline hostnames (the baseline tree uses .r. endpoints).
PDM_HOST_RADS=pdm.r.secure-rads.org
REGISTRY_HOST_RADS=registry.r.secure-rads.org
# Custom domain for the rads execution-service API. Set it before deploying the
# rads tree; its ACM certificate is looked up by hostname at deploy time.
EXECUTION_HOST_RADS=
VERIFIER_HOST=verifier.sr.secure-rads.org
VERIFIER_ATTEST_HOST=verifier-attest.sr.secure-rads.org
VERIFIER_URL=https://verifier.sr.secure-rads.org/algorithms
Expand Down
138 changes: 138 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
name: deploy

on:
workflow_dispatch:
inputs:
tree:
description: Tree to deploy
type: choice
options:
- all
- secure
- rads
default: all
units:
description: Optional space-separated units (see scripts/deploy.sh list)
type: string
default: ""
dry_run:
description: Print commands without changing AWS
type: boolean
default: false
registry_image:
description: Build and ship the secure registry-service image
type: boolean
default: true

permissions:
contents: read
id-token: write

concurrency:
group: deploy-${{ github.ref }}
cancel-in-progress: false

jobs:
deploy:
runs-on: blacksmith-4vcpu-ubuntu-2404-arm
env:
AWS_REGION: ${{ vars.AWS_REGION || 'ap-southeast-2' }}
AVAILABILITY_ZONE: ${{ vars.AVAILABILITY_ZONE }}
RADS_EXECUTOR_AMI_ID: ${{ vars.RADS_EXECUTOR_AMI_ID }}
SCE_ENCLAVE_AMI_ID: ${{ vars.SCE_ENCLAVE_AMI_ID }}
CERT_ARN_RADS: ${{ vars.CERT_ARN_RADS }}
CERT_ARN_SECURE_RADS: ${{ vars.CERT_ARN_SECURE_RADS }}
TRANSPARENCY_DOMAIN_RADS: ${{ vars.TRANSPARENCY_DOMAIN_RADS }}
TRANSPARENCY_DOMAIN_SECURE_RADS: ${{ vars.TRANSPARENCY_DOMAIN_SECURE_RADS }}
LOG_HOST_RADS: ${{ vars.LOG_HOST_RADS }}
LOG_HOST_SECURE_RADS: ${{ vars.LOG_HOST_SECURE_RADS }}
TELEMETRY_HOST_RADS: ${{ vars.TELEMETRY_HOST_RADS }}
TELEMETRY_HOST_SECURE_RADS: ${{ vars.TELEMETRY_HOST_SECURE_RADS }}
COST_HOST_RADS: ${{ vars.COST_HOST_RADS }}
COST_HOST_SECURE_RADS: ${{ vars.COST_HOST_SECURE_RADS }}
PDM_HOST: ${{ vars.PDM_HOST }}
PDM_HOST_RADS: ${{ vars.PDM_HOST_RADS }}
REGISTRY_HOST_RADS: ${{ vars.REGISTRY_HOST_RADS }}
EXECUTION_HOST_RADS: ${{ vars.EXECUTION_HOST_RADS }}
KMS_HOST: ${{ vars.KMS_HOST }}
VERIFIER_HOST: ${{ vars.VERIFIER_HOST }}
VERIFIER_ATTEST_HOST: ${{ vars.VERIFIER_ATTEST_HOST }}
VERIFIER_URL: ${{ vars.VERIFIER_URL }}
VERIFIER_JWKS_URL: ${{ vars.VERIFIER_JWKS_URL }}
VERIFIER_JWKS_PUBKEY_SHA256: ${{ vars.VERIFIER_JWKS_PUBKEY_SHA256 }}
VERIFIER_PUBKEY_SHA256: ${{ vars.VERIFIER_PUBKEY_SHA256 }}
VERIFIER_HOST_PUBKEY_SHA256: ${{ vars.VERIFIER_HOST_PUBKEY_SHA256 }}
PDM_HOST_PUBKEY_SHA256: ${{ vars.PDM_HOST_PUBKEY_SHA256 }}
KMS_HOST_PUBKEY_SHA256: ${{ vars.KMS_HOST_PUBKEY_SHA256 }}
LOG_HOST_PUBKEY_SHA256: ${{ vars.LOG_HOST_PUBKEY_SHA256 }}
TELEMETRY_HOST_PUBKEY_SHA256: ${{ vars.TELEMETRY_HOST_PUBKEY_SHA256 }}
CLOUDFLARE_ZONE: ${{ vars.CLOUDFLARE_ZONE }}
CLOUDFLARE_EMAIL: ${{ secrets.CLOUDFLARE_EMAIL }}
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
steps:
- uses: actions/checkout@v5

- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ vars.AWS_DEPLOY_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION || 'ap-southeast-2' }}

- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.x

- name: Write registry trust material
if: inputs.tree != 'rads'
env:
REGISTRY_TLS_CERT_B64: ${{ secrets.REGISTRY_TLS_CERT_B64 }}
REGISTRY_TLS_KEY_B64: ${{ secrets.REGISTRY_TLS_KEY_B64 }}
run: |
set -euo pipefail
if [ -z "$REGISTRY_TLS_CERT_B64" ] && [ -z "$REGISTRY_TLS_KEY_B64" ]; then
echo "registry TLS secrets are not set; verifier and registry-image will fail if selected" >&2
exit 0
fi
[ -n "$REGISTRY_TLS_CERT_B64" ] && [ -n "$REGISTRY_TLS_KEY_B64" ] || {
echo "REGISTRY_TLS_CERT_B64 and REGISTRY_TLS_KEY_B64 must be set together" >&2
exit 1
}
mkdir -p secure/csav/registry-service/tls
printf '%s' "$REGISTRY_TLS_CERT_B64" | base64 -d >secure/csav/registry-service/tls/client.crt
printf '%s' "$REGISTRY_TLS_KEY_B64" | base64 -d >secure/csav/registry-service/tls/client.key
chmod 600 secure/csav/registry-service/tls/client.crt secure/csav/registry-service/tls/client.key

- name: Write CSAV SSH key
if: inputs.tree != 'rads' && inputs.registry_image
env:
CSAV_SSH_KEY_B64: ${{ secrets.CSAV_SSH_KEY_B64 }}
run: |
set -euo pipefail
[ -n "$CSAV_SSH_KEY_B64" ] || { echo "CSAV_SSH_KEY_B64 is not set" >&2; exit 1; }
mkdir -p .ssh
printf '%s' "$CSAV_SSH_KEY_B64" | base64 -d >.ssh/csav-key.pem
chmod 400 .ssh/csav-key.pem

- name: Log in to GHCR
if: inputs.tree != 'rads' && inputs.registry_image
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ vars.GHCR_USER || github.repository_owner }}
password: ${{ secrets.GHCR_TOKEN }}

- name: Deploy
env:
TREE: ${{ inputs.tree }}
UNITS: ${{ inputs.units }}
DRY_RUN: ${{ inputs.dry_run }}
REGISTRY_IMAGE: ${{ inputs.registry_image }}
run: |
set -euo pipefail
args=("$TREE")
if [ -n "$UNITS" ]; then
read -r -a units <<<"$UNITS"
args+=("${units[@]}")
fi
[ "$DRY_RUN" = "true" ] && args+=(--dry-run)
[ "$REGISTRY_IMAGE" = "false" ] && args+=(--skip-registry-image)
scripts/deploy.sh deploy "${args[@]}"
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ instance. Use a dedicated research account.
full deployment in a new account and region.
- [`docs/running.md`](docs/running.md) - requirements, cost and account safety, make targets, CSV
exports.
- [`docs/deploy.md`](docs/deploy.md) - the single deploy entrypoint and the manual CI pipeline.
- [`docs/cost-service.md`](docs/cost-service.md) - how per-run AWS costs are metered.
- [`docs/e2e-rads.txt`](docs/e2e-rads.txt), [`docs/e2e-secure-rads.txt`](docs/e2e-secure-rads.txt) -
event-by-event plaintext flows for the build and execution phases.
Expand Down
133 changes: 133 additions & 0 deletions docs/deploy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# Deploying from CI

The deploy pipeline has one entrypoint, `scripts/deploy.sh`, and a manual GitHub Actions
workflow, `.github/workflows/deploy.yml`. Both trees deploy through the same script, in dependency
order, deriving inter-service values (queue URLs and ARNs, table names, bucket names, role ARNs)
from CloudFormation stack outputs.

This covers application and infrastructure deployment. It does not seed data, build sample
algorithms, or manage Cloudflare DNS.

## Units

`scripts/deploy.sh list` prints the units in dependency order. The same order is used by the
workflow.

| Tree | Units |
| --- | --- |
| `secure` | storage, verifier-service, result-consumer, log-service, telemetry-service, cost-service, pdm-service, kms-service, csav, execution-service, registry-image, transparency-portal |
| `rads` | result-consumer, log-service, telemetry-service, cost-service, pdm-service, registry-service, execution-service, transparency-portal |

A full tree deploy runs every unit. Named units deploy only what you name; dependencies are not
deployed automatically.

## Local use

The script reads the root `.env` (see `.env.example`) like the Makefiles do. `make env` populates
the discoverable values first.

```sh
scripts/deploy.sh list
scripts/deploy.sh deploy secure --dry-run
scripts/deploy.sh deploy rads rads/log-service
scripts/deploy.sh deploy secure --skip-registry-image
```

`--dry-run` prints the commands but still resolves inputs from AWS (read-only calls).

## Manual workflow

Run `deploy` from the Actions tab (`workflow_dispatch`). Inputs:

- `tree`: `all`, `secure`, or `rads`.
- `units`: optional space-separated unit names, for example `secure/log-service secure/pdm-service`.
- `dry_run`: print commands without changing AWS.
- `registry_image`: build and ship the secure registry-service image (default on).

Deploys are serialized per branch with a concurrency group.

## Authentication

The workflow uses GitHub OIDC; there are no long-lived AWS keys. Set the repository variable
`AWS_DEPLOY_ROLE_ARN` to a role that trusts
`token.actions.githubusercontent.com`. Trust policy:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::<account-id>:oidc-provider/token.actions.githubusercontent.com"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
},
"StringLike": {
"token.actions.githubusercontent.com:sub": "repo:Secure-RADS/st-dp:ref:refs/heads/*"
}
}
}
]
}
```

The role needs CloudFormation, Lambda, S3, DynamoDB, SQS, API Gateway, CloudFront, EC2, IAM, SSM,
ACM, and CloudWatch Logs permissions. The serverless services and the CSAV stack create IAM roles,
so this is close to account admin. Run it in a dedicated account and treat it as production-adjacent.

## Repository variables

Non-secret configuration. Values left blank fall back to the root `.env` defaults, and regional
certificates are looked up by hostname in ACM.

| Variable | Purpose |
| --- | --- |
| `AWS_REGION`, `AVAILABILITY_ZONE` | target region and AZ |
| `AWS_DEPLOY_ROLE_ARN` | role assumed through OIDC |
| `RADS_EXECUTOR_AMI_ID`, `SCE_ENCLAVE_AMI_ID` | arm64 AL2023 AMIs; derived from SSM when blank |
| `CERT_ARN_RADS`, `CERT_ARN_SECURE_RADS` | us-east-1 certificates for the two portals |
| `TRANSPARENCY_DOMAIN_RADS`, `TRANSPARENCY_DOMAIN_SECURE_RADS` | portal hostnames |
| `LOG_HOST_RADS`, `LOG_HOST_SECURE_RADS` | log-service hostnames |
| `TELEMETRY_HOST_RADS`, `TELEMETRY_HOST_SECURE_RADS` | telemetry-service hostnames |
| `COST_HOST_RADS`, `COST_HOST_SECURE_RADS` | cost-service hostnames |
| `PDM_HOST`, `PDM_HOST_RADS` | PDM hostnames |
| `KMS_HOST`, `VERIFIER_HOST`, `VERIFIER_ATTEST_HOST`, `VERIFIER_URL` | secure service hostnames |
| `REGISTRY_HOST_RADS`, `EXECUTION_HOST_RADS` | rads registry and execution API hostnames |
| `VERIFIER_JWKS_URL`, `VERIFIER_JWKS_PUBKEY_SHA256` | JWKS endpoint and pin used by pdm/log/kms |
| `VERIFIER_PUBKEY_SHA256`, `VERIFIER_HOST_PUBKEY_SHA256`, `PDM_HOST_PUBKEY_SHA256`, `KMS_HOST_PUBKEY_SHA256`, `LOG_HOST_PUBKEY_SHA256`, `TELEMETRY_HOST_PUBKEY_SHA256` | TLS pins baked into the registry image |
| `GHCR_USER` | account that can pull `ghcr.io/secure-rads/nsjail-docker` |
| `CLOUDFLARE_ZONE` | Cloudflare zone |

## Repository secrets

| Secret | Purpose |
| --- | --- |
| `GHCR_TOKEN` | pull the private nsjail base image |
| `REGISTRY_TLS_CERT_B64`, `REGISTRY_TLS_KEY_B64` | base64 of the existing `secure/csav/registry-service/tls/client.{crt,key}` |
| `CSAV_SSH_KEY_B64` | base64 of `.ssh/csav-key.pem`, used to ship the registry image |
| `CLOUDFLARE_EMAIL`, `CLOUDFLARE_API_TOKEN` | Cloudflare API credentials |

`REGISTRY_TLS_CERT_B64` must be the same client certificate the verifier truststore was created
from. Generating a new certificate rotates the trust material and breaks registration until the
verifier is redeployed with the new truststore.

## Registry image

The secure registry-service image builds on a Blacksmith arm64 runner, so `--platform linux/arm64`
is native and needs no QEMU. It builds `sce-unit-base` and `eif-build-base` first, pulls the private
`ghcr.io/secure-rads/nsjail-docker` base, ships the tarball over SSH, and restarts the compose
service on the CSAV host.

## Known gaps

- The TLS pins and `VERIFIER_JWKS_PUBKEY_SHA256` are captured from live endpoints after a fresh
deploy. Set them before deploying `secure/pdm-service`, `secure/log-service`, `secure/kms-service`,
or the registry image.
- The verifier JWT signing key must already exist at the SSM path the service reads. See
`docs/getting-started.md`.
- The CSAV key pair and the CSAV instance must already exist before the registry image ships.
- `EXECUTION_HOST_RADS` has no default. Set it before deploying the rads tree.
5 changes: 4 additions & 1 deletion docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,9 @@ This file is ignored by Git and must not be shared through the repository.

## 4. Deploy in Sydney

`scripts/deploy.sh` and the manual CI workflow automate this section; see
[`docs/deploy.md`](deploy.md). The steps below are the manual path.

Set the certificate ARNs before deploying. Devbox already exports `REGION=ap-southeast-2`:

```sh
Expand Down Expand Up @@ -165,7 +168,7 @@ Upload the truststore, then deploy verifier-service again with its immutable S3

```sh
TRUSTSTORE_BUCKET=$(aws cloudformation describe-stacks --region "$REGION" \
--stack-name verifier-service-dev \
--stack-name secure-rads-verifier-service-dev \
--query "Stacks[0].Outputs[?OutputKey=='TruststoreBucketName'].OutputValue" --output text)
TRUSTSTORE_VERSION=$(aws s3api put-object --bucket "$TRUSTSTORE_BUCKET" \
--key truststore.pem --body secure/csav/registry-service/tls/client.crt \
Expand Down
Loading