diff --git a/.claude/skills/build-plugin/SKILL.md b/.claude/skills/build-plugin/SKILL.md index e7723f73..2d6a6192 100644 --- a/.claude/skills/build-plugin/SKILL.md +++ b/.claude/skills/build-plugin/SKILL.md @@ -69,7 +69,7 @@ Before writing a single file, understand and explore the API. **Use `AskUserQues - **Time-range control** — does the endpoint accept a queryable time range at all (a `from`/`to`, `start`/`end`, or `period` parameter), or only return a fixed snapshot / current values? - **Data granularity** — when the endpoint _does_ accept a range, the finest interval it aggregates at: **per-event/raw**, **hourly**, **daily**, or **monthly**. Read it off the API docs (aggregation windows, `granularity`/`interval` params, the minimum queryable range). 5. **Understand pagination** — Cursor/next-token, or offset/limit? Separate concern from response transformation. -6. **Note the auth pattern** — API key in header, Bearer token, OAuth2, Basic auth, JWT Bearer (signed-JWT auth)? Determine from the docs. If the flow is custom — a token-exchange step or per-request signing no standard mode covers — plan a **pre-request script** ([metadata.md](references/metadata.md#pre-request-scripts-custom-auth-flows)). +6. **Note the auth pattern** — API key in header, Bearer token, OAuth2, Basic auth, JWT Bearer (signed-JWT auth)? Determine from the docs. If the flow is custom — a token-exchange step or per-request signing no standard mode covers — plan a **pre-request script** ([metadata.md](references/metadata.md#pre-request-scripts-custom-auth-flows)). Note the token endpoint's host: if it is on a different domain from the base URL, the script can only reach it once it is listed in `allowedScriptHosts` ([Fetching other hosts](references/metadata.md#fetching-other-hosts)). --- diff --git a/.claude/skills/build-plugin/references/data-streams.md b/.claude/skills/build-plugin/references/data-streams.md index 7f074dc3..66d56f82 100644 --- a/.claude/skills/build-plugin/references/data-streams.md +++ b/.claude/skills/build-plugin/references/data-streams.md @@ -873,7 +873,7 @@ Renaming, flattening single-level nesting, value coercion, adding constant colum ### Available globals -Scripts have access to `data`, `response` (the raw HTTP response — `.status`, `.headers`, `.body`), `context`, and **lodash** (`_`): +Scripts have access to `data`, `response` (the raw HTTP response — `.status`, `.headers`, `.body`), `context`, `fetch`, and **lodash** (`_`). `fetch` reaches only the base URL's domain unless the plugin lists more hosts in `allowedScriptHosts` — see [Fetching other hosts](metadata.md#fetching-other-hosts). ```javascript // dataStreams/scripts/myStream.js diff --git a/.claude/skills/build-plugin/references/metadata.md b/.claude/skills/build-plugin/references/metadata.md index 29917bae..362633b2 100644 --- a/.claude/skills/build-plugin/references/metadata.md +++ b/.claude/skills/build-plugin/references/metadata.md @@ -4,7 +4,7 @@ - [metadata.json template and field notes](#metadatajson) - [Auth patterns](#auth-patterns) -- [Pre-request scripts (custom auth flows)](#pre-request-scripts-custom-auth-flows) +- [Pre-request scripts (custom auth flows)](#pre-request-scripts-custom-auth-flows) — [Fetching other hosts](#fetching-other-hosts) --- @@ -222,7 +222,7 @@ In `base.config`: ### Script scope -The script body runs inside an async function — top-level `await` works. `fetch` and `crypto.subtle` are available (for auth calls and HMAC/SHA signing). +The script body runs inside an async function — top-level `await` works. `fetch` and `crypto.subtle` are available (for auth calls and HMAC/SHA signing). `fetch` reaches only the base URL's domain unless you list more hosts — see [Fetching other hosts](#fetching-other-hosts). | Variable | Mutable | Notes | | --------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -265,3 +265,30 @@ if (typeof state?.token !== "string" || (state?.expiryTime ?? 0) <= now) { } headers["Authorization"] = `Bearer ${state.token}`; ``` + +### Fetching other hosts + +A script's `fetch` can only reach hosts in the **registrable domain of `baseUrl`**. A base URL of `https://api.example.com/v2` lets a script fetch `id.example.com` or `example.com`, but not `auth.example-identity.com`. The check runs on every redirect hop, not just the first request. A blocked host fails the request with: + +> Fetching auth.example-identity.com is not allowed by this data source + +This applies to every script the data source runs: the pre-request script, and data streams' [post-request and `errorHandling` scripts](data-streams.md#post-request-scripts). + +When the API's token endpoint, or anything else a script calls, sits on a different domain, list it in `allowedScriptHosts` in `base.config`: + +```json +"showScripting": true, +"preRequestScript": "preRequest.js", +"allowedScriptHosts": ["global.api.greenlake.hpe.com"] +``` + +- **Always a list**, even for one host: `["auth.example.com"]`. A plain string fails `squaredup validate`. +- **Host names only.** No `https://`, path or query string. Matching ignores case. +- **`auth.example.com`** allows that host only. Prefer an exact host. +- **`*.example.com`** allows `example.com` and every host under it. Use a wildcard only when the vendor serves the endpoint from hosts you can't list, such as a per-region subdomain. +- **No wildcard on a public suffix.** `*.com`, `*.co.uk` and `*.github.io` are rejected when the script runs, with "A wildcard must cover one organization's domain". +- **Use a `{{ }}` template inside the list** when the host depends on what the user enters, such as their region: `["{{authHost}}"]`. + +You don't need `allowedScriptHosts` when the other host is in the base URL's domain. Redstor's token endpoint `id.redstor.com` works with its base URL `api.redstor.com`. + +> ⚠️ **In SquaredUp's cloud, no request can reach a private network address.** This covers the data source's own requests, its OAuth2 token request and a script's `fetch`. Loopback, link-local and private ranges (`127.0.0.1`, `169.254.x.x`, `10.x.x.x`, `192.168.x.x` and so on) are refused, and so is a public name that resolves or redirects to one. The error reads "DNS lookup 127.0.0.1(…) is not allowed. Because, It is private IP address." `allowedScriptHosts` does not override this. On a relay agent nothing changes, so a plugin for a self-hosted product on a private network has to run through a relay agent. Say so in the README. diff --git a/REVIEW.md b/REVIEW.md index 3d510a6e..9c33ae18 100644 --- a/REVIEW.md +++ b/REVIEW.md @@ -36,6 +36,7 @@ Versions are compared against `main`, because merging is the only point at which - Check for hardcoded secrets, API keys, or credentials, including in debug or log output from scripts. - Flag imported object names or properties built from URLs that may carry tokens or signed query strings (e.g. webhook URLs). - Flag URL `pattern` validation that would let a user-supplied base URL send credentials to an unintended host. +- `allowedScriptHosts` names hosts a script may send credentials to. Each entry should be a host the vendor's API documents, such as its own host or the identity provider that issues its tokens. Ask for an exact host in place of a wildcard unless the PR explains why the host can't be listed. Flag a template that lets the user add any host they like, unless the plugin is for a self-hosted product. - Do **not** require HTTPS-only URL patterns. Self-hosted targets are commonly reached over HTTP through a relay agent, and a `ui.json` pattern cannot depend on the auth mode. At most, suggest a note in the docs. ## Code formatting @@ -82,6 +83,7 @@ SquaredUp behaviour that reviews have repeatedly got wrong. Treat these as estab - **Tile parameters live under `dataSourceConfig`.** Parameters placed under `dataStream.config` on a tile are silently ignored. Flag them. See [oob-content.md](.claude/skills/build-plugin/references/oob-content.md#auto-scoping-a-stream-via-its-objects-filter). - **`autocomplete` fields store objects**, so interpolating one straight into a header, path or query sends `[object Object]`. See [ui.md, Overview](.claude/skills/build-plugin/references/ui.md#overview) (field value shapes). - **`preRequestScript` needs `showScripting: true`**, or the script is silently skipped. See [metadata.md, Wiring](.claude/skills/build-plugin/references/metadata.md#wiring). +- **Script `fetch` is limited to the base URL's registrable domain.** A script that fetches another domain needs that host in `base.config`'s `allowedScriptHosts`. In SquaredUp's cloud no request reaches a private network address, whatever is listed. See [metadata.md, Fetching other hosts](.claude/skills/build-plugin/references/metadata.md#fetching-other-hosts). - **Imports.** Array properties are JSON-stringified when written to the graph, so they cannot be used as correlation join keys. Metadata columns that are not listed are not displayed. The index refreshes on a schedule (hours, not seconds), so fast-changing records such as alerts or incidents belong in live data streams, not imports. ## File-specific guidelines