diff --git a/README.md b/README.md index 094ab43..8318c6a 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,15 @@ # optimole `optimole` uploads local files into an [Optimole](https://optimole.com) media library, into the folders you -choose, exports the library back as ZIP archives, and builds the CDN URLs of its images, resized or -converted. It is a single binary with no dependencies. +choose, exports the library back as ZIP archives, builds the CDN URLs of its images, resized or +converted, and deletes images. It is a single binary with no dependencies. ```sh optimole login # paste your API key once optimole upload -r ./photos --folder Campaigns # photos/2024/a.jpg -> Campaigns/2024 optimole export start # export the library, download ./optimole-export-ID optimole url KEY --width 800 --format webp # the image's CDN URL, 800 px wide, as WebP +optimole delete --folder Campaigns/2025 # delete every image of a folder (asks first) ``` ## Install @@ -70,7 +71,8 @@ the storage URL the file bytes go to. **Folder-scoped keys** (the ones starting with `optml-s`) work for uploads into the folders they were granted (and their subfolders): always pass `--folder` (or `--folder-id`). They cannot create folders -(`folder_creation_forbidden`), upload into the library root (`folder_required`), list folders, or export. +(`folder_creation_forbidden`), upload into the library root (`folder_required`), list folders, export, or +delete. The API base URL defaults to `https://dashboard.optimole.com/api`; override it with `--api-url` or `OPTIMOLE_API_URL` (for example to try another environment). Only `https` URLs are @@ -413,6 +415,66 @@ and `original_height`; a failed input gives `input`, `error` and `code` (`invali both answer `not_found`. A rejected API key stops the run. Messages never show the API keys in use (flag, environment or saved key), and a line that looks like an API key is not echoed at all. +### `optimole delete` + +Deletes images from the library for good: the CDN stops serving them, so pages that still use them show broken +images. Name the images with exactly one of: + +| Flag | Deletes | +|---|---| +| `--keys FILE` | The images whose asset keys are in `FILE`, one per line (`-` reads stdin). Blank lines and spaces around a key are ignored, and a key listed twice counts once. | +| `--folder PATH` | Every image in this folder and all its subfolders, e.g. `Campaigns/2026` (the paths `optimole folders` prints; case does not matter). For a website's folder, also that website's images that are in no folder. | +| `--folder-id N` | The same, naming the folder by its ID. | +| `--site DOMAIN` | Every image of this website of the account, in any folder. `https://www.example.com/blog` means `example.com`. An internationalized domain must be given in its `xn--` form. | + +| Flag | Meaning | +|---|---| +| `--yes` | Delete without asking. | +| `--json` | Print the result as one JSON object. | +| `--api-key`, `--api-url` | See [Authentication](#authentication). | + +```sh +optimole delete --folder Campaigns/2025 +optimole delete --site example.com +optimole delete --keys old-keys.txt +printf '%s\n' KEY1 KEY2 | optimole delete --keys - --yes # in a script: no question +``` + +Deleting needs a full-access API key; folder-scoped keys cannot delete. There are no approval codes: before +deleting, the command says what it will delete and asks you to type the folder's path (its name, for a path +longer than a terminal line holds), the website's domain, or `delete` for keys, then Enter. Any other answer, +Ctrl-C or Ctrl-D deletes nothing. `--yes` skips the question; without a terminal to ask on (in a script, or with +`--keys -`, where stdin carries the keys) the command refuses to run without `--yes`. + +Folders and websites are kept. A folder is looked up first (`GET /api/folders`), so an unknown folder, or a path +two folders share, is refused before anything is deleted; name the folder with `--folder-id` then. A path is only +matched against the whole folder list: if the list does not come back whole, the command stops. + +- **Keys** are sent 100 per request (`DELETE /api/assets` with `{"keys": [...]}`) and deleted at once. The server + checks every key of a request first: if one is not the account's, or is deleted already, it refuses the whole + request (`422`, naming the key) and deletes none of it. The run stops at the first request that fails; the + images of the requests before it stay deleted, and the command says how many keys to leave out to finish. +- **A folder or a website** (`{"folder": ID}` or `{"domain": "example.com"}`) is emptied in the background: the + server answers at once with how many images go (`202`), and they leave the library within a few minutes. + Images added afterwards are kept. +- **Retries.** A delete request is repeated only when the server cannot have received it (a `429`, or a + connection that failed before sending). A repeat after a lost answer would start a second background deletion, + or be refused for the keys it deleted. When an answer is lost (or is a `5xx`), the command says the images may + be deleted, or the deletion may have started. + +Output: + +```text +Deleted 250 images. +Deleting 1520 images in the background. They leave the library within a few minutes. +Nothing to delete: the folder "Campaigns/2025" has no images. +``` + +With `--json`, one object: `{"scope":"keys","requested":250,"deleted":250}` for keys (after a failure also +`error`, and `maybe_deleted` when the outcome of the failed request is unknown), and +`{"scope":"folder","folder_id":15,"folder":"Campaigns/2025","deleting":1520}` or +`{"scope":"site","domain":"example.com","deleting":1520}` for a folder or a website. + ### `optimole login`, `optimole logout` `login` reads the key from stdin: type or paste it at the prompt, or pipe it (`optimole login < key.txt`; @@ -429,9 +491,9 @@ Prints the version, commit and build date. | Code | Meaning | |---|---| -| 0 | Every selected file was uploaded (skipped files do not count). For `export start` and `export download`: the export completed and every selected archive is on disk (`export start --no-wait`: the export was started, or one was already running). For `url`: every input was printed. | -| 1 | At least one file failed, there was nothing to upload, or the output could not be written. For exports: the export failed or was cancelled, an archive failed, `--timeout` passed, or the account keeps the most exports it may. For `url`: an input failed (the others are still printed), or the API key was rejected. | -| 2 | Usage or configuration error (bad flags, a directory without `-r`, an unknown `--domain`, missing key, non-https URL). Nothing was uploaded or downloaded. For `url`, a missing key stops the run at the first asset key. | +| 0 | Every selected file was uploaded (skipped files do not count). For `export start` and `export download`: the export completed and every selected archive is on disk (`export start --no-wait`: the export was started, or one was already running). For `url`: every input was printed. For `delete`: the images were deleted (a folder or a website: are being deleted). | +| 1 | At least one file failed, there was nothing to upload, or the output could not be written. For exports: the export failed or was cancelled, an archive failed, `--timeout` passed, or the account keeps the most exports it may. For `url`: an input failed (the others are still printed), or the API key was rejected. For `delete`: the folder was not found, the API refused or failed, or the typed answer did not match. | +| 2 | Usage or configuration error (bad flags, a directory without `-r`, an unknown `--domain`, missing key, non-https URL, `delete` without `--yes` and without a terminal). Nothing was uploaded, downloaded or deleted. For `url`, a missing key stops the run at the first asset key. | | 130 | Interrupted with Ctrl-C. | ## Limits @@ -443,6 +505,7 @@ Prints the version, commit and build date. and each key given to `optimole url` one, so very large batches may see `429` answers, which the CLI waits out. URLs given to `optimole url` take none. - A presigned upload URL is valid for 15 minutes. +- `optimole delete --keys` sends 100 keys per request. - An account runs one export at a time and keeps at most 5. Export archives are about 1 GB each (a part can be larger when its first files are), and their download URLs are valid for a day. diff --git a/cmd/optimole/main.go b/cmd/optimole/main.go index 60c0c02..e24cbe4 100644 --- a/cmd/optimole/main.go +++ b/cmd/optimole/main.go @@ -1,5 +1,6 @@ -// Command optimole uploads local files into an Optimole media library and -// exports the library as ZIP archives. +// Command optimole uploads local files into an Optimole media library, +// exports the library as ZIP archives, builds the CDN URLs of its images, and +// deletes them. package main import ( diff --git a/internal/api/client.go b/internal/api/client.go index cb78a87..07bacb7 100644 --- a/internal/api/client.go +++ b/internal/api/client.go @@ -1,7 +1,7 @@ // Package api is a small client for the parts of the Optimole dashboard API // the CLI uses: reserving an asset, sending its bytes to the presigned storage -// URL, completing it, looking an asset up, listing folders, and running asset -// exports and downloading their archives. +// URL, completing it, looking an asset up, deleting assets, listing folders, +// and running asset exports and downloading their archives. package api import ( @@ -362,12 +362,12 @@ const ( // retryIdempotent is for calls that are safe to repeat (listing, // completion): 429, 502, 503, 504 and any network error. retryIdempotent retryMode = iota - // retryUncommitted is for asset creation, where a repeat reserves a - // second asset. Only failures that prove nothing was reserved are - // retried: 429 (refused before the action), 503 with - // upload_service_unavailable (the server deletes the reservation before - // answering), and connection errors before the request was sent (DNS, - // dial, TLS handshake). + // retryUncommitted is for calls a repeat would do twice, or answer + // differently: asset creation (a second reservation), starting an export, + // deleting assets (a deleted key is refused). Only failures that prove nothing was done are retried: 429 + // (refused before the action), 503 with upload_service_unavailable (the + // server deletes the reservation before answering), and connection + // errors before the request was sent (DNS, dial, TLS handshake). retryUncommitted ) diff --git a/internal/api/delete.go b/internal/api/delete.go new file mode 100644 index 0000000..5a39a29 --- /dev/null +++ b/internal/api/delete.go @@ -0,0 +1,78 @@ +package api + +import ( + "context" + "fmt" + "net/http" + "strings" +) + +// OpDeleteAssets names DELETE /assets in errors. +const OpDeleteAssets = "delete assets" + +// MaxDeleteKeys is how many asset keys one DELETE /assets takes. +const MaxDeleteKeys = 100 + +// DeleteAssets deletes up to MaxDeleteKeys assets, named by their keys, at +// once. The server checks every key first: one that is not the account's, or +// is deleted already, refuses the whole request with a 422 naming it, and +// nothing is deleted. +// +// It is retried like asset creation, only when the server cannot have +// received it: a repeat after a lost answer would be refused for the keys it +// deleted, and a deletion that worked would look failed. +func (c *Client) DeleteAssets(ctx context.Context, keys []string) error { + if len(keys) == 0 || len(keys) > MaxDeleteKeys { + return fmt.Errorf("%s: %d keys given; one request takes 1 to %d", OpDeleteAssets, len(keys), MaxDeleteKeys) + } + lower := make([]string, len(keys)) + for i, k := range keys { + if !assetKeyPattern.MatchString(k) { + return fmt.Errorf("%s: invalid asset key %q", OpDeleteAssets, k) + } + lower[i] = strings.ToLower(k) + } + body := struct { + Keys []string `json:"keys"` + }{lower} + return c.doJSON(ctx, OpDeleteAssets, http.MethodDelete, c.endpoint("assets"), body, nil, retryUncommitted) +} + +// DeleteFolderAssets starts deleting every asset of a folder and its +// subfolders in the background; for a website's folder, also the website's +// assets that are in no folder. The folders are kept. It returns how many +// assets the server deletes (unset when it did not say). +// +// It is retried like asset creation, only when the server cannot have +// received it: a repeat would start a second background deletion. +func (c *Client) DeleteFolderAssets(ctx context.Context, folderID int64) (FlexInt, error) { + if folderID <= 0 { + return FlexInt{}, fmt.Errorf("%s: invalid folder ID %d", OpDeleteAssets, folderID) + } + return c.deleteInBackground(ctx, struct { + Folder int64 `json:"folder"` + }{folderID}) +} + +// DeleteSiteAssets starts deleting every asset of one of the account's +// websites, in any folder, in the background, and returns how many assets +// the server deletes (unset when it did not say). It is retried like +// DeleteFolderAssets. +func (c *Client) DeleteSiteAssets(ctx context.Context, domain string) (FlexInt, error) { + if strings.TrimSpace(domain) == "" { + return FlexInt{}, fmt.Errorf("%s: no domain", OpDeleteAssets) + } + return c.deleteInBackground(ctx, struct { + Domain string `json:"domain"` + }{domain}) +} + +func (c *Client) deleteInBackground(ctx context.Context, body any) (FlexInt, error) { + var out struct { + Assets FlexInt `json:"assets"` + } + if err := c.doJSON(ctx, OpDeleteAssets, http.MethodDelete, c.endpoint("assets"), body, &out, retryUncommitted); err != nil { + return FlexInt{}, err + } + return out.Assets, nil +} diff --git a/internal/api/delete_test.go b/internal/api/delete_test.go new file mode 100644 index 0000000..441ef70 --- /dev/null +++ b/internal/api/delete_test.go @@ -0,0 +1,207 @@ +package api + +import ( + "context" + "errors" + "net/http" + "strings" + "testing" + + "github.com/Codeinwp/optimole-cli/internal/testserver" +) + +func TestDeleteAssets(t *testing.T) { + srv := testserver.New(t) + c, _ := newTestClient(t, srv.URL) + ctx := context.Background() + a := srv.AddAsset(testserver.Asset{Filename: "a.jpg"}) + b := srv.AddAsset(testserver.Asset{Filename: "b.jpg"}) + srv.AddAsset(testserver.Asset{Filename: "kept.jpg"}) + + if err := c.DeleteAssets(ctx, []string{strings.ToUpper(a), b}); err != nil { + t.Fatal(err) + } + ev := srv.Events() + if len(ev) != 1 || strings.Join(ev[0].Keys, ",") != a+","+b || ev[0].Status != http.StatusNoContent { + t.Fatalf("events = %+v", ev) + } + if got := strings.Join(srv.Deleted(), ","); got != strings.Join(sortedKeys(a, b), ",") { + t.Fatalf("deleted = %s", got) + } + + // One unknown key refuses the request, naming it. + unknown := strings.Repeat("ab", 16) + var apiErr *Error + err := c.DeleteAssets(ctx, []string{a, unknown}) + if !errors.As(err, &apiErr) || apiErr.StatusCode != http.StatusUnprocessableEntity || !strings.Contains(err.Error(), strings.ToUpper(unknown)) { + t.Fatalf("unknown key: %v", err) + } + + // Checked before anything is sent. + for _, keys := range [][]string{nil, make([]string, MaxDeleteKeys+1), {"not-a-key"}} { + for i := range keys { + if keys[i] == "" { + keys[i] = a + } + } + if err := c.DeleteAssets(ctx, keys); err == nil { + t.Errorf("%d keys: no error", len(keys)) + } + } + if n := srv.Count(testserver.RouteDeleteAssets); n != 2 { + t.Fatalf("requests = %d, want 2", n) + } +} + +func sortedKeys(keys ...string) []string { + if keys[0] > keys[1] { + keys[0], keys[1] = keys[1], keys[0] + } + return keys +} + +// TestDeleteAssetsIsNotRepeatedOnceItMayHaveDeleted: a deleted key is +// refused when named again, so a repeat after a lost answer would make a +// deletion that worked look failed. Only a 429 is repeated. +func TestDeleteAssetsIsNotRepeatedOnceItMayHaveDeleted(t *testing.T) { + srv := testserver.New(t) + c, sl := newTestClient(t, srv.URL) + ctx := context.Background() + a := srv.AddAsset(testserver.Asset{Filename: "a.jpg"}) + srv.Inject(testserver.RouteDeleteAssets, testserver.Response{CommitThenClose: true}) + var netErr *NetworkError + if err := c.DeleteAssets(ctx, []string{a}); !errors.As(err, &netErr) || !netErr.MaybeCommitted { + t.Fatalf("lost answer: %v", err) + } + if n := srv.Count(testserver.RouteDeleteAssets); n != 1 || len(srv.Deleted()) != 1 { + t.Fatalf("requests = %d, deleted = %v", n, srv.Deleted()) + } + var apiErr *Error + if err := c.DeleteAssets(ctx, []string{a}); !errors.As(err, &apiErr) || apiErr.StatusCode != http.StatusUnprocessableEntity || !strings.Contains(err.Error(), strings.ToUpper(a)) { + t.Fatalf("a deleted key named again: %v", err) + } + + b := srv.AddAsset(testserver.Asset{Filename: "b.jpg"}) + srv.Inject(testserver.RouteDeleteAssets, testserver.Response{Status: 429, Body: `{"message":"Too Many Attempts."}`}) + if err := c.DeleteAssets(ctx, []string{b}); err != nil { + t.Fatal(err) + } + if n := srv.Count(testserver.RouteDeleteAssets); n != 4 || len(sl.all()) != 1 { + t.Fatalf("requests = %d, sleeps = %d", n, len(sl.all())) + } +} + +func TestDeleteFolderAndSiteAssets(t *testing.T) { + srv := testserver.New(t) + c, _ := newTestClient(t, srv.URL) + ctx := context.Background() + campaigns := srv.AddFolder("Campaigns") + spring := srv.AddFolder("Campaigns/2026/Spring") + site := srv.AddSite("example.com") + srv.AddAsset(testserver.Asset{Filename: "a.jpg", FolderID: campaigns}) + srv.AddAsset(testserver.Asset{Filename: "b.jpg", FolderID: spring}) + srv.AddAsset(testserver.Asset{Filename: "root.jpg"}) + srv.AddAsset(testserver.Asset{Filename: "offloaded.jpg", Domain: "example.com"}) + srv.AddAsset(testserver.Asset{Filename: "filed.jpg", Domain: "example.com", FolderID: campaigns}) + + n, err := c.DeleteFolderAssets(ctx, spring) + if err != nil || !n.Valid || n.Value != 1 { + t.Fatalf("spring: %+v, %v", n, err) + } + // A website's folder takes the website's images that are in no folder. + if n, err = c.DeleteFolderAssets(ctx, site); err != nil || n.Value != 1 { + t.Fatalf("site folder: %+v, %v", n, err) + } + if n, err = c.DeleteSiteAssets(ctx, "https://WWW.Example.com/blog"); err != nil || n.Value != 1 { + t.Fatalf("site: %+v, %v", n, err) + } + if n, err = c.DeleteFolderAssets(ctx, campaigns); err != nil || n.Value != 1 { + t.Fatalf("campaigns: %+v, %v", n, err) + } + ev := srv.Events() + if ev[0].FolderID != spring || ev[2].Domain != "https://WWW.Example.com/blog" || ev[0].Status != http.StatusAccepted { + t.Fatalf("events = %+v", ev) + } + if n := len(srv.Deleted()); n != 4 { + t.Fatalf("deleted %d, want 4 (root.jpg stays)", n) + } + + var apiErr *Error + if _, err := c.DeleteSiteAssets(ctx, "other.com"); !errors.As(err, &apiErr) || !apiErr.HasField("domain") { + t.Fatalf("unknown site: %v", err) + } + if _, err := c.DeleteFolderAssets(ctx, 999); !errors.As(err, &apiErr) || !apiErr.HasField("folder") { + t.Fatalf("unknown folder: %v", err) + } + if _, err := c.DeleteFolderAssets(ctx, 0); err == nil { + t.Fatal("folder 0: no error") + } + if _, err := c.DeleteSiteAssets(ctx, " "); err == nil { + t.Fatal("blank domain: no error") + } +} + +// TestDeleteFolderAssetsIsNotRepeatedOnceItMayHaveStarted: a repeat would +// start a second background deletion, so only a refusal before the action +// (429) is retried. +func TestDeleteFolderAssetsIsNotRepeatedOnceItMayHaveStarted(t *testing.T) { + for name, resp := range map[string]testserver.Response{ + "lost answer": {CommitThenClose: true}, + "502": {Status: 502, Body: `{"message":"Bad Gateway"}`}, + } { + srv := testserver.New(t) + folder := srv.AddFolder("A") + srv.Inject(testserver.RouteDeleteAssets, resp) + c, _ := newTestClient(t, srv.URL) + _, err := c.DeleteFolderAssets(context.Background(), folder) + if err == nil { + t.Fatalf("%s: no error", name) + } + var netErr *NetworkError + if resp.CommitThenClose && (!errors.As(err, &netErr) || !netErr.MaybeCommitted) { + t.Fatalf("%s: %v", name, err) + } + if n := srv.Count(testserver.RouteDeleteAssets); n != 1 { + t.Fatalf("%s: requests = %d, want 1", name, n) + } + } + + srv := testserver.New(t) + srv.AddSite("example.com") + srv.Inject(testserver.RouteDeleteAssets, testserver.Response{Status: 429, Body: `{"message":"Too Many Attempts."}`}) + c, _ := newTestClient(t, srv.URL) + if _, err := c.DeleteSiteAssets(context.Background(), "example.com"); err != nil { + t.Fatal(err) + } + if n := srv.Count(testserver.RouteDeleteAssets); n != 2 { + t.Fatalf("requests after a 429 = %d, want 2", n) + } +} + +func TestDeleteInBackgroundWithoutACount(t *testing.T) { + srv := testserver.New(t) + srv.Inject(testserver.RouteDeleteAssets, testserver.Response{Status: 202, Body: `{}`}) + c, _ := newTestClient(t, srv.URL) + n, err := c.DeleteSiteAssets(context.Background(), "example.com") + if err != nil || n.Valid { + t.Fatalf("%+v, %v", n, err) + } +} + +func TestDeleteNeedsAFullAccessKey(t *testing.T) { + srv := testserver.New(t) + const scoped = "optml-sScopedKeyForDeletes0123" + srv.AddKey(scoped, testserver.Key{Scoped: true}) + a := srv.AddAsset(testserver.Asset{Filename: "a.jpg"}) + c, err := New(Options{BaseURL: srv.URL, APIKey: scoped, UserAgent: "optimole-cli/test"}) + if err != nil { + t.Fatal(err) + } + var apiErr *Error + if err := c.DeleteAssets(context.Background(), []string{a}); !errors.As(err, &apiErr) || apiErr.StatusCode != http.StatusForbidden { + t.Fatalf("got %v", err) + } + if len(srv.Deleted()) != 0 { + t.Fatal("a scoped key deleted") + } +} diff --git a/internal/cli/cli.go b/internal/cli/cli.go index 0998848..365a444 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -16,6 +16,7 @@ import ( "github.com/Codeinwp/optimole-cli/internal/api" "github.com/Codeinwp/optimole-cli/internal/config" "github.com/Codeinwp/optimole-cli/internal/redact" + "github.com/Codeinwp/optimole-cli/internal/termecho" ) // Exit codes. @@ -48,6 +49,18 @@ type Env struct { Transport http.RoundTripper // Sleep, when set, replaces the wait between retries. Sleep func(ctx context.Context, d time.Duration) error + // StdinIsTerminal, when set, replaces the check that stdin is an + // interactive terminal (tests use it to answer the delete prompt). + StdinIsTerminal func() bool +} + +// interactive reports whether stdin is an interactive terminal. +func (e Env) interactive() bool { + if e.StdinIsTerminal != nil { + return e.StdinIsTerminal() + } + f, ok := e.Stdin.(*os.File) + return ok && termecho.IsTerminal(f) } // DefaultEnv is the real process environment. @@ -62,8 +75,8 @@ func DefaultEnv(build BuildInfo) Env { } } -const mainUsage = `optimole uploads local files into an Optimole media library, exports it, and -builds the CDN URLs of its images. +const mainUsage = `optimole uploads local files into an Optimole media library, exports it, +builds the CDN URLs of its images, and deletes them. Usage: optimole [flags] [arguments] @@ -73,6 +86,7 @@ Commands: folders List the library's folders export Export the library as ZIP archives and download them url Print an asset's CDN URL, resized or converted + delete Delete images by key, or every image of a folder or a website login Save an API key (read from stdin) logout Delete the saved API key version Print the version @@ -96,6 +110,8 @@ func Run(ctx context.Context, args []string, env Env) int { return runExport(ctx, rest, env) case "url": return runURL(ctx, rest, env) + case "delete": + return runDelete(ctx, rest, env) case "login": return runLogin(ctx, rest, env) case "logout": diff --git a/internal/cli/cli_test.go b/internal/cli/cli_test.go index ddf1f65..f97b823 100644 --- a/internal/cli/cli_test.go +++ b/internal/cli/cli_test.go @@ -39,6 +39,8 @@ type harness struct { cfgDir string vars map[string]string stdin io.Reader + // tty makes stdin count as an interactive terminal. + tty bool stdout bytes.Buffer stderr bytes.Buffer } @@ -67,10 +69,11 @@ func (h *harness) env() Env { defer mu.Unlock() return h.vars[k] }, - UserConfigDir: func() (string, error) { return h.cfgDir, nil }, - Build: BuildInfo{Version: "1.2.3", Commit: "abc1234", Date: "2026-10-26"}, - Transport: localOnly{h.t}, - Sleep: func(ctx context.Context, _ time.Duration) error { return ctx.Err() }, + UserConfigDir: func() (string, error) { return h.cfgDir, nil }, + Build: BuildInfo{Version: "1.2.3", Commit: "abc1234", Date: "2026-10-26"}, + Transport: localOnly{h.t}, + Sleep: func(ctx context.Context, _ time.Duration) error { return ctx.Err() }, + StdinIsTerminal: func() bool { return h.tty }, } } diff --git a/internal/cli/delete.go b/internal/cli/delete.go new file mode 100644 index 0000000..1107937 --- /dev/null +++ b/internal/cli/delete.go @@ -0,0 +1,566 @@ +package cli + +import ( + "context" + "encoding/json" + "errors" + "flag" + "fmt" + "io" + "net/http" + "net/url" + "os" + "path" + "sort" + "strconv" + "strings" + "unicode" + "unicode/utf8" + + "github.com/Codeinwp/optimole-cli/internal/api" + "github.com/Codeinwp/optimole-cli/internal/output" + "github.com/Codeinwp/optimole-cli/internal/redact" + "github.com/Codeinwp/optimole-cli/internal/upload" +) + +const deleteUsage = `Usage: + optimole delete --keys FILE|- [flags] + optimole delete --folder PATH|--folder-id N [flags] + optimole delete --site DOMAIN [flags] + +Deletes images from the media library for good: the CDN stops serving them, +so pages that still use them show broken images. Name the images with exactly +one of: + + --keys FILE|- The asset keys in FILE (- reads stdin), one per line; blank + lines are ignored. They go 100 per request; a request that + fails stops the run, and the images before it stay deleted. + --folder PATH Every image in this folder and its subfolders, e.g. + "Campaigns/2026" (see "optimole folders"). For a website's + folder, also the website's images that are in no folder. + --folder-id N The same, naming the folder by its ID. + --site DOMAIN Every image of this website of the account, in any folder + (an internationalized domain in its xn-- form). + +Folders and websites are kept. A folder or a website is emptied in the +background: the command returns at once with how many images go, and they +leave the library within a few minutes. + +First the command says what it will delete and asks you to type the folder's +path (its name, for a path too long to type), the website's domain, or +"delete" for keys, then Enter. --yes skips the question. Without a terminal to +ask on (in scripts, or with --keys -), it deletes only with --yes. Deleting +needs a full-access API key. + +Flags: + --yes Delete without asking + --json Print the result as one JSON object +` + commonUsage + ` +Exit codes: 0 deleted (a folder or a website: being deleted), 1 the folder +was not found, the API refused or failed, or the typed answer did not match, +2 usage or configuration error (such as no --yes without a terminal), 130 +interrupted. +` + +// maxAnswer is the longest answer the question asks to type: a terminal line +// holds 1024 bytes on macOS. A longer folder path is confirmed by its name. +const maxAnswer = 1000 + +// deleteTarget is what "optimole delete" deletes: keys, a folder or a website. +type deleteTarget struct { + keys []string // lower case, each once + keysFrom string // the --keys file, "-" for stdin + folderID int64 + folder string // the folder's path + website bool // the folder is a website's + // site is the --site value, then its host in lower case: what is sent. + // The server strips one "www." from it, into domain, as the prompt does. + // Sending domain would have the server strip a second "www.", and sending + // the whole URL could pass its 255-character limit. + site string + domain string +} + +// keysLine is the JSON of a deletion by keys. +type keysLine struct { + Scope string `json:"scope"` + Requested int `json:"requested"` + Deleted int `json:"deleted"` + // MaybeDeleted counts the keys of a failed request whose outcome is + // unknown: they may be deleted too. + MaybeDeleted int `json:"maybe_deleted,omitempty"` + Error string `json:"error,omitempty"` +} + +// backgroundLine is the JSON of a folder or a website being emptied. +type backgroundLine struct { + Scope string `json:"scope"` + FolderID int64 `json:"folder_id,omitempty"` + Folder string `json:"folder,omitempty"` + Domain string `json:"domain,omitempty"` + // Deleting is how many images go; absent when the server did not say. + Deleting *int64 `json:"deleting,omitempty"` +} + +func runDelete(ctx context.Context, args []string, env Env) int { + const name = "delete" + fs := newFlagSet(name, deleteUsage, env) + var common commonFlags + common.register(fs, true) + fs.SetOutput(flagErrorWriter{w: env.Stderr}) + var t deleteTarget + var folderPath, folderIDRaw string + var yes, jsonOut bool + fs.StringVar(&t.keysFrom, "keys", "", "") + fs.StringVar(&folderPath, "folder", "", "") + fs.StringVar(&folderIDRaw, "folder-id", "", "") + fs.StringVar(&t.site, "site", "", "") + fs.BoolVar(&yes, "yes", false, "") + fs.BoolVar(&jsonOut, "json", false, "") + rest, err := parseArgs(fs, args) + if err != nil { + return flagExit(err) + } + if len(rest) > 0 { + return usageError(env, name, "takes no arguments; name the images with --keys, --folder, --folder-id or --site") + } + set := map[string]bool{} + fs.Visit(func(f *flag.Flag) { set[f.Name] = true }) + named := 0 + for _, f := range []string{"keys", "folder", "folder-id", "site"} { + if set[f] { + named++ + } + } + if named != 1 { + return usageError(env, name, "name the images to delete with exactly one of --keys, --folder, --folder-id and --site") + } + // Flag values are echoed in usage errors: a key given by mistake must + // not be. + red := common.resolve(env).redact + switch { + case set["keys"]: + if strings.TrimSpace(t.keysFrom) == "" { + return usageError(env, name, "--keys needs a file, or - to read stdin") + } + if t.keysFrom == "-" && env.interactive() { + return usageError(env, name, "--keys - reads the keys from stdin: pipe them in, or name a file") + } + case set["folder"]: + if folderPath, err = upload.NormalizeFolderPath(folderPath); err != nil { + return usageError(env, name, "--folder: %s", shown(red, err.Error())) + } + if folderPath == "" { + return usageError(env, name, "--folder must name a folder; the library root is not one") + } + case set["folder-id"]: + if t.folderID, err = parseFolderID(folderIDRaw); err != nil { + return usageError(env, name, "--folder-id: %s", shown(red, err.Error())) + } + default: + var host string + if host, err = siteHost(t.site); err != nil { + return usageError(env, name, "--site: %s %v", strconv.Quote(red.String(t.site)), err) + } + t.site, t.domain = host, strings.TrimPrefix(host, "www.") + } + if !yes && !env.interactive() { + return usageError(env, name, "stdin is not a terminal, so the deletion cannot be confirmed; add --yes to delete without asking") + } + if set["keys"] { + if code := readDeleteKeys(ctx, env, red, &t); code != ExitOK { + return code + } + } + + sess, err := common.connect(env) + if err != nil { + fmt.Fprintf(env.Stderr, "optimole %s: %v\n", name, err) + return ExitUsage + } + if set["folder"] || set["folder-id"] { + if code := findFolder(ctx, env, sess, &t, folderPath); code != ExitOK { + return code + } + } + if !yes { + if code := confirmDelete(ctx, env, sess.redact, &t); code != ExitOK { + return code + } + } + out := output.NewErrWriter(env.Stdout) + enc := json.NewEncoder(out) + enc.SetEscapeHTML(false) + var code int + if t.keys != nil { + code = deleteKeys(ctx, env, sess, out, enc, &t, jsonOut) + } else { + code = deleteInBackground(ctx, env, sess, out, enc, &t, jsonOut) + } + if err := out.Err(); err != nil && code == ExitOK { + return outputFailed(env, err) + } + return code +} + +// readDeleteKeys reads the --keys input: one asset key per line, blank lines +// and surrounding spaces ignored. The keys are kept in lower case, each once. +// red hides the known API keys in errors, which name the file. +func readDeleteKeys(ctx context.Context, env Env, red redact.Redactor, t *deleteTarget) int { + const name = "delete" + interrupted := func() int { + fmt.Fprintf(env.Stderr, "optimole %s: interrupted; nothing was deleted\n", name) + return ExitInterrupted + } + var r io.Reader = env.Stdin + src := "stdin" + if t.keysFrom != "-" { + f, err := openContext(ctx, t.keysFrom) + if ctx.Err() != nil { + return interrupted() + } + if err != nil { + return usageError(env, name, "--keys: %s", shown(red, err.Error())) + } + defer f.Close() + r, src = f, shown(red, t.keysFrom) + } + readCtx, cancel := context.WithCancel(ctx) + defer cancel() + lines, readErr := readLines(readCtx, r, src) + seen := map[string]bool{} + for n := 1; ; n++ { + // A read cannot be interrupted: Ctrl-C leaves it behind. + var line string + var more bool + select { + case line, more = <-lines: + case <-ctx.Done(): + } + if !more { + break + } + k := strings.TrimSpace(line) + if k == "" { + continue + } + // The line is not echoed: it could be anything, an API key too. + if !api.IsAssetKey(k) { + return usageError(env, name, "--keys: line %d of %s is not an asset key (32 hexadecimal characters)", n, src) + } + k = strings.ToLower(k) + if !seen[k] { + seen[k] = true + t.keys = append(t.keys, k) + } + } + if ctx.Err() != nil { + return interrupted() + } + if err := readErr(); err != nil { + return usageError(env, name, "--keys: %s", shown(red, err.Error())) + } + if len(t.keys) == 0 { + return usageError(env, name, "--keys: no asset keys in %s", src) + } + return ExitOK +} + +// openContext opens a file for reading and gives up when ctx is done: opening +// a FIFO waits for a writer. The open left behind ends with the process. +func openContext(ctx context.Context, name string) (*os.File, error) { + type opened struct { + f *os.File + err error + } + ch := make(chan opened, 1) + go func() { + f, err := os.Open(name) + ch <- opened{f, err} + }() + select { + case o := <-ch: + return o.f, o.err + case <-ctx.Done(): + return nil, ctx.Err() + } +} + +// findFolder looks the folder up, by ID or by path, in the folder list: for +// its path, shown and typed to confirm, and to refuse a folder that is not +// there, or a path two folders share, before anything is deleted. A path is +// matched only against a whole list, so a duplicate cannot hide in the part +// that did not come back. +func findFolder(ctx context.Context, env Env, sess *session, t *deleteTarget, want string) int { + folders, complete, err := listAllFolders(ctx, sess.client, "") + if err != nil { + return deleteError(env, sess, err) + } + paths := folderPaths(folders) + found := map[int64]api.Folder{} + for _, f := range folders { + if (t.folderID > 0 && f.ID == t.folderID) || (want != "" && samePath(paths[f.ID], want)) { + found[f.ID] = f + } + } + what := fmt.Sprintf("the ID %d", t.folderID) + if want != "" { + what = "the path " + strconv.Quote(sess.redact.String(want)) + } + switch { + case !complete && (want != "" || len(found) == 0): + fmt.Fprintf(env.Stderr, "optimole delete: the folder list came back incomplete (%d folders), so the folder with %s cannot be told apart for sure; nothing was deleted\n", len(folders), what) + return ExitFailure + case len(found) == 0: + fmt.Fprintf(env.Stderr, "optimole delete: no folder has %s (\"optimole folders\" lists them)\n", what) + return ExitFailure + case len(found) > 1: + ids := make([]string, 0, len(found)) + for id := range found { + ids = append(ids, strconv.FormatInt(id, 10)) + } + sort.Strings(ids) + fmt.Fprintf(env.Stderr, "optimole delete: %d folders have %s (IDs %s); name one with --folder-id\n", len(found), what, strings.Join(ids, ", ")) + return ExitFailure + } + for id, f := range found { + t.folderID, t.folder, t.website = id, paths[id], f.IsDefault + } + return ExitOK +} + +// confirmDelete says what is about to go and asks for the folder's path, the +// website's domain or "delete" to be typed, then Enter; any other answer, or +// none, aborts. red hides the known API keys in what is shown: an answer that +// holds one cannot be typed from the prompt, which leaves --yes. +func confirmDelete(ctx context.Context, env Env, red redact.Redactor, t *deleteTarget) int { + var what, answer string + switch { + case t.keys != nil: + what = fmt.Sprintf("%s (the keys in %s)", images(int64(len(t.keys))), shown(red, t.keysFrom)) + answer = "delete" + case t.domain != "": + what = fmt.Sprintf("every image of the website %s, in any folder", shown(red, t.domain)) + answer = t.domain + default: + what = fmt.Sprintf("every image in the folder %q (ID %d) and its subfolders", red.String(t.folder), t.folderID) + if t.website { + what += fmt.Sprintf(", and the images of the website %s that are in no folder", shown(red, t.folder)) + } + what += "; the folders are kept" + answer = t.folder + if len(answer) > maxAnswer { + answer = path.Base(t.folder) + } + } + fmt.Fprintf(env.Stderr, "About to delete %s. This cannot be undone: the CDN stops serving the images.\nType %q to confirm: ", what, red.String(answer)) + line, err := readOneLine(ctx, env.Stdin, false, "the answer") + complete := strings.HasSuffix(line, "\n") + if !complete { + // Ctrl-C or Ctrl-D left the cursor after the prompt. + fmt.Fprintln(env.Stderr) + } + switch { + case errors.Is(err, context.Canceled): + fmt.Fprintln(env.Stderr, "optimole delete: interrupted; nothing was deleted") + return ExitInterrupted + case err != nil: + fmt.Fprintf(env.Stderr, "optimole delete: %s; nothing was deleted\n", shown(red, err.Error())) + return ExitFailure + case !complete || strings.TrimSpace(line) != answer: + // An answer that ends without Enter (Ctrl-D, or a line too long to + // be read whole) is no answer. + fmt.Fprintln(env.Stderr, "optimole delete: the answer did not match; nothing was deleted") + return ExitFailure + } + return ExitOK +} + +// deleteKeys deletes t.keys, api.MaxDeleteKeys per request, and stops at the +// first request that fails. +func deleteKeys(ctx context.Context, env Env, sess *session, out io.Writer, enc *json.Encoder, t *deleteTarget, jsonOut bool) int { + total, deleted, maybe := len(t.keys), 0, 0 + var err error + for deleted < total { + if err = ctx.Err(); err != nil { + // Stopped between requests: the next one was never sent. + break + } + end := min(deleted+api.MaxDeleteKeys, total) + if err = sess.client.DeleteAssets(ctx, t.keys[deleted:end]); err != nil { + if !untouched(err) { + maybe = end - deleted + } + break + } + deleted = end + } + if jsonOut { + enc.Encode(keysLine{Scope: "keys", Requested: total, Deleted: deleted, MaybeDeleted: maybe, Error: sess.redact.Error(err)}) + } + if err == nil { + if !jsonOut { + fmt.Fprintf(out, "Deleted %s.\n", images(int64(deleted))) + } + return ExitOK + } + code := deleteError(env, sess, err) + other := total - deleted - maybe + switch { + case maybe > 0: + msg := fmt.Sprintf("Deleted %d of %s before it stopped; %d more may be deleted too (the request failed)", deleted, images(int64(total)), maybe) + if deleted == 0 { + msg = fmt.Sprintf("%d of %s may be deleted (the request failed)", maybe, images(int64(total))) + } + if other > 0 { + msg += fmt.Sprintf(", the other %d were not", other) + } + fmt.Fprintln(env.Stderr, msg+".") + case deleted > 0: + fmt.Fprintf(env.Stderr, "Deleted %d of %s before it stopped; the other %d were not deleted.\n", deleted, images(int64(total)), other) + default: + fmt.Fprintln(env.Stderr, "Nothing was deleted.") + } + // A deleted key counts as invalid if named again, and refuses its whole + // request: say which to leave out. + if deleted > 0 { + fmt.Fprintf(env.Stderr, "To finish, run the command again without the first %d keys: a deleted key counts as invalid if named again.\n", deleted) + } + if maybe > 0 { + fmt.Fprintln(env.Stderr, "If the next run lists keys as invalid, the request that failed deleted them; leave them out.") + } + return code +} + +// deleteInBackground asks the server to empty the folder or the website. +func deleteInBackground(ctx context.Context, env Env, sess *session, out io.Writer, enc *json.Encoder, t *deleteTarget, jsonOut bool) int { + var n api.FlexInt + var err error + var line backgroundLine + var where string + if t.domain != "" { + n, err = sess.client.DeleteSiteAssets(ctx, t.site) + line = backgroundLine{Scope: "site", Domain: sess.redact.String(t.domain)} + where = "the website " + shown(sess.redact, t.domain) + } else { + n, err = sess.client.DeleteFolderAssets(ctx, t.folderID) + line = backgroundLine{Scope: "folder", FolderID: t.folderID, Folder: sess.redact.String(t.folder)} + where = "the folder " + strconv.Quote(sess.redact.String(t.folder)) + } + if err != nil { + code := deleteError(env, sess, err) + if !untouched(err) { + fmt.Fprintln(env.Stderr, "The deletion may have started anyway: check the library in a few minutes before running the command again.") + } + return code + } + if jsonOut { + if n.Valid { + line.Deleting = &n.Value + } + enc.Encode(line) + return ExitOK + } + switch { + case !n.Valid: + fmt.Fprintln(out, "Deleting the images in the background. They leave the library within a few minutes.") + case n.Value == 0: + fmt.Fprintf(out, "Nothing to delete: %s has no images.\n", where) + case n.Value == 1: + fmt.Fprintln(out, "Deleting 1 image in the background. It leaves the library within a few minutes.") + default: + fmt.Fprintf(out, "Deleting %d images in the background. They leave the library within a few minutes.\n", n.Value) + } + return ExitOK +} + +// untouched reports a failed delete request that proves nothing was deleted: +// a 4xx (the dashboard checks the key, the scope and the body before it +// deletes anything), or no connection at all. Delete requests are repeated +// only after such failures (see api.DeleteAssets), so earlier attempts did +// nothing either. After a lost answer, a 5xx, a 2xx that does not read or +// Ctrl-C mid-request, the request may have deleted. +func untouched(err error) bool { + var apiErr *api.Error + var netErr *api.NetworkError + switch { + case errors.As(err, &apiErr): + return apiErr.StatusCode >= 400 && apiErr.StatusCode < 500 + case errors.As(err, &netErr): + return !netErr.MaybeCommitted + } + return false +} + +// deleteError prints a failed delete call and returns its exit code. +func deleteError(env Env, sess *session, err error) int { + msg := shown(sess.redact, err.Error()) + var apiErr *api.Error + switch { + case errors.Is(err, context.Canceled): + fmt.Fprintln(env.Stderr, "optimole delete: interrupted") + return ExitInterrupted + case isRejectedKey(err): + fmt.Fprintf(env.Stderr, "optimole delete: %s\nThe API key (from %s) was rejected.\n", msg, sess.settings.KeySource) + case errors.As(err, &apiErr) && apiErr.StatusCode == http.StatusForbidden && isScopedKey(sess.settings.APIKey): + fmt.Fprintf(env.Stderr, "optimole delete: %s\nDeleting images needs a full-access API key; folder-scoped keys cannot delete.\n", msg) + default: + fmt.Fprintf(env.Stderr, "optimole delete: %s\n", msg) + } + return ExitFailure +} + +// siteHost reads the host of a URL or of a bare domain, in lower case, the +// way the dashboard does before it strips one "www.". Only ASCII hosts are +// taken: the dashboard maps the others with IDNA, which the standard library +// lacks, so the domain shown could differ from the one deleted. +func siteHost(s string) (string, error) { + s = strings.TrimSpace(s) + if !strings.Contains(s, "://") { + s = "https://" + s + } else if !hasHTTPScheme(s) { + return "", errors.New("is not a domain") + } + u, err := url.Parse(s) + if err != nil { + return "", errors.New("is not a domain") + } + host := u.Hostname() + if strings.ContainsFunc(host, func(r rune) bool { return r >= utf8.RuneSelf }) { + return "", errors.New("is not in ASCII; give the domain in its xn-- form") + } + host = strings.ToLower(host) + if strings.TrimPrefix(host, "www.") == "" { + return "", errors.New("is not a domain") + } + return host, nil +} + +// samePath reports whether a listed folder path reads as want, a normalized +// path: a folder whose name holds a slash ("2026 / Drafts") reads the same as +// the nested folders it looks like, and counts as a match, so the two are +// told apart by ID. +func samePath(listed, want string) bool { + if n, err := upload.NormalizeFolderPath(listed); err == nil { + listed = n + } + return strings.EqualFold(listed, want) +} + +// images says "1 image" or "N images". +func images(n int64) string { + if n == 1 { + return "1 image" + } + return fmt.Sprintf("%d images", n) +} + +// shown readies text for the terminal: the known API keys hidden, and quoted +// when it holds control characters, which a terminal would otherwise act on. +func shown(red redact.Redactor, s string) string { + s = red.String(s) + if strings.ContainsFunc(s, unicode.IsControl) { + return strconv.Quote(s) + } + return s +} diff --git a/internal/cli/delete_fifo_test.go b/internal/cli/delete_fifo_test.go new file mode 100644 index 0000000..5ec9c3a --- /dev/null +++ b/internal/cli/delete_fifo_test.go @@ -0,0 +1,41 @@ +//go:build darwin || linux + +package cli + +import ( + "context" + "os" + "path/filepath" + "strings" + "syscall" + "testing" + "time" +) + +// TestDeleteInterruptedOpeningAFIFO: opening a FIFO waits for a writer; +// Ctrl-C must not wait with it. +func TestDeleteInterruptedOpeningAFIFO(t *testing.T) { + h := newHarness(t).withKey() + fifo := filepath.Join(t.TempDir(), "keys") + if err := syscall.Mkfifo(fifo, 0o600); err != nil { + t.Skipf("no FIFO here: %v", err) + } + // A writer at the end releases the open the command left behind. + t.Cleanup(func() { + if w, err := os.OpenFile(fifo, os.O_WRONLY|syscall.O_NONBLOCK, 0); err == nil { + w.Close() + } + }) + ctx, cancel := context.WithCancel(context.Background()) + time.AfterFunc(50*time.Millisecond, cancel) + done := make(chan int, 1) + go func() { done <- h.runCtx(ctx, "delete", "--keys", fifo, "--yes") }() + select { + case code := <-done: + if code != ExitInterrupted || !strings.Contains(h.stderr.String(), "interrupted; nothing was deleted") { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + case <-time.After(5 * time.Second): + t.Fatal("opening the keys FIFO ignored Ctrl-C") + } +} diff --git a/internal/cli/delete_test.go b/internal/cli/delete_test.go new file mode 100644 index 0000000..0aba0b1 --- /dev/null +++ b/internal/cli/delete_test.go @@ -0,0 +1,881 @@ +package cli + +import ( + "context" + "fmt" + "io" + "net/http" + "os" + "path/filepath" + "strings" + "sync/atomic" + "testing" + "time" + + "github.com/Codeinwp/optimole-cli/internal/api" + "github.com/Codeinwp/optimole-cli/internal/config" + "github.com/Codeinwp/optimole-cli/internal/testserver" +) + +// keysFile writes lines to a file and returns its path. +func keysFile(t *testing.T, lines ...string) string { + t.Helper() + p := filepath.Join(t.TempDir(), "keys.txt") + if err := os.WriteFile(p, []byte(strings.Join(lines, "\n")+"\n"), 0o644); err != nil { + t.Fatal(err) + } + return p +} + +// addAssets puts n images in the fake library and returns their keys. +func addAssets(h *harness, n int) []string { + keys := make([]string, n) + for i := range keys { + keys[i] = h.srv.AddAsset(testserver.Asset{Filename: fmt.Sprintf("%03d.jpg", i)}) + } + return keys +} + +// deleteRequests returns the DELETE /api/assets events. +func deleteRequests(h *harness) []testserver.Event { + var out []testserver.Event + for _, e := range h.srv.Events() { + if e.Route == testserver.RouteDeleteAssets { + out = append(out, e) + } + } + return out +} + +func TestDeleteKeys(t *testing.T) { + h := newHarness(t).withKey() + keys := addAssets(h, 3) + // Blank lines and spaces are ignored, case does not matter, a repeated + // key is sent once. + file := keysFile(t, "", " "+strings.ToUpper(keys[0])+" ", "", keys[1], keys[0], "\t") + if code := h.run("delete", "--keys", file, "--yes"); code != ExitOK { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + if got := h.stdout.String(); got != "Deleted 2 images.\n" { + t.Errorf("stdout = %q", got) + } + req := deleteRequests(h) + if len(req) != 1 || strings.Join(req[0].Keys, ",") != keys[0]+","+keys[1] { + t.Fatalf("requests = %+v", req) + } + if n := len(h.srv.Deleted()); n != 2 { + t.Fatalf("%d deleted, want 2", n) + } + + // From stdin, which then cannot answer a question: --yes. + h.stdin = strings.NewReader(keys[2] + "\n") + if code := h.run("delete", "--keys", "-", "--yes"); code != ExitOK || h.stdout.String() != "Deleted 1 image.\n" { + t.Fatalf("exit %d: %q %s", code, h.stdout.String(), h.stderr.String()) + } + if n := len(h.srv.Deleted()); n != 3 { + t.Fatalf("%d deleted, want 3", n) + } +} + +func TestDeleteKeysInBatchesOf100(t *testing.T) { + h := newHarness(t).withKey() + keys := addAssets(h, 250) + if code := h.run("delete", "--keys", keysFile(t, keys...), "--yes", "--json"); code != ExitOK { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + lines := h.jsonLines() + if len(lines) != 1 || lines[0]["scope"] != "keys" || lines[0]["requested"] != float64(250) || lines[0]["deleted"] != float64(250) { + t.Fatalf("lines = %v", lines) + } + if _, ok := lines[0]["maybe_deleted"]; ok { + t.Errorf("maybe_deleted on success: %v", lines[0]) + } + var sizes []int + for _, e := range deleteRequests(h) { + sizes = append(sizes, len(e.Keys)) + } + if fmt.Sprint(sizes) != "[100 100 50]" { + t.Fatalf("request sizes = %v", sizes) + } + if n := len(h.srv.Deleted()); n != 250 { + t.Fatalf("%d deleted", n) + } +} + +func TestDeleteKeysStopsAtTheFirstFailure(t *testing.T) { + t.Run("a refused request", func(t *testing.T) { + h := newHarness(t).withKey() + keys := addAssets(h, 250) + unknown := strings.Repeat("ab", 16) + keys[150] = unknown + if code := h.run("delete", "--keys", keysFile(t, keys...), "--yes"); code != ExitFailure { + t.Fatalf("exit %d", code) + } + msg := h.stderr.String() + for _, want := range []string{ + "The following keys are invalid or unauthorized: " + strings.ToUpper(unknown) + " (HTTP 422)", + "Deleted 100 of 250 images before it stopped; the other 150 were not deleted.", + "To finish, run the command again without the first 100 keys: a deleted key counts as invalid if named again.", + } { + if !strings.Contains(msg, want) { + t.Errorf("stderr lacks %q:\n%s", want, msg) + } + } + if n := len(deleteRequests(h)); n != 2 { + t.Errorf("%d requests, want 2: the run goes on after a failure", n) + } + if n := len(h.srv.Deleted()); n != 100 { + t.Errorf("%d deleted, want 100", n) + } + if h.stdout.Len() != 0 { + t.Errorf("stdout = %q", h.stdout.String()) + } + }) + t.Run("an unknown outcome", func(t *testing.T) { + for _, jsonOut := range []bool{false, true} { + h := newHarness(t).withKey() + keys := addAssets(h, 250) + var calls atomic.Int32 + h.srv.Hook = func(route testserver.Route, _ *http.Request) { + if route == testserver.RouteDeleteAssets && calls.Add(1) == 2 { + h.srv.Inject(route, testserver.Response{Status: 500, Body: `{"message":"Server Error"}`}) + } + } + args := []string{"delete", "--keys", keysFile(t, keys...), "--yes"} + if jsonOut { + args = append(args, "--json") + } + if code := h.run(args...); code != ExitFailure { + t.Fatalf("exit %d", code) + } + if want := "Deleted 100 of 250 images before it stopped; 100 more may be deleted too (the request failed), the other 50 were not."; !strings.Contains(h.stderr.String(), want) { + t.Errorf("stderr lacks %q:\n%s", want, h.stderr.String()) + } + if !jsonOut { + continue + } + l := h.jsonLines() + if len(l) != 1 || l[0]["deleted"] != float64(100) || l[0]["maybe_deleted"] != float64(100) || !strings.Contains(l[0]["error"].(string), "HTTP 500") { + t.Errorf("lines = %v", l) + } + } + }) + t.Run("the first request refused", func(t *testing.T) { + h := newHarness(t).withKey() + if code := h.run("delete", "--keys", keysFile(t, strings.Repeat("cd", 16)), "--yes"); code != ExitFailure { + t.Fatalf("exit %d", code) + } + if msg := h.stderr.String(); !strings.Contains(msg, "Nothing was deleted.") || strings.Contains(msg, "To finish") || strings.Contains(msg, "next run") { + t.Errorf("stderr:\n%s", msg) + } + }) +} + +func TestDeleteFolder(t *testing.T) { + h := newHarness(t).withKey() + campaigns := h.srv.AddFolder("Campaigns") + year := h.srv.AddFolder("Campaigns/2026") + spring := h.srv.AddFolder("Campaigns/2026/Spring") + h.srv.AddAsset(testserver.Asset{Filename: "a.jpg", FolderID: year}) + h.srv.AddAsset(testserver.Asset{Filename: "b.jpg", FolderID: spring}) + h.srv.AddAsset(testserver.Asset{Filename: "kept.jpg", FolderID: campaigns}) + h.srv.AddAsset(testserver.Asset{Filename: "root.jpg"}) + + // Paths match case-insensitively, as the server reads them. + if code := h.run("delete", "--folder", " /campaigns/2026/ ", "--yes"); code != ExitOK { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + if got := h.stdout.String(); got != "Deleting 2 images in the background. They leave the library within a few minutes.\n" { + t.Errorf("stdout = %q", got) + } + if req := deleteRequests(h); len(req) != 1 || req[0].FolderID != year || req[0].Keys != nil { + t.Fatalf("requests = %+v", req) + } + if n := len(h.srv.Deleted()); n != 2 { + t.Fatalf("%d deleted, want 2", n) + } + + if code := h.run("delete", "--folder-id", fmt.Sprint(year), "--yes", "--json"); code != ExitOK { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + l := h.jsonLines() + if len(l) != 1 || l[0]["scope"] != "folder" || l[0]["folder_id"] != float64(year) || l[0]["folder"] != "Campaigns/2026" || l[0]["deleting"] != float64(0) { + t.Fatalf("lines = %v", l) + } + if code := h.run("delete", "--folder-id", fmt.Sprint(year), "--yes"); code != ExitOK || h.stdout.String() != "Nothing to delete: the folder \"Campaigns/2026\" has no images.\n" { + t.Fatalf("exit %d: %q", code, h.stdout.String()) + } +} + +func TestDeleteFolderMustBeFoundFirst(t *testing.T) { + h := newHarness(t).withKey() + h.srv.AddFolder("Brand") + for args, want := range map[[2]string]string{ + {"--folder", "Nope"}: `no folder has the path "Nope"`, + {"--folder-id", "999"}: "no folder has the ID 999", + {"--folder", "Brand/x"}: `no folder has the path "Brand/x"`, + } { + if code := h.run("delete", args[0], args[1], "--yes"); code != ExitFailure || !strings.Contains(h.stderr.String(), want) { + t.Errorf("%v: exit %d: %s", args, code, h.stderr.String()) + } + } + // Two folders with one path (a race can make them): neither is guessed. + h.srv.Inject(testserver.RouteFolders, testserver.Response{Status: 200, Body: `{"data":[` + + `{"id":5,"name":"Brand","parent_id":null,"status":"active","is_default":false},` + + `{"id":6,"name":"brand","parent_id":null,"status":"active","is_default":false}],"meta":{"next_cursor":null,"has_more":false}}`}) + if code := h.run("delete", "--folder", "BRAND", "--yes"); code != ExitFailure || !strings.Contains(h.stderr.String(), `2 folders have the path "BRAND" (IDs 5, 6); name one with --folder-id`) { + t.Errorf("exit %d: %s", code, h.stderr.String()) + } + if n := len(deleteRequests(h)); n != 0 { + t.Fatalf("%d delete requests", n) + } +} + +func TestDeleteWebsiteFolder(t *testing.T) { + h := newHarness(t).withKey() + folder := h.srv.AddSite("example.com") + h.srv.AddAsset(testserver.Asset{Filename: "filed.jpg", FolderID: folder, Domain: "example.com"}) + h.srv.AddAsset(testserver.Asset{Filename: "unfiled.jpg", Domain: "example.com"}) + h.srv.AddAsset(testserver.Asset{Filename: "other.jpg", Domain: "other.com"}) + h.tty = true + h.stdin = strings.NewReader("example.com\n") + if code := h.run("delete", "--folder", "example.com"); code != ExitOK { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + if want := `About to delete every image in the folder "example.com" (ID ` + fmt.Sprint(folder) + `) and its subfolders, ` + + `and the images of the website example.com that are in no folder; the folders are kept.`; !strings.Contains(h.stderr.String(), want) { + t.Errorf("stderr lacks %q:\n%s", want, h.stderr.String()) + } + if !strings.HasPrefix(h.stdout.String(), "Deleting 2 images") { + t.Errorf("stdout = %q", h.stdout.String()) + } +} + +func TestDeleteSite(t *testing.T) { + h := newHarness(t).withKey() + folder := h.srv.AddFolder("Blog") + h.srv.AddSite("example.com") + h.srv.AddAsset(testserver.Asset{Filename: "a.jpg", Domain: "example.com"}) + h.srv.AddAsset(testserver.Asset{Filename: "b.jpg", Domain: "example.com", FolderID: folder}) + h.srv.AddAsset(testserver.Asset{Filename: "c.jpg", Domain: "other.com"}) + if code := h.run("delete", "--site", "https://WWW.Example.com/blog", "--yes", "--json"); code != ExitOK { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + l := h.jsonLines() + if len(l) != 1 || l[0]["scope"] != "site" || l[0]["domain"] != "example.com" || l[0]["deleting"] != float64(2) { + t.Fatalf("lines = %v", l) + } + // The host goes, with its "www.": the server strips one, as the prompt did. + if req := deleteRequests(h); len(req) != 1 || req[0].Domain != "www.example.com" { + t.Fatalf("requests = %+v", req) + } + if n := len(h.srv.Events()); n != 1 { + t.Errorf("%d requests, want only the delete", n) + } + + if code := h.run("delete", "--site", "other.com", "--yes"); code != ExitFailure || !strings.Contains(h.stderr.String(), "The account has no website with this domain. (HTTP 422)") { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } +} + +func TestDeleteAsksFirst(t *testing.T) { + type tc struct { + args []string + answer string + prompt string + ok bool + } + h := newHarness(t).withKey() + key := addAssets(h, 1)[0] + folder := h.srv.AddFolder("Campaigns/2026") + h.srv.AddSite("example.com") + file := keysFile(t, key) + for _, c := range []tc{ + {[]string{"--keys", file}, "yes\n", `About to delete 1 image (the keys in ` + file + `). This cannot be undone`, false}, + {[]string{"--keys", file}, "", `Type "delete" to confirm: `, false}, + {[]string{"--folder-id", fmt.Sprint(folder)}, "campaigns/2026\n", `Type "Campaigns/2026" to confirm: `, false}, + {[]string{"--site", "Example.com"}, "Example.com\n", `Type "example.com" to confirm: `, false}, + {[]string{"--keys", file}, " delete \n", `Type "delete" to confirm: `, true}, + {[]string{"--folder", "campaigns/2026"}, "Campaigns/2026\n", "every image in the folder \"Campaigns/2026\" (ID " + fmt.Sprint(folder) + ") and its subfolders; the folders are kept.", true}, + {[]string{"--site", "example.com"}, "example.com\n", "every image of the website example.com, in any folder.", true}, + } { + before := len(deleteRequests(h)) + h.tty = true + h.stdin = strings.NewReader(c.answer) + code := h.run(append([]string{"delete"}, c.args...)...) + if !strings.Contains(h.stderr.String(), c.prompt) { + t.Errorf("%v: stderr lacks %q:\n%s", c.args, c.prompt, h.stderr.String()) + } + sent := len(deleteRequests(h)) - before + if c.ok && (code != ExitOK || sent != 1) { + t.Errorf("%v answered %q: exit %d, %d requests", c.args, c.answer, code, sent) + } + if !c.ok && (code != ExitFailure || sent != 0 || !strings.Contains(h.stderr.String(), "the answer did not match; nothing was deleted")) { + t.Errorf("%v answered %q: exit %d, %d requests: %s", c.args, c.answer, code, sent, h.stderr.String()) + } + } +} + +// TestDeleteInterruptedAtTheQuestion: Ctrl-C at the prompt returns at once, +// with nothing deleted. +func TestDeleteInterruptedAtTheQuestion(t *testing.T) { + h := newHarness(t).withKey() + h.srv.AddFolder("A") + pr, pw := io.Pipe() + t.Cleanup(func() { pw.Close() }) + h.tty = true + h.stdin = pr + ctx, cancel := context.WithCancel(context.Background()) + time.AfterFunc(50*time.Millisecond, cancel) + done := make(chan int, 1) + go func() { done <- h.runCtx(ctx, "delete", "--folder", "A") }() + select { + case code := <-done: + if code != ExitInterrupted || !strings.Contains(h.stderr.String(), "interrupted; nothing was deleted") { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + case <-time.After(5 * time.Second): + pw.Close() + t.Fatal("the question ignored Ctrl-C") + } + if n := len(deleteRequests(h)); n != 0 { + t.Fatalf("%d delete requests", n) + } +} + +// TestDeleteNeedsYesWithoutATerminal: nothing can be confirmed without a +// terminal, so nothing is deleted, or even looked up, without --yes. +func TestDeleteNeedsYesWithoutATerminal(t *testing.T) { + h := newHarness(t).withKey() + key := addAssets(h, 1)[0] + h.srv.AddFolder("A") + h.srv.AddSite("example.com") + for _, args := range [][]string{ + {"--keys", keysFile(t, key)}, + {"--keys", "-"}, + {"--folder", "A"}, + {"--folder-id", "101"}, + {"--site", "example.com"}, + } { + h.stdin = strings.NewReader(key + "\ndelete\n") + code := h.run(append([]string{"delete"}, args...)...) + if code != ExitUsage || !strings.Contains(h.stderr.String(), "add --yes to delete without asking") { + t.Errorf("%v: exit %d: %s", args, code, h.stderr.String()) + } + } + // Keys typed at a terminal could not be told from the answer. + h.tty = true + if code := h.run("delete", "--keys", "-", "--yes"); code != ExitUsage || !strings.Contains(h.stderr.String(), "pipe them in, or name a file") { + t.Errorf("--keys - on a terminal: exit %d: %s", code, h.stderr.String()) + } + if n := len(h.srv.Events()); n != 0 { + t.Fatalf("%d requests", n) + } + if n := len(h.srv.Deleted()); n != 0 { + t.Fatalf("%d deleted", n) + } +} + +func TestDeleteUsageErrors(t *testing.T) { + h := newHarness(t).withKey() + key := addAssets(h, 1)[0] + dir := t.TempDir() + usage := map[string][]string{ + "nothing named": {}, + "keys and site": {"--keys", keysFile(t, key), "--site", "example.com"}, + "folder and folder-id": {"--folder", "A", "--folder-id", "3"}, + "an argument": {"--site", "example.com", "extra"}, + "bad folder ID": {"--folder-id", "abc"}, + "folder ID 0": {"--folder-id", "0"}, + "the root": {"--folder", "/"}, + "empty folder name": {"--folder", "a//b"}, + "empty site": {"--site", ""}, + "not a web URL": {"--site", "ftp://example.com"}, + "empty keys": {"--keys", ""}, + "missing keys file": {"--keys", filepath.Join(dir, "missing.txt")}, + "keys file is a folder": {"--keys", dir}, + "no keys in the file": {"--keys", keysFile(t, "", " ")}, + "a line that is no key": {"--keys", keysFile(t, key, "abc")}, + "bad boolean": {"--keys", keysFile(t, key), "--yes=maybe"}, + } + for name, args := range usage { + code := h.run(append(append([]string{"delete"}, args...), "--yes")...) + if code != ExitUsage { + t.Errorf("%s: exit %d, want 2; stderr: %s", name, code, h.stderr.String()) + } + } + if n := len(h.srv.Events()); n != 0 { + t.Fatalf("%d requests on usage errors", n) + } + + // A bad line is not echoed: it could be anything, an API key too. + const secret = "optml-pasted-api-key-by-mistake-0123456789" + if code := h.run("delete", "--keys", keysFile(t, key, secret), "--yes"); code != ExitUsage { + t.Fatalf("exit %d", code) + } + if msg := h.stderr.String(); strings.Contains(msg, secret) || !strings.Contains(msg, "line 2 of") { + t.Fatalf("stderr: %s", msg) + } +} + +func TestDeleteErrors(t *testing.T) { + t.Run("rejected key", func(t *testing.T) { + h := newHarness(t) + h.vars[config.EnvAPIKey] = "optml-not-a-key-000000" + if code := h.run("delete", "--keys", keysFile(t, strings.Repeat("ab", 16)), "--yes"); code != ExitFailure { + t.Fatalf("exit %d", code) + } + if msg := h.stderr.String(); !strings.Contains(msg, "The API key (from OPTIMOLE_API_KEY) was rejected.") || strings.Contains(msg, "optml-not-a-key") { + t.Fatalf("stderr: %s", msg) + } + }) + t.Run("folder-scoped key", func(t *testing.T) { + h := newHarness(t) + const scoped = "optml-sScopedKeyForDeletes0123" + h.srv.AddKey(scoped, testserver.Key{Scoped: true}) + h.vars[config.EnvAPIKey] = scoped + key := addAssets(h, 1)[0] + h.srv.AddFolder("A") + // The keys are refused by DELETE, the folder already by its lookup. + for _, args := range [][]string{{"--keys", keysFile(t, key)}, {"--folder", "A"}} { + if code := h.run(append([]string{"delete", "--yes"}, args...)...); code != ExitFailure { + t.Fatalf("%v: exit %d", args, code) + } + if msg := h.stderr.String(); !strings.Contains(msg, "Deleting images needs a full-access API key") || strings.Contains(msg, scoped) { + t.Errorf("%v: stderr: %s", args, msg) + } + } + if len(h.srv.Deleted()) != 0 { + t.Fatal("a scoped key deleted") + } + }) + t.Run("other refusals", func(t *testing.T) { + h := newHarness(t).withKey() + folder := h.srv.AddFolder("A") + h.srv.Inject(testserver.RouteDeleteAssets, + testserver.Response{Status: 403, Body: `{"message":"Your plan is not active."}`}, + testserver.Response{Status: 422, Body: `{"message":"The selected folder is invalid","errors":{"folder":["The selected folder is invalid"]}}`}) + for _, want := range []string{"Your plan is not active. (HTTP 403)", "delete assets: The selected folder is invalid (HTTP 422)"} { + if code := h.run("delete", "--folder-id", fmt.Sprint(folder), "--yes"); code != ExitFailure { + t.Fatalf("exit %d", code) + } + if msg := h.stderr.String(); !strings.Contains(msg, want) || strings.Contains(msg, "full-access") || strings.Contains(msg, "may have started") { + t.Errorf("stderr lacks %q (or says too much):\n%s", want, msg) + } + } + }) + t.Run("lost answer", func(t *testing.T) { + h := newHarness(t).withKey() + h.srv.AddSite("example.com") + h.srv.Inject(testserver.RouteDeleteAssets, testserver.Response{CommitThenClose: true}) + if code := h.run("delete", "--site", "example.com", "--yes"); code != ExitFailure { + t.Fatalf("exit %d", code) + } + if !strings.Contains(h.stderr.String(), "The deletion may have started anyway") { + t.Errorf("stderr: %s", h.stderr.String()) + } + if n := len(deleteRequests(h)); n != 1 { + t.Errorf("%d requests, want 1: emptying a website is never repeated blindly", n) + } + }) +} + +func TestDeleteRedactsTheKey(t *testing.T) { + const key = "optml-very-secret-key-for-delete-42" + echo := fmt.Sprintf(`{"message":"Key %s cannot do that.","errors":{"keys":["bad %s"]}}`, key, key) + for _, jsonOut := range []bool{false, true} { + h := newHarness(t) + h.srv.AddKey(key, testserver.Key{}) + h.srv.Inject(testserver.RouteDeleteAssets, testserver.Response{Status: 422, Body: echo}) + args := []string{"delete", "--api-key", key, "--keys", keysFile(t, addAssets(h, 1)...), "--yes"} + if jsonOut { + args = append(args, "--json") + } + if code := h.run(args...); code != ExitFailure { + t.Fatalf("exit %d", code) + } + all := h.stdout.String() + h.stderr.String() + if strings.Contains(all, key) || !strings.Contains(all, "[REDACTED]") { + t.Fatalf("json=%v output:\n%s", jsonOut, all) + } + } + // A key given to --site, --folder-id or --keys by mistake is not echoed. + h := newHarness(t) + h.vars[config.EnvAPIKey] = key + for _, flag := range []string{"--site", "--folder-id", "--keys"} { + if code := h.run("delete", flag, "x/"+key+"://", "--yes"); code != ExitUsage || strings.Contains(h.stderr.String(), key) { + t.Fatalf("%s: exit %d: %s", flag, code, h.stderr.String()) + } + } +} + +func TestDeleteHelp(t *testing.T) { + h := newHarness(t) + if code := h.run("delete", "-h"); code != ExitOK || !strings.Contains(h.stderr.String(), "--folder-id N") { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + if code := h.run("help"); code != ExitOK || !strings.Contains(h.stdout.String(), "delete Delete images") { + t.Fatalf("exit %d: %s", code, h.stdout.String()) + } +} + +func TestDeleteReportsOutputFailures(t *testing.T) { + h := newHarness(t).withKey() + keys := addAssets(h, 1) + env := h.env() + env.Stdout = &failingWriter{} + if code := Run(context.Background(), []string{"delete", "--keys", keysFile(t, keys...), "--yes"}, env); code != ExitFailure || !strings.Contains(h.stderr.String(), "writing output") { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } +} + +// TestDeleteInterruptedWhileReadingKeys: a pipe that never ends must not keep +// Ctrl-C from working. +func TestDeleteInterruptedWhileReadingKeys(t *testing.T) { + h := newHarness(t).withKey() + key := addAssets(h, 1)[0] + pr, pw := io.Pipe() + t.Cleanup(func() { pw.Close() }) + go pw.Write([]byte(key + "\n")) + h.stdin = pr + ctx, cancel := context.WithCancel(context.Background()) + time.AfterFunc(50*time.Millisecond, cancel) + done := make(chan int, 1) + go func() { done <- h.runCtx(ctx, "delete", "--keys", "-", "--yes") }() + select { + case code := <-done: + if code != ExitInterrupted || !strings.Contains(h.stderr.String(), "interrupted; nothing was deleted") { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + case <-time.After(5 * time.Second): + pw.Close() + t.Fatal("reading the keys ignored Ctrl-C") + } + if n := len(h.srv.Events()); n != 0 { + t.Fatalf("%d requests", n) + } +} + +// cancelAfter cancels the run once the first answer to route arrives. +type cancelAfter struct { + next http.RoundTripper + cancel context.CancelFunc +} + +func (c cancelAfter) RoundTrip(r *http.Request) (*http.Response, error) { + resp, err := c.next.RoundTrip(r) + if err == nil && r.Method == http.MethodDelete { + c.cancel() + } + return resp, err +} + +// TestDeleteInterruptedBetweenRequests: Ctrl-C after a request finished stops +// before the next is sent, so the count is exact. +func TestDeleteInterruptedBetweenRequests(t *testing.T) { + h := newHarness(t).withKey() + keys := addAssets(h, 250) + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + env := h.env() + env.Transport = cancelAfter{next: env.Transport, cancel: cancel} + if code := Run(ctx, []string{"delete", "--keys", keysFile(t, keys...), "--yes"}, env); code != ExitInterrupted { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + if want := "Deleted 100 of 250 images before it stopped; the other 150 were not deleted."; !strings.Contains(h.stderr.String(), want) { + t.Errorf("stderr lacks %q:\n%s", want, h.stderr.String()) + } + if n := len(deleteRequests(h)); n != 1 { + t.Errorf("%d requests, want 1", n) + } +} + +// TestDeleteSiteStripsOneWWW: the server strips one "www.", as the prompt +// does, so "www.www.example.com" is the website www.example.com, never +// example.com. +func TestDeleteSiteStripsOneWWW(t *testing.T) { + h := newHarness(t).withKey() + h.srv.AddSite("example.com") + h.srv.AddSite("www.example.com") + h.srv.AddAsset(testserver.Asset{Filename: "a.jpg", Domain: "example.com"}) + h.srv.AddAsset(testserver.Asset{Filename: "b.jpg", Domain: "www.example.com"}) + h.tty = true + h.stdin = strings.NewReader("www.example.com\n") + if code := h.run("delete", "--site", "www.www.example.com"); code != ExitOK || !strings.Contains(h.stderr.String(), "every image of the website www.example.com,") { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + deleted := h.srv.Deleted() + if len(deleted) != 1 { + t.Fatalf("deleted = %v", deleted) + } + for _, a := range h.srv.Assets() { + if a.Deleted && a.Domain != "www.example.com" { + t.Fatalf("deleted an image of %s", a.Domain) + } + } +} + +// TestDeleteNeedsEnter: an answer that ends without Enter (Ctrl-D after +// typing it, or a line too long to read whole) deletes nothing. +func TestDeleteNeedsEnter(t *testing.T) { + h := newHarness(t).withKey() + file := keysFile(t, addAssets(h, 1)...) + for _, answer := range []string{"delete", "delete" + strings.Repeat(" ", maxKeyInput)} { + h.tty = true + h.stdin = strings.NewReader(answer) + if code := h.run("delete", "--keys", file); code != ExitFailure || !strings.Contains(h.stderr.String(), "the answer did not match; nothing was deleted") { + t.Fatalf("answer of %d bytes: exit %d: %s", len(answer), code, h.stderr.String()) + } + } + if n := len(deleteRequests(h)); n != 0 { + t.Fatalf("%d delete requests", n) + } +} + +// TestDeleteLongFolderPathAsksForTheName: a path longer than a terminal line +// cannot be typed, so its name is asked for instead. +func TestDeleteLongFolderPathAsksForTheName(t *testing.T) { + h := newHarness(t).withKey() + segs := make([]string, 17) + for i := range segs { + segs[i] = strings.Repeat(string(rune('a'+i)), 255) + } + long := strings.Join(segs, "/") + folder := h.srv.AddFolder(long) + h.tty = true + h.stdin = strings.NewReader(segs[16] + "\n") + if code := h.run("delete", "--folder-id", fmt.Sprint(folder)); code != ExitOK { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + if !strings.Contains(h.stderr.String(), fmt.Sprintf("Type %q to confirm: ", segs[16])) { + t.Fatalf("stderr: %s", h.stderr.String()) + } + if n := len(deleteRequests(h)); n != 1 { + t.Fatalf("%d delete requests", n) + } +} + +// TestDeleteKeysAfterALostAnswer: a keys request whose answer was lost may +// have deleted. It is not repeated (the repeat would be refused for the keys +// it deleted), and the command says what to do next. +func TestDeleteKeysAfterALostAnswer(t *testing.T) { + h := newHarness(t).withKey() + keys := addAssets(h, 1) + h.srv.Inject(testserver.RouteDeleteAssets, testserver.Response{CommitThenClose: true}) + if code := h.run("delete", "--keys", keysFile(t, keys...), "--yes", "--json"); code != ExitFailure { + t.Fatalf("exit %d", code) + } + msg := h.stderr.String() + for _, want := range []string{"1 of 1 image may be deleted (the request failed).", "If the next run lists keys as invalid, the request that failed deleted them"} { + if !strings.Contains(msg, want) { + t.Errorf("stderr lacks %q:\n%s", want, msg) + } + } + if strings.Contains(msg, "Nothing was deleted") || strings.Contains(msg, "To finish") { + t.Errorf("stderr:\n%s", msg) + } + if n := len(deleteRequests(h)); n != 1 { + t.Errorf("%d requests, want 1", n) + } + if l := h.jsonLines(); len(l) != 1 || l[0]["deleted"] != float64(0) || l[0]["maybe_deleted"] != float64(1) { + t.Errorf("lines = %v", l) + } + if n := len(h.srv.Deleted()); n != 1 { + t.Fatalf("%d deleted: the first attempt did delete", n) + } +} + +// TestDeleteKeysFailuresThatDeleteNothing: a keys request refused with a +// 429 on every attempt, or never sent, deleted nothing, and says so. +func TestDeleteKeysFailuresThatDeleteNothing(t *testing.T) { + check := func(t *testing.T, h *harness, code int, want string) { + t.Helper() + if code != ExitFailure { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + msg := h.stderr.String() + if !strings.Contains(msg, want) || !strings.Contains(msg, "Nothing was deleted.") || strings.Contains(msg, "may be deleted") || strings.Contains(msg, "next run") { + t.Errorf("stderr:\n%s", msg) + } + l := h.jsonLines() + if len(l) != 1 || l[0]["deleted"] != float64(0) { + t.Fatalf("lines = %v", l) + } + if _, ok := l[0]["maybe_deleted"]; ok { + t.Errorf("maybe_deleted: %v", l[0]) + } + } + t.Run("rate limited on every attempt", func(t *testing.T) { + h := newHarness(t).withKey() + keys := addAssets(h, 2) + limited := make([]testserver.Response, api.MaxAttempts) + for i := range limited { + limited[i] = testserver.Response{Status: 429, Body: `{"message":"Too Many Attempts."}`} + } + h.srv.Inject(testserver.RouteDeleteAssets, limited...) + check(t, h, h.run("delete", "--keys", keysFile(t, keys...), "--yes", "--json"), "Too Many Attempts.") + if n := len(deleteRequests(h)); n != api.MaxAttempts { + t.Errorf("%d requests, want %d", n, api.MaxAttempts) + } + if n := len(h.srv.Deleted()); n != 0 { + t.Errorf("%d deleted", n) + } + }) + t.Run("never connected", func(t *testing.T) { + h := newHarness(t).withKey() + env := h.env() + env.Transport = refusingTransport{} + code := Run(context.Background(), []string{"delete", "--keys", keysFile(t, strings.Repeat("ab", 16)), "--yes", "--json"}, env) + check(t, h, code, "connection refused") + }) +} + +// refusingTransport fails every request before it connects. +type refusingTransport struct{} + +func (refusingTransport) RoundTrip(*http.Request) (*http.Response, error) { + return nil, fmt.Errorf("dial tcp: connection refused") +} + +func TestDeleteInBackgroundOutcomes(t *testing.T) { + t.Run("an answer that does not read", func(t *testing.T) { + // The server accepted it, so the deletion runs. + h := newHarness(t).withKey() + h.srv.AddSite("example.com") + h.srv.Inject(testserver.RouteDeleteAssets, testserver.Response{Status: 202, Body: `{"assets":`}) + if code := h.run("delete", "--site", "example.com", "--yes"); code != ExitFailure || !strings.Contains(h.stderr.String(), "The deletion may have started anyway") { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + }) + t.Run("never connected", func(t *testing.T) { + h := newHarness(t).withKey() + env := h.env() + env.Transport = refusingTransport{} + code := Run(context.Background(), []string{"delete", "--site", "example.com", "--yes"}, env) + if code != ExitFailure || !strings.Contains(h.stderr.String(), "connection refused") || strings.Contains(h.stderr.String(), "may have started") { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + }) + t.Run("a 5xx", func(t *testing.T) { + h := newHarness(t).withKey() + h.srv.AddSite("example.com") + h.srv.Inject(testserver.RouteDeleteAssets, testserver.Response{Status: 504, Body: `{"message":"Gateway Timeout"}`}) + if code := h.run("delete", "--site", "example.com", "--yes"); code != ExitFailure || !strings.Contains(h.stderr.String(), "The deletion may have started anyway") { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + if n := len(deleteRequests(h)); n != 1 { + t.Fatalf("%d requests: a 504 is not repeated", n) + } + }) +} + +// TestDeleteHidesKeysAndControlCharacters: whatever is printed (the folder +// asked for, the folder found, the prompt, the JSON, the server's message) +// shows no known API key and no raw control character. +func TestDeleteHidesKeysAndControlCharacters(t *testing.T) { + const key = "optml-very-secret-key-for-delete-77" + h := newHarness(t) + h.vars[config.EnvAPIKey] = key + h.srv.AddKey(key, testserver.Key{}) + folder := h.srv.AddFolder("Shared/" + key) + check := func(what string) { + t.Helper() + if all := h.stdout.String() + h.stderr.String(); strings.Contains(all, key) || strings.Contains(all, "\x1b") { + t.Errorf("%s printed a key or an escape:\n%q", what, all) + } + } + h.run("delete", "--folder", key, "--yes") + check("a folder not found") + if !strings.Contains(h.stderr.String(), `no folder has the path "[REDACTED]"`) { + t.Errorf("stderr: %s", h.stderr.String()) + } + h.tty = true + h.stdin = strings.NewReader("Shared/[REDACTED]\n") + if code := h.run("delete", "--folder-id", fmt.Sprint(folder)); code != ExitFailure { + t.Errorf("a redacted answer was taken: exit %d", code) + } + check("the prompt") + h.tty = false + if code := h.run("delete", "--folder-id", fmt.Sprint(folder), "--yes", "--json"); code != ExitOK { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + check("the JSON") + if l := h.jsonLines(); len(l) != 1 || l[0]["folder"] != "Shared/[REDACTED]" { + t.Errorf("lines = %v", l) + } + h.run("delete", "--keys", filepath.Join(t.TempDir(), "\x1b[2Jmissing.txt"), "--yes") + check("a keys file that does not open") + h.srv.Inject(testserver.RouteDeleteAssets, testserver.Response{Status: 422, Body: `{"message":"Bad \u001b[2J folder"}`}) + h.run("delete", "--folder-id", fmt.Sprint(folder), "--yes") + check("the server's message") + if !strings.Contains(h.stderr.String(), `\x1b[2J`) { + t.Errorf("the escape is not shown quoted: %q", h.stderr.String()) + } +} + +// TestDeleteFolderNeedsTheWholeList: a path two folders share could hide in +// the part of the list that did not come back, so it is not guessed. +func TestDeleteFolderNeedsTheWholeList(t *testing.T) { + h := newHarness(t).withKey() + page := `{"data":[{"id":5,"name":"Brand","parent_id":null,"status":"active","is_default":false}],"meta":{"next_cursor":%s,"has_more":true}}` + for _, next := range []string{`null`, `"same"`} { + h.srv.Inject(testserver.RouteFolders, testserver.Response{Status: 200, Body: fmt.Sprintf(page, next)}, testserver.Response{Status: 200, Body: fmt.Sprintf(page, next)}) + if code := h.run("delete", "--folder", "Brand", "--yes"); code != ExitFailure || !strings.Contains(h.stderr.String(), "the folder list came back incomplete") { + t.Fatalf("next cursor %s: exit %d: %s", next, code, h.stderr.String()) + } + } + // An ID is unique: found in what came back, it is the folder. + h.srv.Inject(testserver.RouteFolders, testserver.Response{Status: 200, Body: fmt.Sprintf(page, `null`)}) + h.srv.Inject(testserver.RouteDeleteAssets, testserver.Response{Status: 202, Body: `{"assets":0}`}) + if code := h.run("delete", "--folder-id", "5", "--yes"); code != ExitOK { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + if n := len(deleteRequests(h)); n != 1 { + t.Fatalf("%d delete requests", n) + } +} + +// TestDeleteSiteSendsTheHost: only the host goes, so a long URL stays under +// the server's 255-character limit, and a domain the CLI cannot map as the +// server does (IDNA) is refused rather than shown wrong. +func TestDeleteSiteSendsTheHost(t *testing.T) { + h := newHarness(t).withKey() + h.srv.AddSite("example.com") + if code := h.run("delete", "--site", "https://Example.com/"+strings.Repeat("p", 300), "--yes"); code != ExitOK { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + if req := deleteRequests(h); len(req) != 1 || req[0].Domain != "example.com" { + t.Fatalf("requests = %+v", req) + } + for _, site := range []string{"İ.com", "bücher.de", "WWW.", "https://Www.", "https://www.", "www."} { + if code := h.run("delete", "--site", site, "--yes"); code != ExitUsage { + t.Errorf("%s: exit %d: %s", site, code, h.stderr.String()) + } + } + if !strings.Contains(h.stderr.String(), `"www." is not a domain`) { + t.Errorf("stderr: %s", h.stderr.String()) + } + h.run("delete", "--site", "bücher.de", "--yes") + if !strings.Contains(h.stderr.String(), "give the domain in its xn-- form") { + t.Errorf("stderr: %s", h.stderr.String()) + } + if n := len(deleteRequests(h)); n != 1 { + t.Fatalf("%d delete requests", n) + } +} + +// TestDeleteFolderLookAlikePaths: a root folder named "2026 / Drafts" reads +// as the nested 2026/Drafts, so neither is guessed. +func TestDeleteFolderLookAlikePaths(t *testing.T) { + h := newHarness(t).withKey() + h.srv.Inject(testserver.RouteFolders, testserver.Response{Status: 200, Body: `{"data":[` + + `{"id":5,"name":"2026 / Drafts","parent_id":null,"status":"active","is_default":false},` + + `{"id":6,"name":"2026","parent_id":null,"status":"active","is_default":false},` + + `{"id":7,"name":"Drafts","parent_id":6,"status":"active","is_default":false}],"meta":{"next_cursor":null,"has_more":false}}`}) + if code := h.run("delete", "--folder", "2026 / Drafts", "--yes"); code != ExitFailure || !strings.Contains(h.stderr.String(), `2 folders have the path "2026/Drafts" (IDs 5, 7)`) { + t.Fatalf("exit %d: %s", code, h.stderr.String()) + } + if n := len(deleteRequests(h)); n != 0 { + t.Fatalf("%d delete requests", n) + } +} diff --git a/internal/cli/folders.go b/internal/cli/folders.go index 7b67f8b..1571d0d 100644 --- a/internal/cli/folders.go +++ b/internal/cli/folders.go @@ -69,24 +69,9 @@ func runFolders(ctx context.Context, args []string, env Env) int { return ExitUsage } - var folders []api.Folder - q := api.FolderQuery{ParentID: parent, PerPage: foldersPerPage} - seen := map[string]bool{} - for page := 0; page < maxFolderPages; page++ { - res, err := sess.client.ListFolders(ctx, q) - if err != nil { - return foldersError(env, sess, err) - } - folders = append(folders, res.Data...) - next := "" - if res.Meta.NextCursor != nil { - next = *res.Meta.NextCursor - } - if !res.Meta.HasMore || next == "" || seen[next] { - break - } - seen[next] = true - q.Cursor = next + folders, _, err := listAllFolders(ctx, sess.client, parent) + if err != nil { + return foldersError(env, sess, err) } lines := make([]folderLine, 0, len(folders)) @@ -140,6 +125,35 @@ func runFolders(ctx context.Context, args []string, env Env) int { return ExitOK } +// listAllFolders returns every active folder, or with parent ("root" or a +// folder ID) that folder's children, following the cursor pages. complete is +// false when the pages stopped before the list did: at the page limit, or at +// a cursor missing or seen before. +func listAllFolders(ctx context.Context, client *api.Client, parent string) (folders []api.Folder, complete bool, err error) { + q := api.FolderQuery{ParentID: parent, PerPage: foldersPerPage} + seen := map[string]bool{} + for page := 0; page < maxFolderPages; page++ { + res, err := client.ListFolders(ctx, q) + if err != nil { + return nil, false, err + } + folders = append(folders, res.Data...) + if !res.Meta.HasMore { + return folders, true, nil + } + next := "" + if res.Meta.NextCursor != nil { + next = *res.Meta.NextCursor + } + if next == "" || seen[next] { + break + } + seen[next] = true + q.Cursor = next + } + return folders, false, nil +} + func foldersError(env Env, sess *session, err error) int { msg := sess.redact.Error(err) var apiErr *api.Error diff --git a/internal/cli/login.go b/internal/cli/login.go index a545815..d5f13e3 100644 --- a/internal/cli/login.go +++ b/internal/cli/login.go @@ -42,8 +42,8 @@ const logoutUsage = `Usage: Deletes the API key saved by "optimole login". ` -// maxKeyInput caps how much of stdin is read for a key; maxKeyLength is the -// longest key accepted. +// maxKeyInput caps how much of stdin is read for a key (or for the answer to +// a prompt); maxKeyLength is the longest key accepted. const ( maxKeyInput = 4096 maxKeyLength = 512 @@ -140,7 +140,7 @@ func readKey(ctx context.Context, env Env) (string, error) { fmt.Fprintln(env.Stderr, "warning: input cannot be hidden on this terminal; the key will be visible as you type.") } fmt.Fprint(env.Stderr, "Optimole API key: ") - line, err = readKeyLine(ctx, f, false) + line, err = readOneLine(ctx, f, false, "the key") if restore != nil { restore() fmt.Fprintln(env.Stderr) @@ -150,7 +150,7 @@ func readKey(ctx context.Context, env Env) (string, error) { } } else { var err error - if line, err = readKeyLine(ctx, env.Stdin, true); err != nil { + if line, err = readOneLine(ctx, env.Stdin, true, "the key"); err != nil { return "", err } } @@ -166,11 +166,11 @@ func readKey(ctx context.Context, env Env) (string, error) { return key, nil } -// readKeyLine reads one line (with skipBlank, the first non-blank one) from -// r, reading at most maxKeyInput bytes. A read cannot be interrupted, so when -// ctx is done it returns at once and leaves the read behind; the process is -// about to exit. -func readKeyLine(ctx context.Context, r io.Reader, skipBlank bool) (string, error) { +// readOneLine reads one line (with skipBlank, the first non-blank one) from +// r, reading at most maxKeyInput bytes; what names the line in errors. A read +// cannot be interrupted, so when ctx is done it returns at once and leaves the +// read behind; the process is about to exit. +func readOneLine(ctx context.Context, r io.Reader, skipBlank bool, what string) (string, error) { type result struct { line string err error @@ -188,7 +188,7 @@ func readKeyLine(ctx context.Context, r io.Reader, skipBlank bool) (string, erro } } if err != nil { - ch <- result{"", fmt.Errorf("reading the key: %w", err)} + ch <- result{"", fmt.Errorf("reading %s: %w", what, err)} return } if !skipBlank || strings.TrimSpace(line) != "" || !strings.HasSuffix(line, "\n") { diff --git a/internal/cli/url.go b/internal/cli/url.go index ed4291d..cdf0692 100644 --- a/internal/cli/url.go +++ b/internal/cli/url.go @@ -240,7 +240,7 @@ func runURL(ctx context.Context, args []string, env Env) int { if file, ok := env.Stdin.(*os.File); ok && termecho.IsTerminal(file) { return usageError(env, "url", "give asset keys or URLs, or pipe them in") } - inputs = func(ctx context.Context) (<-chan string, func() error) { return readLines(ctx, env.Stdin) } + inputs = func(ctx context.Context) (<-chan string, func() error) { return readLines(ctx, env.Stdin, "stdin") } } // The first key connects inside a lookup, whose warnings would otherwise @@ -296,9 +296,9 @@ func (l *lockedWriter) Write(p []byte) (int, error) { return l.w.Write(p) } -// readLines streams r line by line. The returned function gives the read -// error, if any, once the channel is closed. -func readLines(ctx context.Context, r io.Reader) (<-chan string, func() error) { +// readLines streams r line by line; name says what r is in errors. The +// returned function gives the read error, if any, once the channel is closed. +func readLines(ctx context.Context, r io.Reader, name string) (<-chan string, func() error) { ch := make(chan string) var err error go func() { @@ -316,7 +316,7 @@ func readLines(ctx context.Context, r io.Reader) (<-chan string, func() error) { if errors.Is(e, bufio.ErrTooLong) { e = fmt.Errorf("a line is longer than %d bytes", maxURLLine) } - err = fmt.Errorf("reading stdin: %w", e) + err = fmt.Errorf("reading %s: %w", name, e) } }() // err is written before close(ch), and read only after it. diff --git a/internal/testserver/delete.go b/internal/testserver/delete.go new file mode 100644 index 0000000..779e11c --- /dev/null +++ b/internal/testserver/delete.go @@ -0,0 +1,193 @@ +package testserver + +import ( + "bytes" + "encoding/json" + "io" + "net/http" + "net/url" + "sort" + "strings" +) + +// handleDeleteAssets answers DELETE /api/assets like the dashboard. Only +// full-access keys may delete. The body names the assets with exactly one of +// "keys" (1 to 100, every one the account's, deleted at once: 204), "folder" +// (every asset of its tree and, for a website's folder, the website's assets +// in no folder) or "domain" (every asset of one of the account's websites). +// The last two answer 202 with how many assets go; the dashboard deletes them +// in the background, the fake at once. A deleted asset's key is refused like +// an unknown one, as staging does. +func (s *Server) handleDeleteAssets(w http.ResponseWriter, r *http.Request) { + ev, injected := s.begin(RouteDeleteAssets, r) + status := 0 + var drop func() + defer func() { s.finishThenDrop(ev, status, drop) }() + if injected != nil && injected.CommitThenClose { + w, drop = commitThenClose(w) + injected = nil + } + + var body struct { + Keys json.RawMessage `json:"keys"` + Folder json.RawMessage `json:"folder"` + Domain json.RawMessage `json:"domain"` + } + raw, _ := io.ReadAll(r.Body) + decodeErr := json.Unmarshal(raw, &body) + present := func(m json.RawMessage) bool { return m != nil && !bytes.Equal(m, []byte("null")) } + var keys []string + keysErr := json.Unmarshal(body.Keys, &keys) + var folder int64 + folderErr := json.Unmarshal(body.Folder, &folder) + var domain string + domainErr := json.Unmarshal(body.Domain, &domain) + ev.Keys, ev.FolderID, ev.Domain = keys, folder, domain + + if injected != nil { + status, drop = writeInjected(w, injected) + return + } + if ct := r.Header.Get("Content-Type"); ct != "application/json" { + s.violate("DELETE %s: Content-Type %q; the dashboard reads no body without application/json", r.URL.Path, ct) + } + key, ok := s.authenticate(w, r) + if !ok { + status = http.StatusUnauthorized + return + } + if key.Scoped { + status = writeJSON(w, http.StatusForbidden, map[string]string{"message": "Full API access required"}) + return + } + switch { + case decodeErr != nil: + status = validation(w, "keys", "The request body must be JSON.") + return + case present(body.Keys) && (keysErr != nil || len(keys) < 1 || len(keys) > 100): + status = validation(w, "keys", "The keys field must be an array of 1 to 100 items.") + return + case present(body.Folder) && (folderErr != nil || folder < 1): + status = validation(w, "folder", "The folder field must be an integer of at least 1.") + return + case present(body.Domain) && (domainErr != nil || len(domain) > 255): + status = validation(w, "domain", "The domain field must be a string of at most 255 characters.") + return + } + for _, k := range keys { + if len(k) != 32 { + status = validation(w, "keys.0", "The keys.0 field must be 32 characters.") + return + } + } + named := 0 + for _, filled := range []bool{present(body.Keys), present(body.Folder), domain != ""} { + if filled { + named++ + } + } + if named != 1 { + status = validation(w, "keys", `Name the files to delete with exactly one of "keys", "folder" and "domain".`) + return + } + + s.mu.Lock() + defer s.mu.Unlock() + switch { + case keys != nil: + var missing []string + seen := map[string]bool{} + for _, k := range keys { + upper := strings.ToUpper(k) + if seen[upper] { + continue + } + seen[upper] = true + if a := s.assets[strings.ToLower(k)]; a == nil || a.Deleted { + missing = append(missing, upper) + } + } + if len(missing) > 0 { + status = validation(w, "keys", "The following keys are invalid or unauthorized: "+strings.Join(missing, ", ")) + return + } + for _, k := range keys { + s.assets[strings.ToLower(k)].Deleted = true + } + w.WriteHeader(http.StatusNoContent) + status = http.StatusNoContent + case folder > 0: + f := s.folderLocked(folder) + if f == nil { + status = validation(w, "folder", "The selected folder is invalid") + return + } + tree := s.treeLocked(f.ID) + status = s.deleteMatchingLocked(w, func(a *Asset) bool { + return tree[a.FolderID] || (f.IsDefault && a.FolderID == 0 && a.Domain == f.Name) + }) + default: + d := normalizeDomain(domain) + if !s.sites[d] { + status = validation(w, "domain", "The account has no website with this domain.") + return + } + status = s.deleteMatchingLocked(w, func(a *Asset) bool { return a.Domain == d }) + } +} + +// deleteMatchingLocked deletes the assets match selects and answers 202 with +// their count. +func (s *Server) deleteMatchingLocked(w http.ResponseWriter, match func(*Asset) bool) int { + n := 0 + for _, a := range s.assets { + if !a.Deleted && match(a) { + a.Deleted = true + n++ + } + } + return writeJSON(w, http.StatusAccepted, map[string]int{"assets": n}) +} + +// treeLocked returns the IDs of a folder and of every folder below it. +func (s *Server) treeLocked(root int64) map[int64]bool { + tree := map[int64]bool{root: true} + for grew := true; grew; { + grew = false + for _, f := range s.folders { + if !tree[f.ID] && f.ParentID != 0 && tree[f.ParentID] { + tree[f.ID] = true + grew = true + } + } + } + return tree +} + +// normalizeDomain reads a domain the way the dashboard does: the host of a +// URL or of a bare domain, in lower case and without "www.". +func normalizeDomain(d string) string { + d = strings.TrimSpace(d) + if !strings.Contains(d, "://") { + d = "https://" + d + } + u, err := url.Parse(d) + if err != nil { + return "" + } + return strings.TrimPrefix(strings.ToLower(u.Hostname()), "www.") +} + +// Deleted returns the keys of the deleted assets, sorted. +func (s *Server) Deleted() []string { + s.mu.Lock() + defer s.mu.Unlock() + var keys []string + for k, a := range s.assets { + if a.Deleted { + keys = append(keys, k) + } + } + sort.Strings(keys) + return keys +} diff --git a/internal/testserver/testserver.go b/internal/testserver/testserver.go index 9e0277f..1294a54 100644 --- a/internal/testserver/testserver.go +++ b/internal/testserver/testserver.go @@ -1,8 +1,9 @@ // Package testserver is an in-memory fake of the dashboard's unified upload API // and of the presigned storage endpoint, for tests only. It follows the API // contract: POST /api/assets, PUT to the presigned URL, POST -// /api/assets/{key}/completion, GET /api/folders, the /api/asset-exports -// endpoints and GETs of the export archives. +// /api/assets/{key}/completion, GET /api/assets/{key}, DELETE /api/assets, +// GET /api/folders, the /api/asset-exports endpoints and GETs of the export +// archives. package testserver import ( @@ -40,6 +41,8 @@ const ( RouteComplete Route = "complete" RouteAsset Route = "asset" RouteFolders Route = "folders" + // RouteDeleteAssets is DELETE /api/assets. + RouteDeleteAssets Route = "delete-assets" ) // Response is a canned answer injected in front of the normal handling. @@ -77,6 +80,8 @@ type Folder struct { ID int64 Name string ParentID int64 // 0 = root + // IsDefault marks a website's folder, named after its domain. + IsDefault bool } // Asset is an asset reserved through the fake. @@ -96,6 +101,10 @@ type Asset struct { // NoURL makes the asset's answer carry "url": null, as the dashboard // does for a file that is not on the CDN yet. NoURL bool + // Domain is the website the asset was offloaded from ("" for files + // uploaded straight to the library). + Domain string + Deleted bool } // Event records one request the fake handled, with sequence numbers taken @@ -113,6 +122,10 @@ type Event struct { Query string // Range is the Range header of an archive download. Range string + // Keys and Domain are what a DELETE /api/assets named (FolderID holds + // its folder). + Keys []string + Domain string } // Server is the fake. Create it with New. @@ -147,6 +160,7 @@ type Server struct { mu sync.Mutex seq int64 keys map[string]Key + sites map[string]bool folders []*Folder nextFolder int64 assets map[string]*Asset @@ -171,12 +185,13 @@ func New(t testing.TB) *Server { t.Helper() s := &Server{ keys: map[string]Key{DefaultKey: {}}, + sites: map[string]bool{}, assets: map[string]*Asset{}, injected: map[Route][]Response{}, nextFolder: 100, } api := http.NewServeMux() - api.HandleFunc("/api/assets", s.handleCreate) + api.HandleFunc("/api/assets", s.handleAssets) api.HandleFunc("/api/assets/", s.handleAssetPath) api.HandleFunc("/api/folders", s.handleFolders) api.HandleFunc("/api/asset-exports", s.handleExports) @@ -230,6 +245,17 @@ func (s *Server) AddFolder(p string) int64 { return parent } +// AddSite adds a website to the account, with its folder (a root folder +// named after the domain), and returns the folder's ID. +func (s *Server) AddSite(domain string) int64 { + s.mu.Lock() + defer s.mu.Unlock() + s.sites[domain] = true + f := s.newFolderLocked(domain, 0) + f.IsDefault = true + return f.ID +} + // Inject queues canned answers for a route; each request pops one. func (s *Server) Inject(route Route, responses ...Response) { s.mu.Lock() @@ -422,6 +448,15 @@ func (s *Server) authenticate(w http.ResponseWriter, r *http.Request) (Key, bool return k, true } +// handleAssets serves POST (create) and DELETE /api/assets. +func (s *Server) handleAssets(w http.ResponseWriter, r *http.Request) { + if r.Method == http.MethodDelete { + s.handleDeleteAssets(w, r) + return + } + s.handleCreate(w, r) +} + func (s *Server) handleCreate(w http.ResponseWriter, r *http.Request) { ev, injected := s.begin(RouteCreate, r) status := 0 @@ -894,7 +929,7 @@ func (s *Server) handleFolders(w http.ResponseWriter, r *http.Request) { p = f.ParentID } matched = append(matched, map[string]any{ - "id": f.ID, "name": f.Name, "parent_id": p, "user_id": 1, "status": "active", "is_default": false, + "id": f.ID, "name": f.Name, "parent_id": p, "user_id": 1, "status": "active", "is_default": f.IsDefault, "created_at": "2026-10-26T10:00:00.000000Z", "updated_at": "2026-10-26T10:00:00.000000Z", }) }