Field Report · Infrastructure

Eight hours in a bucket.

A platform that sells itself on "no lock-in" has spent two and a half years letting three GitHub issues about bulk export rot on the vine. Here are the receipts.

By Ryan · Belief Engines

Last night I spent eight hours extracting three to four gigabytes of my own synthetic data from a Supabase Storage bucket. Not exotic data. Not a ten-terabyte archive. A small- to medium-sized collection of podcast-derived artifacts I had paid real money to generate, correlated to transcripts that live in the same project. When I went to get my files out, I discovered that Supabase — a company whose entire positioning is your data, your Postgres, no lock-in — has no first-class path for bulk export out of Storage. The tooling that exists is either undocumented-until-you-hunt-for-it, flagged "experimental" for over two years, or relegated to a Sept 2023 feature request that the maintainers closed by relabeling it a "documentation" problem.

I am not claiming malice. I am claiming that egress has been a second-class citizen on this platform for a long time, that three separate tracking issues across two repos all ask for the same thing, and that the path of least resistance for a user with a few gigs in a bucket is: write your own recursive loop.

Four-pane tmux workspace at the peak of the download session: a live status dashboard on the left, a stream of 'Downloading:' log lines on the right, and two terminals checking disk usage across the bucket prefixes.
FIG. 1 What "just write a walker" actually looks like — live status, log tail, and two disk-usage terminals babysitting the crawl.

What actually exists

Supabase Storage is, under the hood, a metadata table in Postgres plus blob storage behind it. The SDK surface has a list() method that returns the contents of one folder at a time — it is not recursive. This is confirmed by a Supabase maintainer in Discussion #17762: "The storage .list files can only do one folder at a time." If your bucket has any nested structure, you write a walker. If your bucket has a lot of nested structure, you write a walker, pagination logic, retry logic, and hope.

There is a CLI. This part surprised me, because several blog posts insist there isn't. It lives at supabase storage cp and it does support a -r recursive flag. The catch — and it is a big catch — is the required --experimental flag, which has been sitting there on all four storage subcommands (ls, cp, mv, rm) for the entire existence of the feature. The command that actually works, documented inside a GitHub issue rather than a marketing page, looks like this:

supabase --experimental storage cp -r "ss:///bucket-name" "/output"

It works. It is also slow. The open issue asking for it to not be slow (cli#1798) was filed December 2023 and is still open.

Download status table. 7,899 files, 614 MB, 25 attempts, 24 crashes. Prefix progress bars: speakers 100%, search 100%, runs 100%, raw 40% active, persons ~98% active, matrices 0% starting, embeddings and beliefs queued. Overall ~3%. Crashes escalating — gateway is struggling.
FIG. 2 One hour in. 24 crashes, auto-recovered each time. Three of eight prefixes done. The two biggest buckets (beliefs/, embeddings/) haven't started.
Metrics card: elapsed 1h 0m 27s, started 21:53, now 22:54, 7,899 files, throughput ~131 files/min, 614 MB, byte rate ~10.2 MB/min.
FIG. 2a The tombstone: ~131 files/min, 10.2 MB/min.
The feature request for bucket.downloadAll() was closed by relabeling it a documentation issue. — supabase/storage #363, Sept 2023

The receipts

I went hunting for the ground truth in the issue trackers. Here is what I found, organized by the GitHub ID so you can verify each claim yourself.

Exhibit A · Open & closed issues, both repos
#363
supabase/storage · "Downloading or backing up entire storage bucket" Closed
Opened Sept 3, 2023. Requested a first-class bucket.downloadAll() or bucket.backup() endpoint. Maintainers closed it and applied the documentation label. No endpoint was built.
#1388
supabase/cli · "Downloading Storage Bucket Contents" Open
Opened Aug 18, 2023. Asked for CLI support to pull bucket contents into local development. Still open.
#1798
supabase/cli · "Download all files from bucket in parallel" Open
Opened Dec 2023. The workaround command (supabase --experimental storage cp -r) exists and is cited in the issue body, but it does not scale for buckets with tens of thousands of files.
CLI
CLI reference: supabase storage cp Experimental
All four storage subcommands — ls, cp, mv, rm — require the --experimental flag to run. No promotion to stable has happened.
DOCS
Backup & Restore guide
Supabase's own official migration guide instructs users to exclude storage bucket tables from pg_dump. Which means their official backup procedure does not back up your storage.

Two and a half years, visualized

TIMELINE · STORAGE EGRESS REQUESTS Filed, still waiting. Aug 2023 Dec 2023 2024 2025 TODAY cli#1388 "Download storage bucket contents" · OPEN storage#363 "downloadAll() endpoint" · CLOSED as "documentation" cli#1798 "parallel bucket download" · OPEN --experimental flag on storage cp/ls/mv/rm · never promoted to stable Open issue Closed issue Still unresolved
FIG. 4 Lifetimes of the three storage-export tracking issues, Aug 2023 → today

So what is actually going on

The boring explanation is the correct one. Supabase Storage was built for ingest and serving — user uploads, CDN delivery, signed URLs for images and video. That is the 95% case for their customer base, and they have built it well. Bulk extraction — the case where a user wants to pack up their data and leave, or even just take a full backup — has been under-invested in because the team has bigger fish to fry. The SDK list() was designed for a UI showing twelve thumbnails at a time, not for a recursive archival walk.

That is a choice, not a conspiracy, and it is a defensible choice on its own. What makes it not okay is the marketing. Supabase sells on open-source, on Postgres, on portability, on "no lock-in." If the path to get your own files out involves writing a recursive walker against an experimental CLI flag that has been experimental for 2.5 years, the marketing does not match the product. That gap is the story.

Status panel at 69 minutes elapsed: 7,925 files, 614 MB, 31 attempts. +26 files in last 8 min (STALLED), no growth in 8 min, +6 crashes in 8 min. Crash cadence sparkline trending up. Throughput sparkline collapsing. Diagnostic: the CLI is spending more time restarting and re-enumerating than downloading. A red-dot table shows crashes per hour at 27 vs healthy 0–2, files/min at 3 vs 300+, data growth flat vs active. Three options to fix it: (A) same command with more jobs, (B) one prefix at a time sequentially, (C) ditch the Supabase CLI and roll a Python httpx script.
FIG. 3 69 minutes in, the verdict. Crashes per hour: 27. Files/min: 3. "This approach is broken."

If you are sitting on a Supabase bucket right now

Use the CLI. It works, it just isn't advertised:

supabase login
supabase link --project-ref YOUR_PROJECT_REF
supabase --experimental storage cp -r "ss:///your-bucket-name" "./local-backup" -j 8

The -j 8 gives you eight parallel download jobs, which materially helps. For buckets with tens of thousands of files, the Supabase CLI is the wrong tool — skip it.

Better: skip the Supabase CLI, use the S3 API

This is the part Supabase themselves will tell you if you hunt for it. In their own troubleshooting doc — "Inefficient folder operations and hierarchical RLS challenges", last edited April 2026 — they say quietly:

Supabase Storage also supports an S3-compatible API. This allows you to use tools like the AWS CLI to perform bulk file operations such as downloading, moving, or reorganizing objects more efficiently. — supabase.com/docs · Troubleshooting · Apr 2026

Translation: the Supabase SDK and CLI are not equipped with the built-in capabilities of the underlying S3 service. The recommended path is to stop using Supabase's tooling entirely, generate S3 credentials from the dashboard, point the aws CLI at your project's storage endpoint, and pull everything with aws s3 cp --recursive. Which is the same thing as saying: the real tool here is the AWS CLI, not ours.

aws configure --profile supabase-s3
aws s3 cp s3://your-bucket-name ./local-backup \
  --profile supabase-s3 \
  --endpoint-url https://YOUR_PROJECT_REF.supabase.co/storage/v1/s3 \
  --recursive \
  --region YOUR_REGION

This works. It is also, by any reasonable reading, an admission that the platform-native egress tooling does not scale and you should reach for a general-purpose S3 client instead. If that is where we are, the "no lock-in" story should say so up front, and the supabase storage cp page should link to this troubleshooting article in bold at the top, not bury it three levels deep in the docs.

Back up your storage separately from your pg_dump, because their own migration docs tell you to.

And file a 👍 on cli#1798 and cli#1388. Those issues have been sitting there long enough. They deserve to move.

§ · § · §