Scaling bharatnet video uploads to 200 TB: from server disks to R2 + presigned URLs

We ingest a large corpus of video — roughly 200 TB today and growing — with individual files ranging from a few hundred MB to tens of GB. Uploads come from both the web app and the mobile app, often many at once. This post is about the upload path specifically: how the bytes get from a client to durable storage reliably and at scale. (For what happens after the bytes land, see Cost-efficient video processing at scale.)

The interesting part isn't the final design — it's why the obvious first design fell over, and what changed.

V1: the server-mediated path (and its hard ceiling)

The first version was the straightforward one. A client asked our backend for an upload endpoint, and the bytes flowed through the server onto storage attached to it. Our backend was a self-hosted Supabase instance, so the database, the API, and the uploaded files all lived on the same box.

                  upload request
   ┌────────┐  ───────────────────▶  ┌──────────────────────┐
   │ Client │                         │  App server          │
   │ web /  │  ◀───── upload URL ───  │  (self-hosted        │
   │ mobile │                         │   Supabase)          │
   └────────┘  ====== bytes ======▶   │  ┌────────────────┐  │
                                       │  │ attached disk  │  │
                                       │  │  / bucket      │  │
                                       │  └────────────────┘  │
                                       └──────────────────────┘

This works fine until it doesn't. Two problems compounded:

• Storage was bound to the server. We could not grow it past ~60 TB — expanding meant resizing volumes on a box that was also serving database traffic, and we were heading well beyond that.
• Every byte traversed the app server. The server became a throughput bottleneck and a single point of failure for an operation (moving large files) that has nothing to do with application logic.

A 200 TB target on a single server's disk simply isn't a thing you can engineer your way into. The storage model was wrong, not the tuning.

The pivot: object storage + direct-to-storage uploads

The fix was to stop treating uploads as something the server carries and start treating them as something the server authorizes. The bytes should go straight from the client to an object store; the server should only mint permission and record state.

We chose Cloudflare R2 for the object store. Three reasons, in order of how much they mattered:

1. It scales past the ceiling entirely. Object storage has no "resize the volume" step — capacity is not our problem anymore.
2. Zero egress fees. With 200 TB that will be read repeatedly (processing, re-processing, serving), R2's no-egress pricing changes the cost math meaningfully versus typical object stores.
3. S3-compatible API. The standard multipart upload API means existing SDKs and tooling work without invention.

For the orchestration layer, we already ran on Supabase — so a Supabase Edge Function became the presigned-URL provider. No new service to operate; it sits right next to the Postgres database that tracks upload state. The Edge Function holds the R2 credentials and never lets them near the client.

The architecture

   ┌────────┐                          ┌──────────────────────┐
   │ Client │ ── 1. start upload ────▶ │  Supabase Edge Fn     │
   │ web /  │                          │  (presign provider)   │
   │ mobile │ ◀─ 2. per-part URLs ──── │   holds R2 creds      │
   └───┬────┘                          └──────────┬────────────┘
       │                                          │
       │                                          │  reads/writes state
       │   3. PUT part 1..N (direct)              ▼
       │  ════════════════════════▶   ┌──────────────────────┐
       │                               │  Supabase Postgres   │
       │                               │  upload_sessions     │
       ▼                               │  upload_parts        │
   ┌──────────────────┐                └──────────────────────┘
   │  Cloudflare R2    │ ◀── 5. CompleteMultipartUpload ── (Edge Fn)
   │  (object store)   │
   └──────────────────┘   4. client sends collected ETags ─▶ Edge Fn

The key property: the bytes never touch our server. Parts go straight from the client to R2 over presigned URLs. The Edge Function only ever handles small JSON — presign requests, part metadata, and the final completion call.

The upload flow, step by step

We use per-part presigned URLs — the standard S3/R2 multipart pattern:

1. Initiate. The client calls the Edge Function with the file's name, size, and content type. The function calls CreateMultipartUpload on R2, gets an uploadId, and writes an upload_sessions row in Postgres (status uploading). It splits the file into N parts and returns a presigned URL per part.
2. Upload parts directly. The client PUTs each part straight to R2 using its presigned URL. Because clients are web and mobile, parts upload concurrently with a bounded pool. R2 returns an ETag for each successful part.
3. Record parts. Each completed part (part number + ETag) is recorded as an upload_parts row. This is what lets us reason about progress and recover from failures.
4. Hand back ETags. Once all parts succeed, the client sends the collected (partNumber, ETag) list to the Edge Function.
5. Finalize server-side. The Edge Function calls CompleteMultipartUpload on R2 with the part list. R2 stitches the parts into the final object. The function flips the upload_sessions row to completed and records the final object key.

Finalizing on the server (not the client) is deliberate: the client can't be trusted to be the source of truth for "this object is now durable and correct." The Edge Function owns that transition.

The state model

Postgres tracks state at two levels — the session and each part:

-- one row per upload
upload_sessions (
  id            uuid primary key,
  r2_upload_id  text not null,        -- R2 multipart uploadId
  object_key    text not null,
  owner_id      uuid not null,
  size_bytes    bigint,
  status        text not null,        -- uploading | completed | failed
  created_at    timestamptz default now()
)

-- one row per part — this is what makes recovery possible
upload_parts (
  session_id    uuid references upload_sessions(id),
  part_number   int not null,
  etag          text,                 -- set once the part lands in R2
  status        text not null,        -- pending | uploaded | failed
  primary key (session_id, part_number)
)

Because each part is tracked independently, a failed part is retried on its own rather than restarting the whole multi-GB file. If part 7 of 40 fails its PUT, the client re-requests a fresh presigned URL for part 7 and re-uploads just that slice. The other 39 parts are untouched.

One honest limitation: today this is in-session retry, not full cross-session resume. If a client drops entirely and comes back later, we don't yet rehydrate "which parts already exist" to continue a stale uploadId — that's the natural next step, and the per-part table already has the data model for it.

Why this scales where V1 didn't

• No storage ceiling. Capacity is R2's problem now, not a volume we resize.
• The server is out of the data path. Presigning is a small, stateless, cheap operation. 200 TB of bytes flowing doesn't translate into 200 TB through our backend — it translates into a few KB of JSON per upload.
• Concurrency is the client's to spend. Web and mobile clients open many part PUTs in parallel directly against R2, which is built for exactly that fan-out.
• Integrity is explicit. ETags per part plus a server-owned CompleteMultipartUpload mean a finished object is verified, not assumed.

Takeaways

The lesson wasn't "use object storage" — it was recognizing that uploads are an authorization problem, not a transport problem for the application server. Once the server stopped carrying bytes and started only minting permission and recording state, the 60 TB wall disappeared and the design got simpler, not more complex: one Edge Function, two tables, and an S3-compatible API doing the heavy lifting.

Next on the list: turning the per-part table into true cross-session resume, so a flaky mobile connection can pick up a 20 GB upload exactly where it left off.