Code10x
Scale & cachingAdvanced

Video streaming, where the bytes never touch your service

A video platform is two systems that share a database: a pipeline that turns one uploaded file into dozens of small ones, and a delivery path in which your servers hand out URLs and the CDN hands out bytes.

The brief

Creators upload videos. Viewers watch them on phones, laptops and televisions, over connections that range from a good fibre line to a bus with two bars of signal.

A video must start playing within a couple of seconds, keep playing as the connection changes, and cost less to serve than it earns.

Requirements

Functional

  • Upload a video of up to several gigabytes and process it into playable form
  • Play a video from any point, adapting quality to the viewer's bandwidth
  • Show the creator when processing is done and the viewer a catalogue and a watch page
  • Count views, and resume where a viewer left off

Non-functional

  • Time to first frame under two seconds on a typical connection
  • Playback must not stall when bandwidth drops; it may lower quality
  • Processing may take minutes; it must never lose an upload
  • Bandwidth is the dominant cost and the design must keep it off the origin

Back-of-envelope

Assume

  • 10,000 uploads a day, averaging 500 MB and 8 minutes
  • 2 million viewing sessions a day, averaging 6 minutes watched
  • Five renditions per video (240p to 1080p), totalling about 1.2× the source in bytes
  • The CDN serves 95% of segment requests from its edge caches
  • A viewer's player asks for a 4-second segment every 4 seconds

Therefore

  • Upload ingest: 10,000 × 500 MB = 5 TB a day into object storage, about 60 MB/s averaged, spiky in practice. Through a presigned URL, none of it passes through an application server.
  • Transcoding: 10,000 videos × 8 minutes × 5 renditions is 400,000 output-minutes a day. At roughly real-time per rendition on one core, that is about 280 core-days of work per day — a fleet of ~12 machines with 24 cores running flat out, or a managed transcoding service billed per minute.
  • Delivery: 2 million sessions × 6 minutes at an average 3 Mbit/s is 2M × 360 s × 0.375 MB/s ≈ 270 TB a day. At 95% edge hit rate the origin (object storage) serves ≈13.5 TB a day; the CDN serves the rest. Egress from the origin is the cost line to watch.
  • Segment requests: 2M sessions × 90 segments each = 180 million requests a day, about 2,100 a second on average. Every one of them is a static file with a stable URL, which is what makes a CDN able to take them.
  • Manifest and metadata reads hit your service: one manifest per session and a handful of API calls, about 25 requests a second averaged. The service is small; the pipeline and the CDN are where the scale lives.

Change the edge hit rate from 95% to 80% and origin egress quadruples. The hit rate is a function of catalogue shape — a few popular videos cache well, a long tail does not — and it is the number that decides the bill more than any code you write.

The interface

POST /videos { title, sizeBytes, contentType } → 201 { videoId, uploadUrl (presigned, expires in 1h), uploadId }The service records the intent and hands back a URL for object storage. For multi-gigabyte files the URL is a multipart upload the client drives in parts, so a dropped connection resumes a part rather than the whole file.
POST /videos/{id}/uploaded → 202 { status: PROCESSING }The client says it finished; the service verifies the object exists and its size, then enqueues transcoding. 202 because processing takes minutes; the creator's page polls or subscribes for READY.
GET /videos/{id}/manifest → 302 to https://cdn.example/{id}/master.m3u8?token=… | 404 | 409 not readyThe service authorises the viewer, mints a short-lived signed token for the CDN, and redirects. From here every byte — manifest, renditions, segments — comes from the CDN. The service is out of the data path.
POST /videos/{id}/progress { positionSeconds } → 204 (batched from the player every 15 s)The only write during playback, and it is a sample: a lost one is replaced 15 seconds later, so it is never retried. Views are counted from these, not from segment requests, which the CDN sees and you do not.

What is stored

videosvideo_id · owner_id · title · status (UPLOADING | PROCESSING | READY | FAILED) · duration_s · source_key · created_at · version
Status is the state machine the pipeline drives, and the conditional update PROCESSING → READY on the version column is what stops a retried transcoding job from overwriting a newer result.
renditionsvideo_id · quality (240p … 1080p) · codec · bitrate_kbps · manifest_key · segment_count · ready_at
One row per rendition, so the master manifest is built from what exists: a video can be READY at 480p while 1080p is still transcoding, and the player never asks for a rendition that is not there.
object storage (bucket per purpose)uploads/{videoId}/source.mp4 · streams/{videoId}/{quality}/{n}.ts · streams/{videoId}/master.m3u8 · thumbnails/{videoId}/{n}.jpg
Immutable, content-addressed layout: a segment's URL never changes once written, which is what lets the CDN cache it for a year. A re-encode writes new keys under a new rendition id rather than overwriting.
watch_progress(user_id, video_id) → position_s · updated_at; TTL 90 days
A key-value shape with one write per 15 seconds per viewer and one read per session start. It does not belong in the relational database that holds the catalogue, and it can be lost without losing anything a user would call data.

The design

Upload serviceCreates the video record and the presigned multipart upload, and on completion verifies the object and enqueues a transcoding job. Never reads or writes video bytes.
Transcoding pipelineWorkers consume jobs from a queue, pull the source from object storage, produce each rendition as a stream of 4-second segments plus a rendition manifest, write them to storage, and mark the rendition ready. One job per (video, rendition), so five workers can process one video in parallel and a failure retries one rendition.
Manifest builderWrites the master manifest listing every rendition that is ready, and rewrites it as more renditions arrive. The player reads the master, then picks a rendition per segment based on measured bandwidth — adaptive bitrate is the player's decision, informed by what the manifest offers.
CDN and signed URLsAll playback traffic. The service signs a token per session that the CDN validates at the edge, so a manifest URL copied to a forum stops working when the token expires, and the origin is never asked for a segment the edge already has.
Catalogue and watch APIThe small relational service: listing, search (handed to the search engine), the watch page's metadata, view counts, and progress. Cached aggressively, because it is read for every session and changes rarely.

The decisions

Each of these could go the other way. The choice, the reason, and what it costs — a design that lists only what it chose teaches the choice; one that lists what it gave up teaches the judgement.

Stream the file, or segment it?Chose — Segmented adaptive streaming (HLS or DASH): 4-second segments per rendition and a manifestBecause — A player that downloads one 500 MB file cannot change quality when bandwidth drops and cannot seek without a range request the CDN may not cache. Segments make every piece a small static file: cacheable, seekable, and switchable per segment.Five renditions is 1.2× the storage and five times the transcoding of one file, and a manifest that must be kept correct. The player is also more complex, which is why nobody writes one; the format exists so you do not have to.
Transcode on upload, or on first view?Chose — On upload, all renditions, before the video is READYBecause — A creator expects to publish and a viewer expects instant play; on-demand transcoding puts minutes of latency on the first viewer of every video and a cold start on every rendition switch.Every upload is transcoded whether or not it is ever watched. With a long tail of unwatched videos that is most of the pipeline's work; the scaling section's lazy lower renditions is the compromise.
Serve segments from your servers or from a CDN?Chose — CDN in front of object storage, with the application entirely out of the data pathBecause — The estimate is 270 TB a day. Serving that from application servers is a fleet whose only job is copying bytes from storage to sockets, and a CDN does it from a cache near the viewer for a fraction of the egress cost and latency.You do not see the requests. View counts, buffering rates and errors come from the player's reports and the CDN's logs, delivered later, not from your access log. Cache invalidation is a request to a third party, which is why the URLs are immutable.
Who counts a view?Chose — The player, reporting progress samples to the service, with a view counted at a threshold (say 30 seconds watched)Because — Segment requests happen at the CDN and include prefetches, retries and bots. A view is a product definition, and the player is the only component that knows a human watched for how long.Counts are late by one reporting interval and can be inflated by a modified client; the fraud problem moves from the network to the player, where it is at least visible.

What breaks first

In order. Each names what you would actually observe, and each fix carries its cost.

The transcoding queue after a spikeSymptom — A viral event or a bulk import lands 50,000 uploads in an hour; the queue depth climbs, time-to-READY goes from minutes to a day, and the creator support queue fills with 'my video is stuck'.Fix — Autoscale workers on queue depth with a ceiling, and prioritise: the lowest rendition first so a video is playable in minutes while the rest queue.A ceiling means the backlog is real during a spike; priority means 1080p can be hours behind 480p and the player switches up late. Both need the creator page to say so honestly.
Origin egress on a cache miss stormSymptom — A new popular video: the first viewers in every region miss the edge at once, origin egress spikes, and object storage throttles or the bill does.Fix — Origin shielding — one regional cache layer between the edges and storage — so a segment is fetched from the origin once per region, not once per edge.Another cache tier to operate and pay for, and one more place a stale object can live when a segment must be replaced.
The catalogue database on the watch pageSymptom — Every session start reads the video row, the creator row, the renditions and the progress; at 25 requests a second it is fine and at a 100× event it is the only relational store in the path and it is what falls over.Fix — Cache the watch page's metadata by video id with a TTL, invalidated on the rare edit, and keep progress in the key-value store where it already is.A creator's title edit is visible after the TTL, not instantly, unless the edit path invalidates — which is a second code path to keep correct.

When something fails

A transcoding worker dies mid-renditionThe job's visibility timeout expires and another worker picks it up. It re-encodes the rendition from the source into new segment keys and marks the rendition ready with a conditional update; the half-written segments from the dead worker are orphans under the old keys and are removed by a lifecycle rule. Nothing the player can reach was ever half-written.
The CDN cannot reach the originEdges keep serving what they have cached — popular videos continue — and miss requests fail. Players fall back to a lower rendition that is cached, then stall on segments no edge has. The service's own API is unaffected, which is how you know it is the origin and not you.
A creator uploads a corrupt fileThe first rendition job fails validation in the decoder and marks the video FAILED with a reason the creator can read. The other four jobs for the same video see FAILED and exit. No retries: the input will not improve.
The progress store is lostEvery viewer starts from the beginning of the next video they open. Annoying, not data loss; the catalogue and the videos are intact, and the store refills as people watch.

Scaling it

Each step is triggered by a number, not a feeling — and carries what it costs.

10× uploadsMove — Nothing in the design changes: the queue absorbs it and the worker fleet scales on depth. What changes is the transcoding bill, which scales linearly with minutes uploaded.Money, and a decision about whether to transcode the higher renditions lazily on first view for the long tail, which reintroduces the first-viewer latency for those videos only.
Live streamingMove — The same segment format with a manifest that grows every few seconds, a transcoder that runs in real time per stream, and a much shorter CDN TTL on the manifest.Latency of at least a few segments (12 seconds at 4-second segments) unless you shorten segments and pay in overhead; and a transcoder per live stream that cannot be queued.
A global audienceMove — Multi-region object storage replication so each region's CDN has a nearby origin, and regional transcoding so the source is not shipped across the world.Storage cost per region, and a replication lag during which a video is READY in one region and 404 in another — the manifest builder must know where the segments actually are.

What gets probed

The design is the easy half. These are where the conversation goes, and each has a defensible answer above.

  • A 3 GB upload fails at 90%. What does the client resend, and what does your service have to do about it?
  • Trace a segment request from a player in a city you have no servers in. Which components does it touch, and which of them are yours?
  • Why is a segment's URL immutable, and what happens to viewers when a rendition must be re-encoded?
  • The transcoding queue is 8 hours deep. What do you change first, and what does the creator see meanwhile?
  • View counts on the watch page disagree with the CDN's request counts by 40%. Which is right, and why?