URL Shortener
Turn long URLs into short, unique links and redirect billions of reads with sub-100ms latency.
By Ratish Jain · Updated
Problem
A URL shortener takes a long URL like https://example.com/very/long/path?ref=123 and returns a short alias such as short.ly/aB9dK2. Visiting the short link redirects (HTTP 301/302) to the original. The hard part isn't shortening — it's serving billions of redirects cheaply, with very low latency, while guaranteeing every generated key is unique.
Functional Requirements
- Given a long URL, return a unique short URL.
- Given a short URL, redirect to the original long URL.
- Optional custom aliases and link expiration.
- Basic analytics: click counts per link.
Non-Functional Requirements
- Low latency redirects (< 100ms) — this is the hot path.
- High availability — a dead redirect breaks every link ever shared.
- Read-heavy: reads outnumber writes ~100:1.
- Short keys must be unique and hard to guess sequentially.
Capacity Estimation
- New URLs / month
- 500M writes
- Read : Write ratio
- ~100 : 1 (50B reads/mo)
- Read QPS (avg)
- ~20,000/s
- Storage (5 yr)
- 500M × 60 × ~500B ≈ 15 TB
- Key length
- 7 Base62 chars ≈ 3.5T combos
- Cache working set
- ~20% hot links → fits in RAM
APIs
- POST /shorten
{ longUrl, alias? }→{ shortUrl }- GET /:key
- →
301redirect to longUrl - GET /:key/stats
- →
{ clicks, createdAt }
Components
Load Balancer spreads traffic across stateless API nodes. The URL Service handles both shorten and redirect. The Key Generation Service hands out unique IDs (a distributed counter encoded to Base62) so services never collide. Redis caches the key → longUrl mapping for the hot read path, and a sharded database is the source of truth.
Redirect Request Flow
- 1Client requests
short.ly/aB9dK2. - 2Load balancer routes to any API node (nodes are stateless).
- 3Service checks Redis for the key — a cache hit returns the long URL immediately.
- 4On a miss, read from the database, then populate the cache (cache-aside).
- 5Return a
301redirect and asynchronously increment the click counter.
Flow Diagram
Key Generation & Database Design
To avoid collisions, generate a globally unique 64-bit integer per URL and encode it to a 7-character Base62 string — no coordination or 'is this taken?' check needed. Store shortKey (PK) → longUrl, createdAt, expiresAt, ownerId. The table is a giant key-value map, so it shards cleanly by hashing the short key. NoSQL (e.g. DynamoDB/Cassandra) fits well: no joins, simple access pattern, horizontal scale.
Data Model
Why Base62, not a hash?
Hashing the long URL (MD5) risks collisions and makes custom expiry/ownership awkward. A counter encoded to Base62 ([A-Za-z0-9]) is guaranteed unique and compact — 62^7 ≈ 3.5 trillion keys.
Cache Strategy
Use cache-aside with Redis: on read, check cache → DB → backfill. Because links follow a power-law (a few links get most clicks), caching the hot 20% serves the vast majority of reads from memory. Set a TTL and evict with LRU so cold links don't waste RAM.
Scaling Strategy
API nodes are stateless → scale horizontally behind the LB. The database shards by short key using consistent hashing. Redis scales via a cluster or read replicas. Put static assets and even cached redirects behind a CDN / edge to cut latency globally.
Bottlenecks & Failure Scenarios
- Key generation is a hot spot — batch out ranges of IDs to each node to avoid a single-counter bottleneck.
- Redis outage → thundering herd on the DB; mitigate with request coalescing and DB read replicas.
- Hot key (a viral link) → replicate that key across cache nodes or serve from CDN edge.
- Analytics writes must never block redirects — fire them async via a queue.
Design Tradeoffs
- Stateless services scale trivially.
- Cache-aside keeps the hot path in memory.
- Counter + Base62 = zero-coordination unique keys.
- 301 (permanent) caching helps latency but breaks click analytics; 302 fixes analytics but adds load.
- Distributed counter needs careful range allocation.
- Custom aliases require a uniqueness check the counter path avoids.
Common Interview Questions
302 (temporary) if you need accurate click analytics, since browsers re-hit your server each time. 301 (permanent) if latency/load matters more, because browsers cache it and skip your server on repeat visits.
Generate a unique 64-bit number (distributed counter with pre-allocated ranges, or a Snowflake-style ID) and Base62-encode it. Uniqueness of the number guarantees uniqueness of the key.
Attempt an insert with a uniqueness constraint on the alias; on conflict, reject and ask the user for another. This is the one path that needs a read-before-write.
- →Read-heavy (100:1) → cache-aside with Redis on the hot path.
- →Unique keys = distributed counter → Base62 (7 chars ≈ 3.5T).
- →Stateless API + sharded KV DB → scale horizontally.
- →301 caches (fast) vs 302 (analytics) — know the tradeoff.
- →Async analytics via queue; never block the redirect.
Related
Everything on Archprimer is connected — follow the graph.