How to Design a URL Shortener (TinyURL / bit.ly)

1. Requirements Clarification

Before jumping into architecture, nail down requirements. Interviewers want to see you drive this conversation.

Functional Requirements

• Given a long URL, generate a unique short URL (e.g. https://fb.in/aB3x9Z)
• Redirect users from short URL to original long URL
• Support custom aliases (e.g. fb.in/my-product-launch)
• URL expiry — optional TTL per link
• Click analytics — count clicks, geographic breakdown, referrer
• User accounts — link ownership and management

Non-Functional Requirements

• Scale: 100 million URLs created per day, 10:1 read-to-write ratio = 1 billion redirects/day
• Redirect latency: <10ms at P99
• High availability: 99.99% uptime (52 minutes downtime/year)
• Storage: URLs kept for 5 years; ~500 bytes per record → 100M × 365 × 5 × 500B ≈ 91 TB
• Eventual consistency acceptable for analytics; strong consistency required for URL creation (no duplicate short codes)

2. High-Level Architecture

  Client
    │
    ▼
┌─────────────┐     ┌─────────────────────────────────────────┐
│   CDN Edge  │────▶│         Load Balancer (L7)              │
│ (CloudFront)│     └────────────────┬────────────────────────┘
└─────────────┘                      │
                          ┌──────────┴──────────┐
                          │                     │
                   ┌──────▼──────┐     ┌────────▼───────┐
                   │  Redirect   │     │  Write Service  │
                   │  Service    │     │  (URL Creation) │
                   │  (read-only)│     └────────┬───────┘
                   └──────┬──────┘              │
                          │              ┌───────▼──────┐
                   ┌──────▼──────┐       │  ID Generator│
                   │  Redis Cache│       │  (Snowflake) │
                   │  (hot URLs) │       └───────┬──────┘
                   └──────┬──────┘              │
                          │              ┌───────▼──────┐
                   ┌──────▼──────────────▼──────────────┐
                   │           MySQL / PostgreSQL        │
                   │         (Primary + Read Replicas)   │
                   └────────────────────────────────────┘
                                    │
                             ┌──────▼──────┐
                             │    Kafka    │  ← async click events
                             └──────┬──────┘
                                    │
                             ┌──────▼──────┐
                             │  Analytics  │
                             │  Pipeline   │
                             └─────────────┘

3. ID Generation Strategies

The short code is the heart of the system. Getting ID generation right determines scalability and correctness.

Base62 Encoding Explained

Base62 uses digits 0–9, uppercase A–Z, and lowercase a–z — 62 characters total. It produces URL-safe codes without special characters. Here is the encoding algorithm:

4. Database Schema

The schema is intentionally simple. The urls table is the single source of truth. We optimise read performance through indexing and caching rather than schema complexity.

5. Redirect Flow — Cache-First

The redirect path must be as fast as possible. Every millisecond adds up when you serve 11,500 redirects per second. The cache-first pattern keeps database load near zero for popular URLs.

User clicks short URL
        │
        ▼
  CDN Edge Cache?  ──YES──▶  301 Redirect (cached at edge, ~2ms)
        │NO
        ▼
  Redirect Server
        │
        ▼
  Redis Cache?  ──YES──▶  302 Redirect + async Kafka event (~5ms)
        │NO
        ▼
  Database Read  ──FOUND──▶  Populate Redis + Redirect (~15ms)
        │NOT FOUND
        ▼
  Return 404

Redis key structure: url:{short_code} → JSON with long_url, expires_at, user_id. TTL on Redis key matches URL expiry, or defaults to 24 hours for non-expiring URLs (with lazy refresh on access). A single Redis node handles 100,000+ ops/second — far above our 11,500 redirects/second requirement.

6. Write Path — URL Creation

URL creation is less frequent than reads (10:1 ratio) but requires strong consistency — two concurrent requests must not get the same short code.

1. Client sends POST /api/shorten with long_url, optional custom_alias, optional expires_at
2. Write Service validates the URL (regex + optional reachability check)
3. If custom alias: check DB for availability, return 409 if taken
4. Otherwise: request next ID from ID Generator (Snowflake service or DB auto-increment)
5. Encode ID to base62 to get short_code
6. Insert row into urls table (UNIQUE constraint on short_code prevents races)
7. Return short URL to client

7. Analytics Pipeline

Analytics must never slow down the redirect. The pattern is fire-and-forget asynchronous logging.

Redirect Server
    │
    ├──▶  302 Redirect (synchronous, <5ms)
    │
    └──▶  Kafka Producer (async, non-blocking)
              topic: click-events
              key: short_code (partitioned for ordering)
              value: { short_code, timestamp, ip, user_agent, referrer }
                   │
                   ▼
         Flink / Spark Streaming
         (enriches IP → country, aggregates counts)
                   │
          ┌────────┴────────┐
          ▼                 ▼
    ClickHouse           Redis Counters
  (analytics queries)   (real-time counts)

8. Scaling Considerations

At 1 billion redirects per day (~11,500 rps), a single server is a bottleneck. Here is how to scale each layer:

9. URL Expiry

URLs with an expires_at timestamp must return a 410 Gone response after expiry. Implement this at two levels:

• At redirect time: Check expires_at in the cached URL object. If expired, return 410 and delete from cache.
• Background cleanup job: Nightly cron deletes expired rows from the DB (DELETE FROM urls WHERE expires_at < NOW() AND expires_at IS NOT NULL LIMIT 10000). Use batched deletes to avoid locking the table.

10. Trade-offs and Design Decisions