Storage, Bandwidth & Cache Sizing

Three capacity numbers, three formulas, three classic mistakes

Storage, bandwidth, and cache are the three capacity numbers that decide your datastore, your CDN strategy, and your cache tier. Each has a formula and a classic mistake.

Storage: metadata and blobs are different systems

The base formula is:

storage = objects/day  x  object size  x  retention (days)

The critical discipline is to separate metadata from blobs. A photo is 2 MB of blob but only ~1 KB of metadata (id, owner, timestamps, storage key, caption). These belong in different systems: blobs in an object store (S3), metadata in a sharded database. Estimating them together hides the fact that your database only needs to hold gigabytes while your object store holds petabytes.

Two multipliers people forget, both of which change the answer materially:

• Replication factor. Durable stores keep 3 copies (RF=3), so multiply raw storage by 3. Erasure coding can bring this down to ~1.3x to 1.5x for cold blobs, a real cost lever worth naming.
• Index and overhead. Secondary indexes, B-tree overhead, and free space commonly add 20 to 50% on top of raw row size for databases.

Bandwidth: ingress and egress separately

bandwidth = QPS  x  payload size
ingress = write QPS x write payload      (upload path)
egress  = read QPS  x read payload       (download/serve path)

Egress is usually the larger and more expensive number, and in cloud pricing egress leaves your provider's network at real dollar cost. A read-heavy media service serving 2 MB objects at 250k QPS is pushing 500 GB/s of egress, which is a "you must use a CDN" signal, not a "size your app servers" signal, because the CDN serves it from the edge and shields the origin.

Cache sizing with the 80/20 rule

You do not cache the whole corpus; you cache the hot working set. The Pareto assumption is that ~20% of data serves ~80% of requests, and often it is far more skewed (the recent and the viral). Size the cache from that hot fraction:

cache size ~= hot fraction (~20%) of the actively-read dataset

For a service with a 4.5 TB actively-read dataset, an 80/20 cut suggests roughly 900 GB of hot data, but in practice the truly hot set is the last few days plus trending items, often a much smaller absolute number like tens to low hundreds of GB. Verify the hit rate assumption: if 100 GB of cache yields a 90%+ hit rate, you have removed 90% of read load from the datastore, which is what justifies the cache economically.

Interview nuance: interviewers probe "how big is your cache and why that size." The winning answer ties cache size to a target hit rate and to the read load removed from the origin, not to a fraction of total storage pulled from thin air.

raw payload -> x replication -> + index/overhead = provisioned storage
hot 20% of reads -> cache size -> target hit rate -> read load removed

Recap: size storage as objects x size x retention with metadata and blobs kept separate, multiply by replication factor and add index overhead, compute ingress and egress bandwidth separately (egress drives CDN), and size the cache from the hot ~20% against a target hit rate.