Serialization, Content Negotiation & Compression

A CPU-versus-bytes trade, decided by your bottleneck

Serialization and compression are a CPU-versus-bytes trade, and the right answer depends entirely on where your bottleneck is: bandwidth (mobile, cross-region, metered) or CPU (very high QPS internal traffic). You also have to keep schemas evolvable when producers and consumers deploy independently.

Serialization formats

• JSON: human-readable, self-describing, universal, debuggable in a browser. But verbose (field names repeat on every object) and comparatively slow to parse. It is the correct default for public APIs where developer ergonomics and debuggability beat raw efficiency.
• Protobuf: compact binary driven by an IDL, fast to encode/decode, with generated types. Fields are tagged by number, not name, so payloads are small. Ideal for internal high-QPS RPC (it pairs with gRPC).
• Avro: the schema is registered centrally or travels with the data (in the file header), which makes it strong for data pipelines and Kafka, where a schema registry lets producers and consumers evolve independently.
• Thrift: RPC plus serialization from one IDL, similar niche to Protobuf, common in older Facebook-lineage stacks.

Compression, negotiated via Accept-Encoding

• gzip: universal and cheap, the safe default.
• Brotli: better ratio than gzip, especially on text over HTTPS to browsers.
• zstd: excellent ratio and speed with tunable levels, great for internal transfer where you control both ends.

The tradeoff to state explicitly: compression and binary encoding cut bytes but add CPU, and aggressive compression can add tail latency on large responses (the compressor has to run before the first byte goes out). So set a payload-size threshold below which you do not compress (compressing a 200-byte response is a net loss), and do not double-compress already-compressed data (images, video).

Schema evolution: the part people forget

Protobuf, Avro, and Thrift all support forward and backward compatibility if you follow the rules: add only optional/new fields, and never reuse or renumber a field tag (mark removed tags reserved). That is what lets a new producer and an old consumer coexist during a rolling deploy. JSON has no built-in schema, so it relies on the tolerant-reader discipline: consumers ignore unknown fields and tolerate missing optional ones.

Putting it together with content negotiation: a public API defaults to JSON, honors Accept-Encoding to pick Brotli for browsers, and sets Vary: Accept-Encoding so a shared cache does not hand a Brotli body to a client that only speaks gzip. An internal mesh uses Protobuf with zstd because both ends are controlled and CPU/bytes dominate.

Interview nuance: the trap is "Protobuf everywhere because it is faster." On a public browser API the network savings are usually tiny relative to the developer and debugging cost, and you lose curl-ability. The senior move is to locate the bottleneck first (bandwidth vs CPU) and choose per surface.

Recap: choose format and codec by bottleneck (JSON+Brotli for public/bandwidth, Protobuf+zstd for internal/CPU), never compress tiny or already-compressed payloads, and keep schemas evolvable by adding optional fields and never reusing field tags.