Protobuf Schema Evolution Without Breaking Clients

Protobuf schema evolution has a small set of rules, and following them is the difference between shipping a new field and triggering a multi-service outage. Keep field numbers stable forever, reserve anything you remove, and add rather than mutate. Do that, and old and new clients keep talking through every deploy.

Protobuf is built for backward and forward compatibility, but only if you respect how it encodes data. The wire format identifies fields by number, not name, so the field number is the real contract. Break that and a “harmless” schema change silently corrupts data on clients you forgot existed.

Why Protobuf schema evolution matters

In a polyglot system, one .proto file generates clients in Go, Java, Python, TypeScript, and more. A single schema change ripples to every one of them, and they do not all deploy at the same instant.

That means you always run old and new code against the same schema during a rollout. If a change is not backward and forward compatible, the window between the first and last deploy is an outage waiting to happen. This post is part of the Language choices in polyglot microservices series, and it builds on the cross-language failures in Why Language Boundaries Break Polyglot Microservices.

The field number is the contract

Protobuf serializes each field as a tag (the field number plus wire type) followed by the value. Names are a source-code convenience that never travels on the wire in binary encoding.

This single fact explains every rule that follows. The decoder on the other side matches incoming bytes to fields by number. If your number means one thing to the sender and another to the receiver, you do not get an error. You get the wrong value, parsed confidently, with no exception to alert anyone.

That silent-corruption property is why schema discipline matters more than it seems. A type error would at least crash loudly. A field-number collision just quietly hands service B the wrong data.

What changes break Protobuf backward compatibility?

The breaking changes are reusing or changing a field number, changing a field’s type, and removing a field without reserving its number. Each one makes new code misread bytes that old code wrote, or vice versa, with no error raised. Treat all three as forbidden in a live system.

Here is the cheat sheet I keep next to any .proto review.

Change	Safe?	Why
Add a field with a new number	✅ Safe	Old clients ignore it; new clients default it
Remove a field, then reserve its number	✅ Safe	No one can recycle the number later
Rename a field (same number)	✅ Binary-safe	Binary uses numbers; JSON/text mapping does change
Change a field’s type	❌ Breaking	Decoders misread the bytes
Reuse an old field number for a new field	❌ Breaking	New code misreads old serialized data
Remove a field without reserving	⚠️ Risky	A future edit can recycle the number
Change field number of an existing field	❌ Breaking	It is a different field on the wire
Move a field in/out of a oneof	❌ Breaking	Changes wire semantics

Can I safely add fields to a Protobuf message?

Yes. Adding a field with a brand-new, never-used field number is the safest change you can make. Old clients silently ignore fields they don’t recognize, and new clients see the default value for fields that old servers don’t send. This is the additive path every safe evolution uses.

The discipline is to make every change additive. Need to “change” a field? Add a new one with a new number, migrate readers to it, and retire the old one later by reserving it. You never edit an existing field in place; you deprecate and add alongside.

message User {
  string id = 1;
  string email = 2;
  // Deprecated: use full_name (5). Kept for old clients.
  string name = 3 [deprecated = true];

  reserved 4;              // a field we removed; never reuse 4
  reserved "legacy_flag";  // and its old name

  string full_name = 5;    // the additive replacement
}

Why should I reserve removed field numbers in Protobuf?

Because reserved stops a future engineer from recycling the number or name for an unrelated field. If number 4 once held a timestamp and someone later reuses 4 for a string, new code reading old data interprets timestamp bytes as a string. Reserving turns that silent corruption into a compile-time error.

Reserve both the number and the name. Reserving the number protects the binary wire format; reserving the name protects JSON and text encodings and prevents accidental source-level reuse. It costs one line and removes an entire class of 2 a.m. incidents.

How do I handle unknown enum values across languages?

Reserve the zero value as an explicit UNKNOWN and handle that case on every client. When a newer server sends an enum value an older client was never compiled with, each language surfaces the “unknown” differently, so an explicit unknown branch is what stops a silent misroute.

enum OrderState {
  ORDER_STATE_UNKNOWN = 0;   // zero value: always a safe default
  ORDER_STATE_PENDING = 1;
  ORDER_STATE_SHIPPED = 2;
  ORDER_STATE_DELIVERED = 3;
}

Proto3 also collapses the distinction between an absent field, a zero field, and an explicitly-set-to-zero field unless you opt into field presence. When that distinction matters to your logic, use explicit presence (optional in proto3) so a missing value and a real zero are not confused across runtimes.

Should you version your Protobuf API?

Prefer in-place additive evolution over hard version bumps. Because Protobuf changes can almost always be made backward compatible, you rarely need a v2 package; you add fields and deprecate old ones within the same message. Reserve a new versioned package for a genuine, incompatible redesign you cannot reach additively.

The reason matters. A v2 package means generating, deploying, and maintaining two full sets of clients and servers, plus a migration window where both run. That is a large, recurring cost. In-place evolution avoids it entirely as long as you obey the field-number rules: old clients keep working against the new schema, and you migrate readers to new fields at your own pace.

When you genuinely do need a breaking redesign, a new package version (mypackage.v2) run alongside v1 is the clean path: stand up the new contract, migrate consumers one at a time, and retire v1 once traffic drains. The key is that this is the exception you reach for deliberately, not the default cadence for every change.

A schema-change review checklist

Run this before merging any .proto change.

• No existing field number is changed, reused, or retyped.
• Every removed field has a reserved number and name.
• New fields use new numbers and have sensible defaults.
• Every enum has an explicit zero UNKNOWN, handled on all clients.
• Field presence is explicit wherever absent-versus-zero matters.
• A round-trip test encodes with the old schema and decodes with the new one, and vice versa.

That last point is the one teams skip. A compatibility test that serializes with version N and deserializes with version N+1 (and the reverse) catches the breakage your code review missed.

How do you enforce compatibility automatically?

Every rule in this post is a rule a human can forget, and schema mistakes are uniquely expensive because they are discovered by clients in production rather than by tests. The whole class becomes preventable with a CI check.

Breaking-change detection is the highest-value piece. Tooling such as buf breaking compares a proto change against a baseline — the previous commit, or the version currently deployed — and fails the build on a reused field number, a changed type, a removed field that was not reserved, or a renamed enum value. It is a few lines of CI configuration and it eliminates the entire category mechanically.

Three supporting practices make it hold:

• Lint the schemas. Consistent naming, required file structure, and a ban on the constructs your organisation has decided against. A linter is how conventions survive their author.
• Generate clients from one source in CI. If each consumer generates its own stubs from its own copy of a proto, versions drift and someone is always compiling against a schema nobody else has. Generate centrally, publish versioned artefacts.
• Keep a schema registry or a versioned artefact history. You need to know which schema version is deployed where in order to reason about compatibility at all, and to answer “can we remove this field yet” with evidence rather than optimism.

That last question is the one automation answers best. A field is safe to remove when no deployed client still reads it — which is a fact about your fleet, not a judgement call. Without tracking, teams either remove fields optimistically and break a client, or never remove anything and the schema accretes forever. With it, deprecation becomes a routine, scheduled activity.

The general principle worth extracting: a contract that is only enforced by convention is not enforced. Field numbers, reservations, and enum defaults are exactly the kind of detail that is obvious to whoever wrote the schema and invisible to whoever edits it eighteen months later, which is precisely what a build gate is for.

What does versioning actually cost, and when is it worth it?

Adding v2 beside v1 sounds clean and is not free, so it is worth being precise about when the cost is justified.

Compatible evolution — adding fields, reserving removals — costs nothing beyond discipline, and it handles the large majority of changes. Reach for a new version only when a change cannot be made compatibly: a field’s meaning changes, a required relationship is restructured, or the message’s semantics are genuinely different.

When you do version, the cost is real: two schemas to maintain, two code paths in the server, twice the tests, and a migration to run to completion before you can delete the old one. The trap is starting a v2 and never finishing the migration, leaving both alive indefinitely — which is worse than either alternative, because every future change must now be made twice.

Two practices keep versioning honest. Make the version structural, so v2 lives in its own directory beside v1 rather than mutating it in place; this makes the two independently readable and lets clients migrate on their own schedule. And set the deprecation date when you create the version, not later. A v1 with an announced end-of-life gets migrated; a v1 with a vague intention to retire it becomes permanent, and you have doubled your maintenance surface forever in exchange for one change you could not make compatibly.

How do you deploy a schema change safely?

Compatibility rules tell you what is safe to write. Deployment order determines whether it is safe to ship, and getting it wrong breaks clients even when the schema change itself was perfectly compatible.

The asymmetry to internalise: readers must be updated before writers. A consumer that does not know about a new field handles it gracefully — Protobuf preserves unknown fields — but a consumer that expects a field the producer is not yet sending will fail. So:

1. Update the schema and publish the new generated artefacts.
2. Deploy consumers first, able to handle both the old and new shape.
3. Deploy producers, now emitting the new shape.
4. Wait, then remove the old handling once no producer emits the old form.

For a removal, the order reverses: stop producing the field, wait until no consumer depends on it, then remove and reserve it.

Two details that matter in practice. Rolling deploys mean both versions run simultaneously, so the change must be safe in both directions during the window — not just after it completes. And a rollback must also be safe: if you deploy a producer emitting a new required-in-practice field and then roll back the consumers, you need the old consumers to still work. Designing for rollback is what makes schema changes routine rather than one-way doors.

The habit that makes this manageable: never combine a schema change with a behaviour change in one deploy. Ship the schema change alone, verify it, then ship the logic that uses it. Combined changes are how a compatible schema edit becomes an incident, because the failure could be either half and you cannot roll back one without the other.

What are the subtle compatibility traps?

Beyond the well-known rules, a few behaviours surprise people because the schema looks unchanged.

Changing a field from optional to repeated, or the reverse. Wire-compatible in some cases and semantically different in all of them. A consumer expecting one value receiving a list, or vice versa, behaves unpredictably rather than failing cleanly.

Renaming a field. Wire-compatible, since the number is the contract — and it breaks anything using JSON serialisation, where the name is the contract. If your protos are also served as JSON over a gateway, field names are part of your public API and renaming one is a breaking change.

Changing a default. Proto3 scalar fields have no explicit presence by default, so a zero value and an unset field are indistinguishable. Code that changes what a zero means is a breaking change with no schema diff at all.

Adding a value to an enum. Safe only if every consumer handles unknown values correctly. It is a compatible schema change that breaks consumers written without a default case, which is why the reserved UNKNOWN zero value and defensive handling matter so much.

Moving a message between files or packages. Changes the fully-qualified type name, which matters for anything doing dynamic resolution, reflection, or Any fields, even though the wire format for the message itself is unchanged.

The pattern across all five: wire compatibility is not the same as semantic compatibility. The tooling checks the former. The latter needs a human asking what the consumer will do with the change, which is the part of schema review that cannot be automated.

What should a schema review actually check?

Schema changes deserve more scrutiny than ordinary code, because a mistake ships to every consumer and cannot be quietly fixed in a follow-up. A short, specific review list catches nearly everything.

• Is every new field a new number? Never reused, never renumbered.
• Is every removed field reserved, by both number and name?
• Does any existing field change type? Even a “compatible” widening deserves scrutiny.
• Does any enum lack an UNKNOWN zero value, or does the change add a value consumers might not handle?
• Is presence handled deliberately where absent, zero, and null must be distinguishable?
• Are large identifiers strings rather than int64, given JavaScript consumers?
• Does the deploy order work, and is a rollback safe?
• Is this change compatible, or does it need a new version?

The reviewer question that catches the most: “what does an old client do when it receives this?” Answering it explicitly forces the compatibility analysis that the rules are shorthand for, and it surfaces the semantic breakages the tooling cannot see.

Two organisational practices support the list. Require a second reviewer for schema changes specifically, since the author is the person least likely to notice an implicit assumption. And keep the schemas in one repository with clear ownership rather than scattered beside each service — a contract owned by everyone is reviewed by no one, and the whole value of the schema is that it is a single agreed definition.

The framing worth leaving with: a Protobuf schema is a public API with a very long tail of consumers, some of which you cannot upgrade. Mobile clients in particular may run versions from years ago. Every schema decision should be made with the assumption that some consumer will still be reading it long after everyone has forgotten why the field exists.

One consequence of that long tail is worth planning for explicitly: decide your support window and write it down. “We support clients up to eighteen months old” turns every deprecation question into arithmetic instead of a negotiation, and it gives mobile teams a deadline they can plan releases around. Without a stated window, fields are never removed because nobody can prove they are unused, and the schema grows monotonically for the life of the product.

Pair the window with telemetry on which schema versions are actually in use, so the deadline is enforced against evidence rather than assumption.

Most gateways and client SDKs can report a version header cheaply, and that single field turns every future deprecation from a debate into a query you can run.

Without it, deprecation decisions default to the most cautious voice in the room, and nothing is ever removed.

Evidence beats caution here, and it is cheap to collect.

What I’d do differently

The lesson I learned the slow way is that schema review cannot be vibes. “It’s just a small change” is exactly how field numbers get reused. The fix is to make the rules mechanical: a linter or buf check in CI that rejects field-number changes and unreserved removals, so the discipline does not depend on a tired reviewer at the end of a sprint.

If you treat the .proto as a versioned, owned interface with automated compatibility checks, schema evolution becomes boring, which is exactly what you want from the contract that ties your whole system together. For how those contracts then behave at runtime across languages, see gRPC Across Languages: Production Lessons.

Sources

• Protocol Buffers, Updating a message type: protobuf.dev/programming-guides/proto3/#updating
• Protocol Buffers, Field presence: protobuf.dev/programming-guides/field_presence
• Buf, Breaking change detection: buf.build/docs/breaking/overview