Protocol Buffers (Protobuf) is a schema-driven format for serializing structured data.
Developed by Google for efficient data exchange, it provides a language-neutral way to define durable contracts and encode application data into compact binary payloads.
How it works
Protobuf works by combining a pre-defined schema with your data to produce a compact binary payload. Unlike JSON, which repeats field names in every object, Protobuf identifies fields by numeric IDs from the schema. That is the core tradeoff: less self-description in each payload, more value from a shared contract.
"Protobuf" refers to both an Interface Definition Language (IDL) and a high-performance Wire Format. While the machine-optimized binary encoding is the primary target, the ecosystem also defines standardized mappings for human-readable representations and diagnostic tools. Explore how a single User message can be represented across these different specifications:
The Schema (.proto)
The Compilation Pipeline
How your human-readable schema becomes high-performance code.
Compilation translates your language-neutral schema into high-performance source code for your specific language. This generated code handles all the complexity of bit-packing and validation.
The compiler itself only parses schemas — plugins do the generating, one per target language. Historically each developer installed those plugin binaries locally and kept versions in sync by hand. Remote plugins let buf generate run them from the registry instead, so everyone gets identical output from a checkout and a single command.
Most teams do not hand-write serializers. They generate code from .proto files. The generated code provides typed message constructors, binary serialization, JSON mapping, and service bindings depending on the plugin.
edition = "2023"; package demo.v1; option go_package = "github.com/sudorandom/protobuf.kmcd.dev/gen/go/demo/v1;demov1"; import "buf/validate/validate.proto"; message User { string id = 1 [(buf.validate.field).string.uuid = true]; string name = 2; string email = 3; uint32 age = 4 [(buf.validate.field).uint32.lt = 150]; float height_cm = 5; double weight_kg = 6; Role role = 7; Date birth_date = 8; User manager = 9; enum Role { ROLE_UNSPECIFIED = 0; ROLE_USER = 1; ROLE_ADMIN = 2; } } message Date { int32 year = 1; int32 month = 2; int32 day = 3; }
From Contract to Runtime API
This schema defines a User message with three fields. Each field has a type, a generated-code name, and a stable field number used by the binary format.
Once code is generated from this schema, you can:
- Instantiate: Create
Userobjects in your language with type checking and editor support. - Serialize: Convert objects into compact binary buffers for transmission or storage.
- Validate: Apply schema and business rules before data reaches application logic.
Generating Code with buf (Recommended)
buf keeps generation declarative with a buf.gen.yaml file, making the workflow reproducible and easier to share across a team without needing local plugin binaries installed manually.
version: v2 plugins: - local: protoc-gen-es out: web/src/gen opt: target=ts - local: protoc-gen-go out: gen/go opt: paths=source_relative
$ buf generateRuns multi-language schema generation in one command.
Using the Generated Code
With code generated by protobuf-es, the schema becomes a native TypeScript API.
import { create, toBinary, toJsonString } from "@bufbuild/protobuf"; import { UserSchema } from "./gen/demo/v1/user_pb"; const user = create(UserSchema, { id: "usr_123", name: "cyber_ninja", email: "ninja@example.com", age: 28, }); const bytes = toBinary(UserSchema, user); const json = toJsonString(UserSchema, user); console.log("JSON Output:", json); console.log("Binary Output:", bytes);