gRPC and Protocol Buffers: Typed APIs Across Languages

Overview

I built a service-to-service API in REST/JSON for years, and it worked fine, but there was always a class of bugs I could never quite eliminate. The Go service sent a field called user_id, the Python client expected userId, and the mismatch only surfaced when someone hit that code path. Every JSON API I've ever built has had at least one of these.

gRPC and Protocol Buffers solve this at the schema level. You define the API in a .proto file, generate client and server code in every language you need, and the compiler catches every mismatch before runtime.

What gRPC buys you

The schema requirement is what makes the difference. Every field has a number, every type is explicit, and the compiler enforces compatibility across services. It's more ceremony than REST, and it's worth it once you have more than one team involved.

Defining your first service

// proto/user.proto
syntax = "proto3";

package user.v1;

option go_package = "GitHub.com/mycompany/api/gen/user/v1;userv1";

message User {
    string id = 1;
    string email = 2;
    string name = 3;
    int64 created_at = 4;
}

message GetUserRequest {
    string id = 1;
}

message GetUserResponse {
    User user = 1;
}

message ListUsersRequest {
    int32 page_size = 1;
    string page_token = 2;
}

message ListUsersResponse {
    repeated User users = 1;
    string next_page_token = 2;
}

service UserService {
    rpc GetUser(GetUserRequest) returns (GetUserResponse);
    rpc ListUsers(ListUsersRequest) returns (ListUsersResponse);
}

Some conventions worth following from day one:

• Number every field. Once released, the number is the identity — the name can change, the number can't. Number 2 in v1 stays number 2 forever.
• Never remove a field number. Mark it reserved instead. Reusing a number with a different type is the most common proto mistake.
• Use v1 in the package. When you need breaking changes, create user.v2 and run them side by side.
• Don't use int32 for timestamps. Use int64 or google.protobuf.Timestamp. The 2038 problem is real.

Code generation

# Install protoc
apt install protobuf-compiler

# Go plugins
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

# Generate
protoc --go_out=. --go-grpc_out=. proto/user.proto

For Python:

pip install grpcio grpcio-tools
python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. proto/user.proto

For TypeScript (using Connect, which works in browsers):

npm install @bufbuild/protoc-gen-es @connectrpc/protoc-gen-connect-es

One thing to internalize early: the generated code is committed to the repository. It's not a build artifact you regenerate on every build. This means PRs show the diff when the schema changes, and reviewers can see the impact.

A Go server

package main

import (
    "context"
    "log"
    "net"

    "google.golang.org/grpc"
    "google.golang.org/grpc/codes"
    "google.golang.org/grpc/status"

    userv1 "github.com/mycompany/api/gen/user/v1"
)

type server struct {
    userv1.UnimplementedUserServiceServer
}

func (s *server) GetUser(ctx context.Context, req *userv1.GetUserRequest) (*userv1.GetUserResponse, error) {
    if req.Id == "" {
        return nil, status.Error(codes.InvalidArgument, "id is required")
    }

    user, err := db.FindUser(ctx, req.Id)
    if err != nil {
        return nil, status.Errorf(codes.Internal, "Database error: %v", err)
    }
    if user == nil {
        return nil, status.Error(codes.NotFound, "user not found")
    }

    return &userv1.GetUserResponse{
        User: &userv1.User{
            Id:        user.ID,
            Email:     user.Email,
            Name:      user.Name,
            CreatedAt: user.CreatedAt.Unix(),
        },
    }, nil
}

func main() {
    lis, err := net.Listen("tcp", ":50051")
    if err != nil {
        log.Fatal(err)
    }

    s := grpc.NewServer()
    userv1.RegisterUserServiceServer(s, &server{})

    log.Println("gRPC server listening on :50051")
    if err := s.Serve(lis); err != nil {
        log.Fatal(err)
    }
}

The status code system is worth internalizing. gRPC has its own codes (NotFound, InvalidArgument, PermissionDenied, Internal, Unavailable) that map cleanly to HTTP status codes when you put a gateway in front. Using them consistently makes client error handling predictable.

A Python client

import grpc
import user_pb2
import user_pb2_grpc

def main():
    with grpc.insecure_channel("localhost:50051") as channel:
        stub = user_pb2_grpc.UserServiceStub(channel)

        try:
            response = stub.GetUser(user_pb2.GetUserRequest(id="user-123"))
            print(f"{response.user.name} <{response.user.email}>")
        except grpc.RpcError as e:
            if e.code() == grpc.StatusCode.NOT_FOUND:
                print("User not found")
            else:
                raise

Note that insecure_channel is for local development only. In production you use secure_channel with TLS credentials, or rely on service mesh mTLS. Plain gRPC over the network without TLS is a mistake I've seen in production more than once.

The four RPC types

service StreamService {
    rpc Tail(LogRequest) returns (stream LogEntry);
    rpc Upload(stream Chunk) returns (UploadResult);
    rpc Chat(stream Message) returns (stream Message);
}

Server streaming is the one I use most. Instead of paginating through 10,000 rows, the server streams them as it reads them and the client processes each one. Memory stays flat on both sides.

Error handling done properly

// Server
return nil, status.Error(codes.InvalidArgument, "email is required")

// Client
st, ok := status.FromError(err)
if !ok {
    // not a gRPC error
}
switch st.Code() {
case codes.NotFound:
    // handle
case codes.PermissionDenied:
    // handle
default:
    // log st.Message()
}

Don't use status.Error to wrap regular errors — it loses type information. Use status.Errorf with a code, or wrap with fmt.Errorf and let the gRPC layer assign codes.Unknown, which is honest about the fact that you didn't categorize the error.

Interceptors: middleware for gRPC

// Unary interceptor for auth
func authInterceptor(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
    md, ok := metadata.FromIncomingContext(ctx)
    if !ok {
        return nil, status.Error(codes.Unauthenticated, "missing metadata")
    }

    tokens := md.Get("authorization")
    if len(tokens) == 0 {
        return nil, status.Error(codes.Unauthenticated, "missing token")
    }

    user, err := verifyToken(tokens[0])
    if err != nil {
        return nil, status.Error(codes.Unauthenticated, "invalid token")
    }

    ctx = context.WithValue(ctx, "user", user)
    return handler(ctx, req)
}

// Register
s := grpc.NewServer(grpc.UnaryInterceptor(authInterceptor))

Interceptors are where you put logging, metrics, auth, and rate limiting. Same pattern as HTTP middleware, different signature.

REST gateway for browsers

gRPC uses HTTP/2 with binary framing, which browsers don't support directly. Two solutions:

I've used Connect more recently. Same proto definitions, same server code, but the JS client works directly in the browser without an Envoy proxy in the middle. The server can accept both gRPC and HTTP/JSON on the same port, which is convenient for gradual migrations.

When gRPC is the wrong choice

• Public APIs consumed by third parties. REST/JSON has a lower barrier. Third-party developers can hit a REST endpoint with curl; they need a code generator for gRPC.
• Small internal services with two endpoints. The proto ceremony isn't worth it.
• Browser-first APIs. The grpc-web workaround adds complexity that REST doesn't have.
• Debugging by hand. If you regularly inspect traffic by eye, the binary payload is a downside. grpcurl helps, but it's not curl.

For anything with multiple services, a shared schema, and cross-language clients, gRPC removes an entire class of bugs. The proto file is the contract, and the compiler enforces it everywhere.

grpcurl: the tool you'll use daily

# List services
grpcurl -plaintext localhost:50051 list

# Describe a service
grpcurl -plaintext localhost:50051 describe user.v1.UserService

# Call a method
grpcurl -plaintext -d '{"id": "user-123"}' \
    localhost:50051 user.v1.UserService/GetUser

grpcurl uses server reflection if it's enabled, which means you don't need the proto files locally. Enable reflection in development but consider disabling it in production — it exposes your entire API surface to anyone who can reach the port.