Protocol Buffers Interview Questions - Medium
Medium-level Protocol Buffers interview questions covering advanced features, schema evolution, and best practices.
Q1: How do you handle schema evolution and backward compatibility?
Answer:
Backward Compatibility Rules:
- Never change field numbers:
1// ❌ BAD: Changing field number breaks compatibility
2message User {
3 string email = 3; // Was field 2, now field 3 - BREAKS!
4}
5
6// ✅ GOOD: Keep field numbers
7message User {
8 string name = 1;
9 int32 age = 2;
10 string email = 3; // New field, new number
11}
- Don't remove fields (mark as reserved):
1message User {
2 string name = 1;
3 // int32 age = 2; // ❌ DON'T DELETE
4
5 // ✅ GOOD: Reserve the number
6 reserved 2;
7 reserved "age";
8
9 string email = 3;
10}
- Add new fields at the end:
1message User {
2 string name = 1; // Existing
3 int32 age = 2; // Existing
4 string email = 3; // Existing
5 string phone = 4; // ✅ New field
6 bool verified = 5; // ✅ New field
7}
- Change types carefully:
1// ❌ BAD: Changing type breaks compatibility
2message User {
3 int32 age = 2; // Was string, now int32
4}
5
6// ✅ GOOD: Use new field
7message User {
8 string age_str = 2; // Old field (deprecated)
9 int32 age = 6; // New field
10}
Versioning Strategy:
1message User {
2 // Version 1 fields
3 string name = 1;
4 int32 age = 2;
5
6 // Version 2: Added
7 string email = 3;
8
9 // Version 3: Added
10 repeated string tags = 4;
11
12 // Version 4: Deprecated old field, added new
13 reserved 2; // Old age field
14 int64 age_v2 = 5; // New age field (int64)
15}
Documentation: Updating A Message Type
Q2: How do you use Any type for dynamic messages?
Answer:
Any Type:
1import "google/protobuf/any.proto";
2
3message Event {
4 int64 timestamp = 1;
5 string event_type = 2;
6 google.protobuf.Any payload = 3; // Can hold any message type
7}
Usage:
1# Python
2from google.protobuf import any_pb2
3import user_pb2
4import order_pb2
5
6# Create event with User payload
7user = user_pb2.User(name="John", age=30)
8event = event_pb2.Event()
9event.timestamp = 1234567890
10event.event_type = "user_created"
11event.payload.Pack(user) # Pack message into Any
12
13# Unpack
14if event.payload.Is(user_pb2.User.DESCRIPTOR):
15 unpacked_user = user_pb2.User()
16 event.payload.Unpack(unpacked_user)
17 print(unpacked_user.name)
1// Go
2import (
3 "google.golang.org/protobuf/types/known/anypb"
4 pb "path/to/generated"
5)
6
7// Pack
8user := &pb.User{Name: "John", Age: 30}
9any, _ := anypb.New(user)
10
11event := &pb.Event{
12 Timestamp: 1234567890,
13 EventType: "user_created",
14 Payload: any,
15}
16
17// Unpack
18if event.Payload.MessageIs(&pb.User{}) {
19 user := &pb.User{}
20 event.Payload.UnmarshalTo(user)
21 fmt.Println(user.Name)
22}
Use Cases:
- Plugin systems
- Event sourcing
- Generic message handlers
- Extensible APIs
Documentation: Any
Q3: How do you use Well-Known Types?
Answer:
Common Well-Known Types:
- Timestamp:
1import "google/protobuf/timestamp.proto";
2
3message Order {
4 int64 id = 1;
5 google.protobuf.Timestamp created_at = 2;
6 google.protobuf.Timestamp updated_at = 3;
7}
1# Python
2from google.protobuf.timestamp_pb2 import Timestamp
3from datetime import datetime
4
5order = order_pb2.Order()
6order.id = 123
7
8# Set timestamp
9now = datetime.now()
10timestamp = Timestamp()
11timestamp.FromDatetime(now)
12order.created_at.CopyFrom(timestamp)
13
14# Get timestamp
15dt = order.created_at.ToDatetime()
- Duration:
1import "google/protobuf/duration.proto";
2
3message Task {
4 string name = 1;
5 google.protobuf.Duration estimated_time = 2;
6}
- Struct (JSON-like):
1import "google/protobuf/struct.proto";
2
3message Config {
4 google.protobuf.Struct settings = 1;
5}
1# Python
2from google.protobuf.struct_pb2 import Struct
3
4config = config_pb2.Config()
5config.settings["key1"] = "value1"
6config.settings["key2"] = 123
7config.settings["key3"] = True
- Empty:
1import "google/protobuf/empty.proto";
2
3service UserService {
4 rpc DeleteUser(DeleteUserRequest) returns (google.protobuf.Empty);
5}
Documentation: Well-Known Types
Q4: How do you define and use services for gRPC?
Answer:
Service Definition:
1syntax = "proto3";
2
3package user;
4
5import "google/protobuf/empty.proto";
6
7service UserService {
8 // Unary RPC
9 rpc GetUser(GetUserRequest) returns (User);
10
11 // Server streaming
12 rpc ListUsers(ListUsersRequest) returns (stream User);
13
14 // Client streaming
15 rpc CreateUsers(stream CreateUserRequest) returns (CreateUsersResponse);
16
17 // Bidirectional streaming
18 rpc ChatUsers(stream ChatMessage) returns (stream ChatMessage);
19}
20
21message GetUserRequest {
22 int64 user_id = 1;
23}
24
25message ListUsersRequest {
26 int32 page = 1;
27 int32 page_size = 2;
28}
29
30message CreateUserRequest {
31 string name = 1;
32 string email = 2;
33}
34
35message CreateUsersResponse {
36 repeated int64 user_ids = 1;
37}
38
39message ChatMessage {
40 int64 user_id = 1;
41 string message = 2;
42}
Server Implementation (Go):
1type server struct {
2 pb.UnimplementedUserServiceServer
3}
4
5func (s *server) GetUser(ctx context.Context, req *pb.GetUserRequest) (*pb.User, error) {
6 // Fetch user
7 user := &pb.User{
8 Id: req.UserId,
9 Name: "John Doe",
10 Email: "john@example.com",
11 }
12 return user, nil
13}
14
15func (s *server) ListUsers(req *pb.ListUsersRequest, stream pb.UserService_ListUsersServer) error {
16 // Stream users
17 for i := 0; i < 10; i++ {
18 user := &pb.User{
19 Id: int64(i),
20 Name: fmt.Sprintf("User %d", i),
21 Email: fmt.Sprintf("user%d@example.com", i),
22 }
23 stream.Send(user)
24 }
25 return nil
26}
Client Usage (Go):
1conn, _ := grpc.Dial("localhost:50051", grpc.WithInsecure())
2client := pb.NewUserServiceClient(conn)
3
4// Unary
5user, _ := client.GetUser(context.Background(), &pb.GetUserRequest{UserId: 123})
6
7// Streaming
8stream, _ := client.ListUsers(context.Background(), &pb.ListUsersRequest{Page: 1})
9for {
10 user, err := stream.Recv()
11 if err == io.EOF {
12 break
13 }
14 fmt.Println(user)
15}
Documentation: gRPC Services
Q5: How do you handle errors in gRPC services?
Answer:
gRPC Status Codes:
1import (
2 "google.golang.org/grpc/codes"
3 "google.golang.org/grpc/status"
4)
5
6func (s *server) GetUser(ctx context.Context, req *pb.GetUserRequest) (*pb.User, error) {
7 if req.UserId <= 0 {
8 return nil, status.Error(codes.InvalidArgument, "user_id must be positive")
9 }
10
11 user, err := s.db.GetUser(req.UserId)
12 if err == ErrNotFound {
13 return nil, status.Error(codes.NotFound, "user not found")
14 }
15 if err != nil {
16 return nil, status.Error(codes.Internal, "internal error")
17 }
18
19 return user, nil
20}
Status Codes:
OK: SuccessINVALID_ARGUMENT: Invalid requestNOT_FOUND: Resource not foundALREADY_EXISTS: Resource already existsPERMISSION_DENIED: Permission deniedUNAUTHENTICATED: Authentication requiredRESOURCE_EXHAUSTED: Rate limitingFAILED_PRECONDITION: Precondition failedABORTED: Operation abortedOUT_OF_RANGE: Out of valid rangeUNIMPLEMENTED: Not implementedINTERNAL: Internal errorUNAVAILABLE: Service unavailableDATA_LOSS: Data loss
Error Details:
1import (
2 "google.golang.org/grpc/codes"
3 "google.golang.org/grpc/status"
4 "google.golang.org/genproto/googleapis/rpc/errdetails"
5)
6
7st := status.New(codes.InvalidArgument, "invalid user_id")
8br := &errdetails.BadRequest{
9 FieldViolations: []*errdetails.BadRequest_FieldViolation{
10 {
11 Field: "user_id",
12 Description: "must be positive",
13 },
14 },
15}
16st, _ = st.WithDetails(br)
17return nil, st.Err()
Documentation: gRPC Status Codes
Q6: How do you use options and custom options?
Answer:
Standard Options:
1import "google/protobuf/descriptor.proto";
2
3message User {
4 string name = 1 [(google.protobuf.field_options) = {
5 deprecated: true
6 }];
7
8 string email = 2;
9
10 int32 age = 3 [(validate.rules).int32 = {
11 gte: 0,
12 lte: 150
13 }];
14}
Service Options:
1service UserService {
2 option (google.api.default_host) = "api.example.com";
3
4 rpc GetUser(GetUserRequest) returns (User) {
5 option (google.api.http) = {
6 get: "/v1/users/{user_id}"
7 };
8 }
9}
Custom Options:
1import "google/protobuf/descriptor.proto";
2
3// Define custom option
4extend google.protobuf.FieldOptions {
5 string my_custom_option = 50000;
6 int32 my_number_option = 50001;
7}
8
9message User {
10 string name = 1 [(my_custom_option) = "custom_value"];
11 int32 age = 2 [(my_number_option) = 42];
12}
Reading Options:
1// Go
2import "google.golang.org/protobuf/reflect/protoreflect"
3
4field := desc.Fields().ByName("name")
5opt := field.Options().(*descriptorpb.FieldOptions)
6customOpt := proto.GetExtension(opt, my_custom_option).(string)
Documentation: Custom Options
Q7: How do you optimize Protocol Buffer performance?
Answer:
1. Use Appropriate Field Types:
1// ❌ BAD: Using int64 for small numbers
2message Order {
3 int64 quantity = 1; // Usually small, wastes space
4}
5
6// ✅ GOOD: Use int32 for small numbers
7message Order {
8 int32 quantity = 1; // More efficient
9}
2. Use Packed Repeated Fields:
1// proto3: Repeated scalar fields are packed by default
2message Data {
3 repeated int32 values = 1; // Automatically packed
4}
5
6// proto2: Explicitly pack
7message Data {
8 repeated int32 values = 1 [packed=true];
9}
3. Use sint32/sint64 for Negative Numbers:
1// ❌ BAD: int32 for negative numbers
2message Temperature {
3 int32 celsius = 1; // Negative values encoded inefficiently
4}
5
6// ✅ GOOD: sint32 for negative numbers
7message Temperature {
8 sint32 celsius = 1; // ZigZag encoding for negatives
9}
4. Avoid Unnecessary Nesting:
1// ❌ BAD: Deep nesting
2message A {
3 message B {
4 message C {
5 string value = 1;
6 }
7 C c = 1;
8 }
9 B b = 1;
10}
11
12// ✅ GOOD: Flatten when possible
13message A {
14 string value = 1; // Direct access
15}
5. Use bytes for Large Binary Data:
1message Image {
2 bytes data = 1; // Efficient for binary
3 string format = 2;
4}
6. Reuse Message Instances:
1// ❌ BAD: Creating new instances
2for i := 0; i < 1000; i++ {
3 user := &pb.User{} // New allocation each time
4 // ...
5}
6
7// ✅ GOOD: Reuse instances
8user := &pb.User{}
9for i := 0; i < 1000; i++ {
10 user.Reset() // Reuse
11 // ...
12}
Documentation: Encoding
Q8: How do you handle large messages and streaming?
Answer:
Problem: Large messages can cause memory issues.
Solution 1: Streaming:
1service DataService {
2 // Stream large dataset
3 rpc GetLargeDataset(GetDatasetRequest) returns (stream DataChunk);
4
5 // Stream upload
6 rpc UploadLargeFile(stream FileChunk) returns (UploadResponse);
7}
Solution 2: Pagination:
1message ListUsersRequest {
2 int32 page = 1;
3 int32 page_size = 2;
4}
5
6message ListUsersResponse {
7 repeated User users = 1;
8 int32 total_count = 2;
9 int32 page = 3;
10 bool has_next = 4;
11}
Solution 3: Chunking:
1message LargeData {
2 int64 total_size = 1;
3 repeated DataChunk chunks = 2;
4}
5
6message DataChunk {
7 int32 chunk_index = 1;
8 bytes data = 2;
9 bool is_last = 3;
10}
Streaming Implementation:
1func (s *server) GetLargeDataset(req *pb.GetDatasetRequest, stream pb.DataService_GetLargeDatasetServer) error {
2 data := s.getLargeData()
3 chunkSize := 1024 * 1024 // 1MB chunks
4
5 for i := 0; i < len(data); i += chunkSize {
6 end := i + chunkSize
7 if end > len(data) {
8 end = len(data)
9 }
10
11 chunk := &pb.DataChunk{
12 ChunkIndex: int32(i / chunkSize),
13 Data: data[i:end],
14 IsLast: end == len(data),
15 }
16
17 if err := stream.Send(chunk); err != nil {
18 return err
19 }
20 }
21
22 return nil
23}
Documentation: gRPC Streaming
Q9: How do you validate Protocol Buffer messages?
Answer:
Using protoc-gen-validate:
1import "validate/validate.proto";
2
3message User {
4 string email = 1 [(validate.rules).string.email = true];
5 int32 age = 2 [(validate.rules).int32 = {
6 gte: 0,
7 lte: 150
8 }];
9 string phone = 3 [(validate.rules).string.pattern = "^\\+?[1-9]\\d{1,14}$"];
10 repeated string tags = 4 [(validate.rules).repeated = {
11 min_items: 1,
12 max_items: 10
13 }];
14}
Validation Rules:
- String:
min_len,max_len,pattern,email,uri - Numbers:
const,lt,lte,gt,gte,in,not_in - Repeated:
min_items,max_items,unique - Maps:
min_pairs,max_pairs - Nested: Validate nested messages
Usage:
1import "github.com/envoyproxy/protoc-gen-validate/validate"
2
3user := &pb.User{
4 Email: "invalid-email",
5 Age: 200,
6}
7
8if err := user.Validate(); err != nil {
9 // Handle validation error
10 fmt.Println(err)
11}
Custom Validation:
1func (u *User) Validate() error {
2 if u.Age < 0 || u.Age > 150 {
3 return fmt.Errorf("age must be between 0 and 150")
4 }
5 if !strings.Contains(u.Email, "@") {
6 return fmt.Errorf("invalid email")
7 }
8 return nil
9}
Documentation: protoc-gen-validate
Q10: How do you use Protocol Buffers with different serialization formats?
Answer:
1. Binary (Default):
1// Most efficient
2data, _ := proto.Marshal(user)
3user := &pb.User{}
4proto.Unmarshal(data, user)
2. JSON:
1import "google.golang.org/protobuf/encoding/protojson"
2
3// To JSON
4jsonData, _ := protojson.Marshal(user)
5
6// From JSON
7user := &pb.User{}
8protojson.Unmarshal(jsonData, user)
3. Text Format:
1import "google.golang.org/protobuf/encoding/prototext"
2
3// To text
4textData, _ := prototext.Marshal(user)
5
6// From text
7user := &pb.User{}
8prototext.Unmarshal(textData, user)
4. Wire Format (for debugging):
1import "google.golang.org/protobuf/encoding/protowire"
2
3// Inspect wire format
4data, _ := proto.Marshal(user)
5fmt.Println(protowire.FormatBytes(data))
Performance Comparison:
- Binary: Fastest, smallest
- JSON: Human-readable, larger
- Text: Human-readable, largest
- Wire: Debugging only
Documentation: Encoding Formats
Q11: How do you handle versioning and multiple proto files?
Answer:
Using buf for Dependency Management:
buf simplifies managing multiple proto files and dependencies:
buf.yaml Configuration:
1version: v1
2name: buf.build/your-org/your-repo
3deps:
4 - buf.build/googleapis/googleapis
5 - buf.build/your-org/common-proto
6modules:
7 - path: proto
8lint:
9 use:
10 - DEFAULT
11breaking:
12 use:
13 - FILE
Import Strategy:
1// common.proto
2syntax = "proto3";
3package common;
4
5message Timestamp {
6 int64 seconds = 1;
7 int32 nanos = 2;
8}
9
10// user.proto
11syntax = "proto3";
12package user;
13
14import "common.proto";
15
16message User {
17 int64 id = 1;
18 string name = 2;
19 common.Timestamp created_at = 3;
20}
Versioning in Package Names:
1// v1/user.proto
2syntax = "proto3";
3package user.v1;
4
5message User {
6 int64 id = 1;
7 string name = 2;
8}
9
10// v2/user.proto
11syntax = "proto3";
12package user.v2;
13
14import "user/v1/user.proto";
15
16message User {
17 int64 id = 1;
18 string name = 2;
19 string email = 3; // New field
20}
With buf:
1# Update dependencies
2buf mod update
3
4# Generate code (handles all imports automatically)
5buf generate
6
7# Check for breaking changes between versions
8buf breaking --against 'buf.build/your-org/your-repo:main'
Legacy: Using protoc with Import Paths:
1# Compile with import paths
2protoc \
3 --proto_path=. \
4 --proto_path=./third_party \
5 --go_out=. \
6 user.proto
Benefits of buf for Versioning:
- Automatic dependency resolution
- Breaking change detection
- Centralized dependency management
- Works with Buf Schema Registry
Documentation:
Q12: How do you use Protocol Buffers with REST APIs?
Answer:
gRPC-Gateway (gRPC to REST):
1import "google/api/annotations.proto";
2
3service UserService {
4 rpc GetUser(GetUserRequest) returns (User) {
5 option (google.api.http) = {
6 get: "/v1/users/{user_id}"
7 };
8 }
9
10 rpc CreateUser(CreateUserRequest) returns (User) {
11 option (google.api.http) = {
12 post: "/v1/users"
13 body: "*"
14 };
15 }
16
17 rpc UpdateUser(UpdateUserRequest) returns (User) {
18 option (google.api.http) = {
19 patch: "/v1/users/{user_id}"
20 body: "user"
21 };
22 }
23}
JSON over HTTP:
1import (
2 "google.golang.org/protobuf/encoding/protojson"
3 "net/http"
4)
5
6func GetUserHandler(w http.ResponseWriter, r *http.Request) {
7 userID := r.URL.Query().Get("user_id")
8
9 user := &pb.User{
10 Id: parseUserID(userID),
11 Name: "John Doe",
12 }
13
14 // Convert to JSON
15 jsonData, _ := protojson.Marshal(user)
16
17 w.Header().Set("Content-Type", "application/json")
18 w.Write(jsonData)
19}
Documentation: gRPC-Gateway
Related Snippets
- Protocol & Design Interview Questions - Easy
Easy-level protocol and design interview questions covering fundamental … - Protocol & Design Interview Questions - Hard
Hard-level protocol and design interview questions covering advanced distributed … - Protocol & Design Interview Questions - Medium
Medium-level protocol and design interview questions covering advanced … - Protocol Buffers Interview Questions - Easy
Easy-level Protocol Buffers interview questions covering basics, syntax, and … - Protocol Buffers Interview Questions - Hard
Hard-level Protocol Buffers interview questions covering advanced topics, …