Posted on

Improving Efficiency: LinkedIn’s Transition From JSON to Protocol Buffers

Programs usually work with data in at least two different representations:

  1. In memory representation: In memory, data is kept in objects, structs, lists, arrays, hash tables, trees, and so on. These data structures are optimized for efficient access and manipulation by the CPU.
  2. Data on file and data over the network: When you want to write data to a file or send it over the network, you need to convert it as a self-contained sequence of bytes.

in-memory representation

Posted on

Getting Started With Protobuf Using C# Runtime Library for Protocol Buffers

In one of our client projects, we were using JSON as a default message format for data interchange between various distributed services. We were sending out large amounts and frequently updated data among our distributed services. So, we were evaluating other formats (eg. MessagePack, Protobuf, BSON) to compact message size and improve the performance. After completing our evaluation, we finalized Protobuf as our default message format.

Protobuf is the binary format crafted by Google which surpasses JSON performance (i.e. it is smaller, faster, and simpler). It is a language-neutral, platform-neutral, extensible mechanism for serializing structured data.

Posted on

Protobuf API Contract Guideline

This post provides guidelines to manage APIs using protocol buffers. Check out this blog post to learn more about Protobuf API contract management.

API Contracts

API contracts describe the surface area of the request and response of each individual API method being offered. It is something that both API providers and API consumers can agree upon and get to work developing and delivering, and then integrating and consuming. An API contract is a shared understanding of what the capabilities of a digital interface are, allowing for applications to be programmed on top of.

Posted on

How to Auto-Generate gRPC Code Using protoc

Creating gRPC code from scratch is a laborious undertaking that, for many companies, makes the cost of implementing gRPC outweigh the benefits. At the very least, starting from scratch means converting gRPC data structures (a.k.a messages) expressed in text and numbers to and from the Protocol Buffers binary format that's intrinsic to facilitating communication between gRPC clients and a server. (See Figure 1, below)

Posted on

Compile Protocol Buffers Using Maven

Hi folks! In this article, we will explore how to compile protocol buffers using Maven. We assume the reader has a basic idea about protocol buffers and Maven.

Google developed protocol buffers for use in their internal services. It is a binary encoding format that allows you to specify a schema for your data using a specification language, like so:

Posted on

How to Build a Streaming API Using gRPC

gRPC is an alternative architectural pattern to REST, GraphQL, and other legacy patterns for providing and consuming APIs. It's becoming a popular way among many companies to create APIs intended to run at web-scale compared to the other architectures that often rely on data formatting standards such as JSON or XML. gRPC uses the Protocol Buffers binary format to exchange data between API providers and API consumers (ie: applications).