Using SSL with gRPC in Python

We have recently started using gRPC at Sandtable and we really like it.

gRPC is an HTTP/2 based open-source RPC framework released by Google in 2015. It uses Google’s Protocol Buffers as the interface definition language and data serialisation format. gRPC can be used from a number of different languages, for example, C++, Python, and Golang, to name a few. Adoption of gRPC is picking up, particularly for building microservices, and it is already used in open source projects like Kubernetes, etcd and Docker Swarm.

It’s very easy to get going with gRPC, to start we suggest playing with the official examples. See the official guide for the basics of how to use gRPC with Python.

SSL and server-side authentication with gRPC

We are using gRPC in a number of ways, including to connect to services in the cloud over the internet. We want these connections to be secure and hence we’re interested in using SSL and authentication with gRPC. For background information on authentication and gRPC, see the official guide.

In a nutshell, gRPC supports SSL and includes a number of built-in mechanisms for authentication. It also supports adding metadata to channels or per call; we briefly discuss how to add metadata to RPC calls below. We also show how to turn on client-side payload compression.

At the moment, the gRPC website only includes client-side code for using SSL and server-side authentication. For Python code, see here. After some searching, however, we found the pwserverd project (thanks!) that includes code for both sides. It was very helpful, and we thought we would share the entire process, including a simple example, with others.

To demonstrate using SSL and authentication, we will walkthrough a simple example. The full code can be found here. We assume familiarity with implementing gRPC clients and servers in Python.

The required steps are:

  1. Generate a root certificate and private key.
  2. Define an RPC service
  3. Write server code
  4. Write client code

Server certificate

Firstly, generate a certificate and private key. We use openssl to do this. The certificate can be self-signed.

To generate a root certificate and private key, run and then follow the required steps:
openssl req -newkey rsa:2048 -nodes -keyout server.key -x509 -days 365 -out server.crt

or, for ease of use, we defined a Makefile target:
make gen_key

Two files should be output: server.crt and server.key. The crt file includes the root certificate that is used to authenticate the server; this must be available to the server and shared with the client. The key file includes the private key that only the server has access to.

RPC definition

For our simple example, we define a single service: Server, with a single end-point Foo that requires an Empty message as its argument and returns a message also of type Empty. The Empty message contains no data; it’s the simplest message.

You can see the code here.

Then generate the stubs, do:
make stubs

Two Python files will be output: service_pb2.py and service_pb2_grpc.py.

Server code

You can see the full code here.

To run the server, do:
make server

Client code

You can see the full code here.

To run the client, do:
make client

Call metadata

It’s easy to add metadata to RPC calls. Examples of metadata that could be included with RPC calls are user credentials or Zipkin trace ids.

On the client-side, for example, if we want to add a key-value pair, such as foo: bar, to the metadata of a call to the Foo method, do:

Then, on the server, to read the metadata you call the invocation_metadata method of the context argument:

Compression

It’s also possible to turn on payload compression for RPC calls (it’s off by default). Here is some example client code. Note that this code is by horkhe and taken from an gRPC issue.

Currently supported compression types are: none, deflate, and gzip; and compression levels: none; low (equivalent to gzip -3); medium; high (-9).

Finally

Next, we plan integrate our SSO solution using the call metadata feature. More to follow.

A big thanks to Google for open sourcing gRPC.

Comments

  1. Frontware says:

    Hi,

    Our grpc server authentication uses TLS instead of SSL.
    We couldn’t find much information about how to use TLS in python GRPC client.

    Any idea how to do that?

    Thanks.

  2. Thomas says:

    Hi Frontware.

    I’m not sure. The gRPC documentation treats them as the same.

    Thanks.
    Thomas

Leave a comment

Please prove that you are human: