Getting started with Etcd

As in many other modern cloudy technologies, I have been reading from the sidelines but not actually engaging with the products. I have a new system which I want to use etcd in order to decouple the configuration APIs from the distributed nodes which will provide the service. After my deep dive into Kubernetes and understanding its architecture, I could not help but want to follow suit and use etcd as the glue for my service.

I figured etcd is one of these nice, new shiny pieces of software that is just going to work out of the box. While yes, it did, there were some major stumbling blocks which consumed more time than I care to admit figuring out. Let me save you this pain and suffering!

The Stumbling Blocks

Side-by-side v2 and v3 APIs — Danger!

I want to wave the biggest red flag I can possibly wave right here. I spent multiple hours trying to figure out why my command line etcdctl and my script I was tinkering with seemingly were writing to two separate databases. I finally discovered that they were, in fact, writing to two separate databases.

A modern etcd 3.x installation supports both the etcd v2 and etcd v3 APIs. These two APIs provide access to two entirely separated databases — separate users, separate roles, separate keyspaces, separate everything!

The documentation does a very poor job of telling you this — I lost where I found the knowledge, but it was not from the actual etcd documentation. The only thing it tells me is:

By default, etcdctl talks to the etcd server with the v2 API for backward compatibility. For etcdctl to speak to etcd using the v3 API, the API version must be set to version 3 via the ETCDCTL_API environment variable.

There is migration documentation which alludes to this fact, but for someone who is coming in for the very first time using this technology it is a bad user experience to have to discover on my own that I have two databases, not one. I am a new user and therefore not very likely to read the migration documentation.

Globs in v2, Ranges in v3

A big change in v3 is the move to a flat keyspace and away from the “directory” based space of v2. This is also talked about in the migration documentation, but it is worthy of a clear understanding for newcomers as well.

In the v2 API you could ask for /path/* glob-style syntax and use recursive options to obtain everything matching that prefix. Now with v3 you have to specify it as a range of /path/ inclusively to /path0 exclusively. Note: In the ASCII character coding, / is followed by 0 — hence the replacement of / with 0 for the last character. The end of the range is exclusive, therefore results will stop before anything with /path0 in the name. Direct documentation of this is here under the Key Ranges subheading.

For convenience, the etcdctl command line utility provides a --prefix qualifier which will calculate the end of the range for you and leaves you without having to do ugly 0 work. The one client library I tested for Javascript also provides a prefix functionality — so this implementation-caused mechanism should hopefully be masked to everyone. However, if you find yourself talking directly via the gRPC interface, you will need to deal with this range mechanism yourself as the API does not directly support a prefix concept.

Installing Etcd

There are many ways you can install etcd depending on your environments. One of the simplest is using Docker to run their official image. If you use the prepared package by your flavor of Linux you may find, due to the quick releases of etcd, that your version is out dated. I think the best way is really to grab their official Docker image using the instructions found on the official Releases page.

✉️ Subscribe to CodeBurst’s once-weekly Email Blast, 🐦 Follow CodeBurst on Twitter, view 🗺️ The 2018 Web Developer Roadmap, and 🕸️ Learn Full Stack Web Development.