Every Resident Writes Its Own Sign - the cover: Pip next to a stack of signs: who am I, migrations, schemas, history, env

Every Resident Writes Its Own Sign

Every server answers "who are you" by itself: its contract, migrations, tables, history and settings - and it cannot forget to, because nobody writes that code by hand.

👋 Hi, I'm Anton - a software engineer. These days I work on a platform: contracts, generated code, data paths and the tooling around them. Running notes live on my GitHub: github.com/brilliant-almazov.

The self-description is built into the shared library and runs on every gRPC server that is built on it. Two of its eight answers - the guarded data query and the settings override - are off by default; I describe them as built, not as used.

This is part 1 of the series on what a shared platform does for every service, told as a zoo. Maybe you already do it better, maybe you'd do it another way - this is how it is going, and where it stops.

In real life

Our platform is a zoo. Every service is a resident with its own enclosure, the bus is the paths between the enclosures, acceptance is the vet, and tickets are tokens. The residents in this series are always the same: Pip, who shows visitors around; Nora, the keeper who still checks some things by hand; Biscuit, the dog who carries messages along the paths; Miso, the cat who owns a resource and ignores most requests; Pickle, the guinea pig who helps the vet; and Ember, the small dragon who guards the shared gold.

Every zoo has signs. A board on a post in front of the enclosure: the name, what the resident eats, where it comes from. In most zoos a keeper writes the sign once, and the sign stays the same for years. So one day Nora stops in front of Miso's enclosure and reads: "eats fish". Miso looks at her with the face cats keep for this kind of moment. "Since spring I eat cake."

The sign was right on the day it was painted. Then the resident changed, and the sign did not, because the sign and the resident are two different things and only one of them is alive.

A four-panel comic: Nora reads a sign saying Miso eats fish, Miso says since spring it eats cake; Miso next to a sign it writes itself; Pickle asks what Miso ate last week and Miso offers its diary page by page; Pip asks who writes the sign when Miso naps and Miso says the sign writes itself

The fix in the comic is the fix in the platform. The resident writes its own sign. Pickle can walk up and ask what Miso ate last week, and Miso hands over the diary, page by page. And when Pip asks who writes the sign while Miso naps, the answer is the punchline of the whole idea: nobody. The sign writes itself, because it is not a separate thing any more.

A page about a service is a sign that lies

Every service I have worked with had a page about it somewhere: which tables it owns, which migrations ran, which settings it reads, which version is deployed. Every such page was correct when it was written and drifted afterwards. Not because people are careless, but because the page and the service change at different moments, for different reasons, by different hands.

The shared Go platform library starts every gRPC server of every service through the same lifecycle. That gave us a place to put the sign where the resident cannot lose it: on the server itself, registered by the library, answering from the running process and its own database. A service does not add a line of code for it, and it has no switch to forget to turn on.

Three tiles with faces: an admin screen asks any gRPC server with eight questions; the server, proud, reads its own database read only; pills read built in the platform, and the data query is off by default

Eight questions every server answers

Any server built on the library answers eight questions. They fall into four groups: who I am, what my data looks like, a guarded query, and my settings.

Bars: who am I and my contract 1, data - migrations, schemas, types, history 4, a guarded query 1, environment - read and override 2; Pickle asks: eight answers, zero lines?; pills read built on every server, and two of eight are off by default

The shape of the answers, as a form with values left out:

DescribeService -> name, version, contract fingerprint (sha-256) ListMigrations -> applied migrations ListEntityTypes -> the entities this service owns ListDataSchemas -> tables, columns, indexes, partitions, foreign keys ListEntityHistory -> change records, cursor after_id QueryData -> a guarded read query (off by default) ListEnvironment -> the keys this service reads SetEnvironmentOverride -> LOG_LEVEL in memory until restart (off by default) 

A dark card listing the eight self-description calls with what each returns; the first line, the service description with the contract fingerprint, is highlighted; pills read built, and no service code, no env key for health

A few details are worth a sentence each.

The contract fingerprint is a SHA-256 of the compiled descriptors the server actually serves. Two servers with the same fingerprint speak the same contract, whatever their version strings say. That is the line on the sign that is hardest to fake.

The schema answer is richer than a list of migration files: tables, columns, indexes, partitions and foreign keys, as the service describes its own database. Migrations say how the tables were meant to change; the schema answer says what they look like.

The history answer reads the change journal that the library already keeps for entities and pages through it with a cursor, so the admin screen can show "what happened to this record" without a direct connection to that service's database.

The settings override is the only answer that changes anything, and it changes as little as possible: the log level, in memory, until the next restart, and the call carries the id of whoever made it. It is off by default.

Next to the sign sits a health service that every server also gets for free - "no line of code and no environment variable" was the requirement, and that is how it is built.

Who gets the sign

The sign is not something a service opts into. It comes with the server resource of the library. Counted by imports across our services, about thirty of them use the library's resources package, where that server lives - all but one of the services that import the library at all.

A donut: about 30 services; import the resources package about 30, the rest 1; Miso, proud, says: I did not write a line; a pill reads counted by imports, rounded

What the sign answers Where the answer comes from Can a service forget it?
who I am, contract fingerprint the running server's descriptors no
migrations, entity types, schemas the service's own database no
change history the entity change journal no
a guarded read query the database, through a fenced path off by default
settings it reads, log level override the running process override off by default

That last column is the whole point. A page about a service is a promise someone has to keep. A sign the server writes is not a promise at all - it is a side effect of starting.

What changed for the people around the zoo

The admin screen is meant to stop needing a database address for every service: it asks the service, and the service answers about itself. That is a quieter change than it sounds. It means the only component that holds the key to a resident's database is the resident, and the next part of this series is about the one place where we let people look inside anyway.

The "is it deployed?" question became a call instead of a conversation. The fingerprint answers "which contract", the migrations answer "which schema", and nobody has to remember to update a page after a release.

Before and after: a page about a service - by hand, someone writes and forgets; stale, true on the day it was written - and the sign it writes - 0 lines of service code; live, answered by the running server

An animation: an admin screen, any server and its database, all bored; the admin asks who are you, the server answers itself, reads schemas and history read only, and the sign is filled in

What went wrong

Nothing dramatic broke on the way to the sign; what went wrong is the problem it answers. Pages about services drifted, as they always do, and every reader paid for it by asking a person instead of the service. The lesson behind putting the sign into the library is simple: anything that describes a service and is written by a person is out of date by the time a second person reads it.

The second lesson is about scope. Two of the eight answers can do more than describe: one reads arbitrary data, one changes a setting. Both are off by default, and the guarded read got a part of its own, because "a sign" and "a key" are very different things to hand out.

And the honest part: the sign is built and present on every server built on the library, but the reader it was built for is one admin screen. The value of a sign grows with the number of people who stop by to read it.

Service tiles with faces: identity, tokens, project data and the scheduler are happy; the admin screen is proud; the data query and the env override are bored, off by default; health is happy

What I do not claim

  • Not that every service has every answer turned on. Two of eight are off by default.
  • Not that the sign replaces documentation of why a service exists. It answers what it is now.
  • Not that the admin screen uses all eight answers today.
  • Not exact counts. The number of services is rounded from imports.

What of this is method

  • A description written by the thing itself does not drift. Put the sign on the server, not on a page.
  • Zero lines of service code, zero switches to forget. If it can be forgotten, it will be.
  • Describe freely, change narrowly, off by default. The one answer that changes something changes the least possible.

Zoo keeping, built in - Part 1. Next: A Viewing Window, Not A Key To The Cage.

If your services describe themselves, what do they answer - version, schema, or more? If you keep pages about services, how do you notice when one starts to lie? If you tried both, which one did your on-call engineers actually read? What does the sign in front of your service say today?