Building ichi
An engineering journal. A few milestones, told as stories — background, challenge, decision, and the lesson that outlasted it.
Principles are dry until you see where they came from. Each of these is a real turning point, told in the same shape: what we faced, what we chose, and what it taught.
The night the monitor lied
Background. A service had been failing for days. Nothing alerted.
Challenge. The monitoring reported everything green — because it was watching the wrong thing. A port had drifted; the healthcheck pinged a name that no longer matched the running service. Silence was read as health.
Decision. Rather than patch the one check, we treated the class of failure as the bug: a system that cannot tell you it is broken will eventually convince you it is fine.
Unexpected result. Fixing the naming was trivial. The lasting change was cultural — every new service now has to prove it can report its own failure before it is trusted to run unattended.
Lesson. The most dangerous state is not “broken.” It is “broken and quiet.”
The database we never built
Background. The content was growing. The instinct was to reach for a database.
Challenge. A database is a running service to secure, back up, patch, and eventually migrate. For a body of work that was small and almost entirely text, that is a large standing cost paid forever.
Decision. Plain Markdown files in version control. No database.
Unexpected result. Version control gave us, for free, the thing a database would have made hard: a complete, diffable history of every change, and instant revert. The “primitive” choice turned out to be the more powerful one.
Lesson. Reach for the boring tool. It fails in ways you already understand.
The gate that slowed everything down
Background. The agents were capable enough to act on their own.
Challenge. Letting them act — publish, deploy, spend — would have been faster. Every human approval is a bottleneck by definition.
Decision. We kept the human gate anyway. Agents propose; a person disposes.
Unexpected result. The bottleneck became the feature. Because nothing shipped without review, we could let the agents work aggressively and even overnight — the gate made their speed safe to use. Remove the gate and we would have had to slow the agents down instead.
Lesson. A well-placed constraint does not limit a system. It is what lets the rest of it run fast.