6 posts tagged with "infrastructure"

The Fibe control plane is the Rails app behind fibe.gg — it schedules your Playgrounds, talks to your Marquee, moves Mana around your wallet, and answers the SDK. It is the one genuinely centralized piece of Fibe: your environments run on your own Docker host, but the control plane runs on ours. So "drop a few requests, nobody will notice" is not a posture we get to take. A dropped request is a Genie that loses its socket, a fibe command that times out, a half-completed checkout.

The deploy strategy is boring on purpose: zero unavailable pods, a real health probe, and a load balancer that won't route to a pod until that probe is green. We push to main in the afternoon and watch capacity stay flat while a new version rolls in under the old one.

A Player clicks "buy" on a tutorial bundle. A minute or two later they have a real Docker host on the public internet, reachable at a domain that already serves a valid HTTPS certificate for every subdomain it will ever use. No certificate warnings, no "give it five minutes for DNS to propagate," no SSH keys to paste. It just works.

That last clause is doing a lot of work. Between the checkout and the green light, a payment becomes an event, an event becomes a database row, a row dispatches a GitHub Actions workflow, Terraform conjures a VM and a fistful of DNS records, a series of webhooks walk the row through six discrete steps, and finally a background job coaxes Traefik into talking ACME with ZeroSSL until a wildcard cert exists and a real HTTPS client can verify it. If any of those steps stalls, the row doesn't sit in limbo forever — it has a clock on it. This is the story of that path, and of the small pile of state we keep so that "it just works" stays true even when a cloud API has a bad day.

A tutorial Marquee is a single Docker host that serves a small zoo of subdomains: one per Playground, one per service inside a Playground, one for each AgentChat sidecar, plus a few for routing and error pages. They appear and disappear as you work — you can't enumerate them up front, so you can't mint a cert per name. You need one wildcard cert, *.domain, the moment the box comes online — no humans in the loop.

That sounds like a one-liner in a Traefik config. It is not. A publicly trusted wildcard cert means proving you control the whole DNS zone, on a brand-new VM, before a single browser has connected — then not lying to yourself about whether it worked. This is the story of that chain: ZeroSSL with External Account Binding, acme-dns for DNS-01, a cert-warmup router, and a readiness probe that refuses to declare victory on faith.

Every Fibe environment is a little pile of Docker containers on someone's machine, and the platform's job is to make that pile feel like a website: app.your-env.your-marquee.dev should resolve, terminate TLS, and land on the right container. When the environment is mid-rebuild with nothing healthy to land on, the URL should show a 503 that says "this is coming back," not a browser's "can't connect" screen.

That last part is where the interesting engineering lives. The happy path is solved; the hard question is the unhappy path: how do you serve a maintenance page from a router that has to beat the one that would otherwise serve the broken app, without playing whack-a-mole tearing routers down and back up? Our answer is a maintenance router with priority: 1_000_000 — blunt, exactly right, and the subject of this post.

A Player buys a tutorial plan. Somewhere out in GitHub Actions, a Terraform workflow spins up a Scaleway VM, wires up Cloudflare DNS, registers an acme-dns account, and reports back over HTTP. We don't poll it, and we don't hold a connection open for the eight minutes it takes — we wait for it to call us, and when it does, we have to decide in a few milliseconds whether to believe it.

That last part is the whole problem. The workflow is out of band, eventually consistent, and occasionally late. A deploy you kicked off ten minutes ago might still be running when you've given up and started a fresh one — and when that slow old workflow finally phones home, trusting it clobbers the newer environment with its stale view of the world. This post is about the machinery — generations, a drift guard, a two-confirmation null — that lets us listen to an untrusted, asynchronous system without getting burned.

Some people don't want us to conjure them a VM. They already have a box — a beefy machine under a desk, a hardened instance in a VPC their security team trusts, the staging server they've been paying for anyway — and they want their host to be the thing Fibe deploys onto.

So we let them. You hand Fibe an SSH endpoint — host, port, user, a private key — and that box becomes a Marquee: a first-class Docker host that runs Playgrounds, Tricks, and AgentChats just like one we provisioned ourselves. The interesting part is how little we had to add: a bring-your-own Marquee is the same Marquee, minus the part that conjures and tears down a VM.