7 posts tagged with "runtime"

Provisioning a Playground touches more moving parts than we'd like to admit. We SSH into a Marquee, clone source from a Prop, merge registry credentials, build an image, generate a Compose file, and bring it up behind Traefik. Each step can fail, and for wildly different reasons: a flaky network, a typo in a branch name, a server that ran out of disk, a Docker daemon that blinked.

For a long time, "what do we do when step N fails?" was answered by a human reading a Sidekiq backtrace. That doesn't scale, and worse, it's inconsistent — the same failure handled two different ways on two different days. So we turned operational judgment into data. Every failure gets a category, and the category — not a person, not an ad-hoc rescue clause — decides what's next.

A user clicks "Start" on a Genie. Forty seconds later there's a Claude Code agent running in a container on their Marquee, reachable over TLS at a stable subdomain, talking to their repo through MCP, with their skills and main prompt already staged on disk. If the container later falls over, something notices within a minute and quietly puts it back.

That whole arc — from a config row in Postgres to a self-healing sidecar — is one job, one Redis lock, six steps, and a surprising amount of opinion about what counts as a failure. This post is a walk through the AgentChat deploy pipeline: what we build, in what order, which steps are allowed to fail and retry, and the one design decision that quietly shaped everything else — that a missed health check is not a reason to kill anything.

There are two stories about what your Playground is doing. One lives in Postgres — a status that says running, error, or stopped. The other lives on a Docker host we don't own, reachable only over SSH, and you read it by running docker compose ps. Most of the time they agree; the interesting engineering is all about what happens when they don't.

This is a post about one reconciler and the small set of rules it uses to decide which story is true. The headline rule is easy to say and weirdly hard to honor: trust the daemon as the source of truth — but only when you can actually see it. A Playground on a host that clearly threw its containers away should be marked stopped; one on a host that simply didn't answer our SSH knock should be left exactly as it is. Telling those two apart is the whole job.

You click "create," and a few seconds later there's a URL that serves your app. In between, a surprising amount happens — most of it on a machine you've never logged into. A Playground on Fibe isn't a container we start. It's a small, ordered ritual: claim a database row, clone source onto a remote host, merge registry credentials, build an image, render a compose file from a template, ship it over SSH, and only then run docker compose up — on the Marquee, not on us.

This post walks that ritual end to end. Eight steps, nine statuses, one set of legal transitions, and a fair amount of paranoia about what happens when a step fails halfway through. If you've ever wondered what "in progress" actually means while the spinner spins, this is the tour.

A Marquee is one Docker host. On it, a Player might run half a dozen Playgrounds at once — a couple of long-lived dev environments, a headless Trick chewing through a job, an AgentChat sidecar where a Genie is editing code. They share the same daemon, Traefik instance, range of host ports, and wildcard domain. And every one needs a URL that goes to exactly one place.

That's the whole problem: shared host, distinct addresses. Get it wrong and two environments fight over app.your-host or both grab host port 8080 — Traefik routes arbitrarily, or Docker refuses the second container, and the Player gets a confusing failure deep in the creation pipeline. Our answer is boring on purpose: we refuse to save an environment that would collide, before it touches the host. Validate at write time, routing stays dumb.

Here is the mental model most people start with: Fibe checks out your repo, runs docker compose up from the repo root, and that's your environment. Clean picture. Wrong in every load-bearing detail.

There is no repo root Fibe cds into, no checkout you pushed. The control plane generates a compose file from your Template, rsyncs it to a precise path on your Marquee, and runs docker compose -p <project> -f compose.yml up over SSH — explicit -p and -f, never "whatever's in this directory." This post walks that machinery: the /opt/fibe layout, the generate → rsync → up pipeline, and the marker that makes cleanup safe.

Every Playground on Fibe is born with an expiration date. Spin one up, walk away, and eight hours later it's gone — containers down, volumes reclaimed, the database row deleted. That's not a quota we enforce grudgingly; it's a default we're proud of. Idle environments are a tax everyone pays: they burn compute on a Marquee someone funds daily, pile up as ghost subdomains, and make "what's actually running here?" unanswerable. So we let them die.

But there's exactly one thing we will not do, no matter how expired an environment is: throw away your uncommitted work. If a Playground has dirty files when its clock runs out, the expiry sweep parks it rather than destroying it. This post is about that one rule — and how little code it takes to hold ephemerality and data safety at once.