syncode/control

feat: keep the runs a restart would forget #5

Merged

day01 merged %!d() commits from %!s() into %!s()

2026-08-06 18:14:41 +00:00

day01 commented

2026-08-06 18:10:16 +00:00

Owner

The aggregate was already event sourced, so its history was already the truth and simply had nowhere to live. It lives in Postgres now.

What this changes in the model

Deciding and recording are separate. Run::decide says what a command would do; Run::record takes it as having happened. The registry writes the decision down between the two. A run that took an event in memory and failed to store it would hand out a lease nobody could later prove.

The registry became async. The write happens under the same lock as the decision, so the two cannot drift. That reaches through trigger, intake, the sweep and the node session.

The schema has no state column. State is what replaying the events produces. A run and its opening event are written in one transaction — a run with no history is not something anything could rebuild.

RunLog is a port in the domain, implemented by Postgres in crates/control-store. check-architecture.sh still forbids sqlx in the domain and still passes. Forgotten is the in-memory log the tests drive; the binary cannot reach it, because forgetting every lease on restart should be a choice rather than something arrived at by omission.

Verification

tests/store.rs restarts the control plane against the same database and asks what survived. Each test gets its own schema, and there is no skip when the database is absent — a store test that quietly passes without a store would be worse than none.

a run nobody took is still waiting afterwards, with its plan unchanged
a run being worked on is still held, and is not handed out a second time
the grant outlives the restart: the original fence still works, a fabricated one is refused
a lease that lapsed before the restart is queued again after it

CI now runs Postgres as a service, pinned by digest like every other image. 38 tests, fmt, clippy -D warnings, check-architecture.sh and check-rust-loc.sh all pass. The binary was started against a real database: it listens on both ports and creates its schema.

Not in this slice

Nodes and enrolment are still in memory — a restart still makes every node enrol again. Log lines are accepted and not yet stored.

Every run lived in memory. A restart lost the queue, the leases and the fences — and losing the fence is the part that matters: a node that reconnected still believed it held a run, and the control plane had no grant left to recognise its report against. Fencing is the one thing standing between a lost process and the same job running twice, and it only works if the grant outlives the process that issued it. The aggregate was already event sourced, so its history was already the truth and simply had nowhere to live. It lives in Postgres now. ## What this changes in the model **Deciding and recording are separate.** `Run::decide` says what a command would do; `Run::record` takes it as having happened. The registry writes the decision down between the two. A run that took an event in memory and failed to store it would hand out a lease nobody could later prove. **The registry became async.** The write happens under the same lock as the decision, so the two cannot drift. That reaches through `trigger`, `intake`, the sweep and the node session. **The schema has no state column.** State is what replaying the events produces. A run and its opening event are written in one transaction — a run with no history is not something anything could rebuild. **`RunLog` is a port in the domain**, implemented by Postgres in `crates/control-store`. `check-architecture.sh` still forbids `sqlx` in the domain and still passes. `Forgotten` is the in-memory log the tests drive; the binary cannot reach it, because forgetting every lease on restart should be a choice rather than something arrived at by omission. ## Verification `tests/store.rs` restarts the control plane against the same database and asks what survived. Each test gets its own schema, and there is no skip when the database is absent — a store test that quietly passes without a store would be worse than none. - a run nobody took is still waiting afterwards, with its plan unchanged - a run being worked on is still held, and is not handed out a second time - the grant outlives the restart: the original fence still works, a fabricated one is refused - a lease that lapsed before the restart is queued again after it CI now runs Postgres as a service, pinned by digest like every other image. 38 tests, `fmt`, `clippy -D warnings`, `check-architecture.sh` and `check-rust-loc.sh` all pass. The binary was started against a real database: it listens on both ports and creates its schema. ## Not in this slice Nodes and enrolment are still in memory — a restart still makes every node enrol again. Log lines are accepted and not yet stored.

day01 added 1 commit 2026-08-06 18:10:17 +00:00

feat: keep the runs a restart would forget

		The control plane held every run in memory. A restart lost the queue, the
leases and the fences with it, which is worse than losing work: a node that
reconnected still believed it held a run, and the control plane had no grant to
recognise the report against. The one mechanism that stops a job running twice
only works if the grant outlives the process that issued it.

The aggregate was already event sourced — it decides, produces an event and
appends it to a log with a monotonic sequence — so the run's history was
already the truth and only had nowhere to live. It lives in Postgres now.

Deciding and recording are separated so a decision can be written down before
it is believed. A run that took an event in memory and failed to store it would
hand out a lease nobody could later prove. Because the write happens under the
same lock as the decision, the registry became async, and that reaches through
trigger, intake, the sweep and the node session.

The schema has no column for a run's state. State is what replaying the events
produces, so there is nothing stored that could disagree with them. A run and
the event that opened it are written in one transaction: a run with no history
is not something anything could rebuild.

RunLog is a port in the domain and Postgres implements it in a crate of its
own, so the aggregate is still exercised without a database and the boundary
check still holds. Forgotten is the in-memory log tests drive; it is nowhere
near the binary, because forgetting every lease on restart should be a choice
and not something arrived at by omission.

day01 merged commit 86228356df into %!s()

2026-08-06 18:14:41 +00:00

day01 deleted branch feat/keep-the-runs-a-restart-would-forget

2026-08-06 18:14:42 +00:00

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: syncode/control#5