feat: lease a run under a grant that can be taken away

day01 commented

2026-08-05 11:32:57 +00:00

Owner

Stacked on #1. First half of the second slice: a run is held under a grant that
the control plane can take away.

Leases

Assign grants a lease with a term and a fence — the number of the grant. The
assignment carries it, every report states it, and the aggregate refuses a
message whose fence is not the current one. That is what makes a report from a
node that has already lost the run land nowhere instead of overwriting the state
of whoever holds it now.

Heartbeat renews the leases it names. reclaim_expired takes back every run
whose holder stopped saying it is there, puts it back in the queue with its
plan, and leaves the old grant behind, so the next hand-out outranks it.

Time enters the aggregate as an argument. Nothing in control-runs reads a
clock, so expiry is testable without waiting for it.

Verification

cargo fmt --check, cargo clippy --workspace --all-targets --all-features -- -D warnings,
cargo test --workspace --all-targets --all-features, check-architecture.sh,
check-rust-loc.sh.

tests/lease.rs is the scenario this slice exists for: a node takes a run and
vanishes, the lease is reclaimed, the run is handed to a successor with its
plan, the vanished node's conclusion is refused as a stale grant, and the
successor's conclusion lands. Plus: a renewed lease is not reclaimed, and a
grant nobody was given can neither renew nor conclude.

Next in this slice

Enrollment and node identity, credential rotation and revocation, typed
capabilities and capacity on the heartbeat, cancellation delivered on the
stream. The runner side follows once enrollment exists, because that is what
lets the binary select this control plane at all.

Stacked on #1. First half of the second slice: a run is held under a grant that the control plane can take away. ## Leases `Assign` grants a lease with a term and a fence — the number of the grant. The assignment carries it, every report states it, and the aggregate refuses a message whose fence is not the current one. That is what makes a report from a node that has already lost the run land nowhere instead of overwriting the state of whoever holds it now. `Heartbeat` renews the leases it names. `reclaim_expired` takes back every run whose holder stopped saying it is there, puts it back in the queue with its plan, and leaves the old grant behind, so the next hand-out outranks it. Time enters the aggregate as an argument. Nothing in `control-runs` reads a clock, so expiry is testable without waiting for it. ## Verification `cargo fmt --check`, `cargo clippy --workspace --all-targets --all-features -- -D warnings`, `cargo test --workspace --all-targets --all-features`, `check-architecture.sh`, `check-rust-loc.sh`. `tests/lease.rs` is the scenario this slice exists for: a node takes a run and vanishes, the lease is reclaimed, the run is handed to a successor with its plan, the vanished node's conclusion is refused as a stale grant, and the successor's conclusion lands. Plus: a renewed lease is not reclaimed, and a grant nobody was given can neither renew nor conclude. ## Next in this slice Enrollment and node identity, credential rotation and revocation, typed capabilities and capacity on the heartbeat, cancellation delivered on the stream. The runner side follows once enrollment exists, because that is what lets the binary select this control plane at all.

day01 changed target branch from feat/hand-a-compiled-plan-to-a-node to develop

2026-08-05 15:35:46 +00:00

day01 added 13 commits 2026-08-05 15:35:46 +00:00

feat: hand a compiled plan to a node over the stream

		The control plane compiles a workflow, keeps the run as an aggregate advanced
by commands, and pushes the versioned plan down a bidirectional gRPC stream.
The node says hello and is given work; nothing polls, and a report only counts
from the node that holds the run.

refactor: drop the commentary and propagate through one session error

		The stream handler converted every failure at its call site. It now has one
error type with From impls and a single conversion to Status at the boundary.

refactor: drop the last conversion the question mark already does

refactor: name the crates after what they hold

		control-domain said nothing about its contents. The runs the control plane
holds, the aggregate that decides about them and the queue in front of them are
one crate; the gRPC stream a node opens is the other.

ci: pin the toolchain action to its version branch

		dtolnay/rust-toolchain publishes versions as branches whose heads move, so a
commit pinned today is unreachable from any ref once the branch advances and a
fresh mirror no longer carries it. The branch is what a mirror can serve.

ci: take the toolchain from a pinned image

		The runner on production cannot resolve an action reference named like a
version, so dtolnay/rust-toolchain is unusable there until it ships the fix.
The toolchain is pinned by image digest instead, which the job container
resolves without any action at all.

feat: lease a run under a grant that can be taken away

		A run is handed out under a lease with a term and a grant number. Everything
the node says about that run states the grant, and a message stating an older
one is refused, so a report from a node that has already lost the run cannot
land. A lease that stops being renewed is reclaimed and the run goes back in the
queue under a new grant.

Heartbeat renews every lease it names; the runs it does not name lapse.

feat: enrol a node and give it a credential that expires

		A node exchanges an enrolment token for an identity and a short-lived
credential. The token is spent on use, rotation replaces the way in rather than
adding one, and a revoked node is refused on the next thing it says. Silence
past the deadline marks a node offline.

Capabilities are typed — architecture, system, container runtime and version,
cores, memory — with labels kept beside them as the compatibility projection.
A node that has not said what it can do, or how much room it has, cannot be
given work.

feat: make a node prove who it is before it is given work

		A session now starts with enrolment: the node spends a token, gets an identity
and a credential, and opens the session with it. Hello carries typed
capabilities and capacity, so a node that has not said what it can do or how
much room it has cannot be given anything. Heartbeat refreshes capacity as well
as leases, which is what makes the answer current rather than what was true at
registration.

feat: refuse work to a node that cannot take it, and say why

		A node below the free space floor is passed over instead of being handed a job
that runs out of disk halfway through, and it is told the reason rather than
left waiting for work that will never arrive. The same sentence answers an
operator asking why a job is not starting.

fix: actually let a lease lapse

		The aggregate expired leases correctly and nothing ever asked it to. A sweep now
runs on a timer: nodes that stopped speaking are marked offline once, and runs
whose lease lapsed go back in the queue under a new grant.

feat: turn a workflow into one run for each combination it describes

		Triggering compiles the workflow once, expands its matrix and queues a run per
combination, each carrying its own plan. A workflow that does not compile is
refused when the run is triggered and queues nothing, instead of failing halfway
through a job.

chore: pin the compiler to the merged workflow develop

day01 merged commit fec7933dc0 into %!s()

2026-08-05 15:50:09 +00:00

day01 deleted branch feat/lease-a-run-with-a-fence

2026-08-05 15:50:09 +00:00

feat: lease a run under a grant that can be taken away #2

Leases

Verification

Next in this slice