soma.cell

Release pipeline — "tests pass before deploy" as a theorem, not a habit

On August 1, 2012, Knight Capital deployed new trading code to seven of eight production servers. The eighth still ran a dead code path that the new deploy re-activated by reusing its feature flag. In 45 minutes Knight bought and sold $7 billion of stock it didn't want and lost $440 million — the firm was sold within the year. The SEC's post-mortem: no enforced deployment process. A human checklist, partly followed. Below, the pipeline is a state machine: production is reachable ONLY through green tests and a human approval, soma verify proves no other path exists, and the deploy ledger lets each build artifact reach production AT MOST ONCE — re-pushing a rolled-back artifact (Knight's exact sin: stale code reactivated) is a write the memory refuses.

A Soma program from the governance part of the verified corpus: 133 lines, cells Pipeline, PipelineTests. It passes soma check with soma 3.2.4.

How it is built

The cell Pipeline declares 6 signals in its face: record_build(rel: String) -> Map, report_tests(rel: String, failures: Int) -> Map, sign_off(rel: String, approver: String) -> Map, deploy(rel: String) -> Map, roll_back(rel: String) -> Map and deploys_of(rel: String) -> Int; keeps 2 memory slots: prod_pushes (Map<String, Int>, persistent and consistent) under the invariant prod_pushes >= 0 && prod_pushes <= 1 and test_failures (Map<String, Int>, persistent and consistent) under the invariant test_failures >= 0. Its state machine release starts in committed and allows 7 transitions: committed -> built, built -> tested, built -> red, tested -> approved, approved -> live, live -> rolled_back and red -> committed. rolled_back is terminal. Handlers: record_build (line 54), report_tests (line 60), sign_off (line 71), deploy (line 77), roll_back (line 91) and deploys_of (line 97).

The file also holds 1 test cell (PipelineTests) with 19 assertions, run by soma test.

Generated from soma describe on this file.

The source

release_pipeline.cell

// ============================================================================
// RELEASE PIPELINE — "tests pass before deploy" as a theorem, not a habit
// ============================================================================
//
// On August 1, 2012, Knight Capital deployed new trading code to seven
// of eight production servers. The eighth still ran a dead code path
// that the new deploy re-activated by reusing its feature flag. In 45
// minutes Knight bought and sold $7 billion of stock it didn't want and
// lost $440 million — the firm was sold within the year. The SEC's
// post-mortem: no enforced deployment process. A human checklist, partly
// followed.
//
// Below, the pipeline is a state machine: production is reachable ONLY
// through green tests and a human approval, soma verify proves no other
// path exists, and the deploy ledger lets each build artifact reach
// production AT MOST ONCE — re-pushing a rolled-back artifact (Knight's
// exact sin: stale code reactivated) is a write the memory refuses.
// ============================================================================

cell Pipeline {
    face {
        signal record_build(rel: String) -> Map
        signal report_tests(rel: String, failures: Int) -> Map
        signal sign_off(rel: String, approver: String) -> Map
        signal deploy(rel: String) -> Map
        signal roll_back(rel: String) -> Map
        signal deploys_of(rel: String) -> Int
    }

    memory {
        // artifact -> times deployed to production. AT MOST ONCE:
        // a rolled-back build cannot crawl back into prod.
        prod_pushes: Map<String, Int> [persistent, consistent]
        invariant prod_pushes >= 0 && prod_pushes <= 1

        test_failures: Map<String, Int> [persistent, consistent]
        invariant test_failures >= 0
    }

    // rolled_back is TERMINAL: a bad artifact is dead. You cut a new
    // release; you do not resurrect the old one. soma verify proves
    // `live` is reachable only through `tested` and `approved`.
    state release {
        initial: committed
        committed -> built
        built -> tested
        built -> red
        tested -> approved
        approved -> live
        live -> rolled_back
        red -> committed
    }

    on record_build(rel: String) {
        let r = try { transition(rel, "built") }
        if r.error != () { return map("ok", false, "reason", "{r.error}") }
        return map("ok", true, "state", "built")
    }

    on report_tests(rel: String, failures: Int) {
        if get_status(rel) != "built" {
            let s = get_status(rel)
            return map("ok", false, "reason", "no build to test (state: {s})")
        }
        test_failures.set(rel, failures)
        if failures == 0 { transition(rel, "tested") }
        if failures > 0 { transition(rel, "red") }
        return map("ok", true, "state", get_status(rel), "failures", failures)
    }

    on sign_off(rel: String, approver: String) {
        let r = try { transition(rel, "approved") }
        if r.error != () { return map("ok", false, "reason", "{r.error}") }
        return map("ok", true, "approver", approver)
    }

    on deploy(rel: String) {
        if get_status(rel) != "approved" {
            let s = get_status(rel)
            return map("ok", false, "reason", "deploy from {s} is not a thing")
        }
        // THE WALL: one production push per artifact, ever
        let w = try { prod_pushes.set(rel, (prod_pushes.get(rel) ?? 0) + 1) }
        if w.error != () {
            return map("ok", false, "reason", "{rel} already reached prod once; cut a new release")
        }
        transition(rel, "live")
        return map("ok", true, "state", "live")
    }

    on roll_back(rel: String) {
        let r = try { transition(rel, "rolled_back") }
        if r.error != () { return map("ok", false, "reason", "{r.error}") }
        return map("ok", true, "state", "rolled_back")
    }

    on deploys_of(rel: String) {
        return prod_pushes.get(rel) ?? 0
    }
}

cell test PipelineTests {
    rules {
        // a fresh commit cannot leap to production — the edges don't exist
        assert_fails transition("r1", "live")
        assert_fails transition("r1", "approved")
        assert deploy("r1").ok == false

        // red tests bounce the release back to committed
        assert record_build("r1").ok == true
        assert report_tests("r1", 3).state == "red"
        assert sign_off("r1", "dana").ok == false        // red -> approved: no edge
        assert_fails transition("r1", "tested")          // can't relabel red as green

        // fix, rebuild, go green, approve, ship
        assert transition("r1", "committed").to == "committed"
        assert record_build("r1").ok == true
        assert report_tests("r1", 0).state == "tested"
        assert deploy("r1").ok == false                  // green but unapproved
        assert sign_off("r1", "dana").ok == true
        assert deploy("r1").ok == true
        assert deploys_of("r1") == 1

        // Knight Capital's sin: re-activating a dead artifact.
        // rolled_back is terminal AND the push ledger refuses a second 1.
        assert roll_back("r1").ok == true
        assert_fails transition("r1", "live")
        assert_fails transition("r1", "committed")
        assert_fails prod_pushes.set("r1", 2)
        assert deploy("r1").ok == false
    }
}

What the toolchain says

Run on a fresh copy of the program at build time; the outputs are pasted unedited.

$ soma check release_pipeline.cell
✓ All checks passed.
$ soma test release_pipeline.cell
test PipelineTests ...
  ✓ assert_fails transition("r1", "live") — raised invalid transition: committed → live. Current state: 'committed'. Valid targets: [built]
  ✓ assert_fails transition("r1", "approved") — raised invalid transition: committed → approved. Current state: 'committed'. Valid targets: [built]
  ✓ assert deploy("r1").ok == false
  ✓ assert record_build("r1").ok == true
  ✓ assert report_tests("r1", 3).state == "red"
  ✓ assert sign_off("r1", "dana").ok == false
  ✓ assert_fails transition("r1", "tested") — raised invalid transition: red → tested. Current state: 'red'. Valid targets: [committed]
  ✓ assert transition("r1", "committed").to == "committed"
  ✓ assert record_build("r1").ok == true
  ✓ assert report_tests("r1", 0).state == "tested"
  ✓ assert deploy("r1").ok == false
  ✓ assert sign_off("r1", "dana").ok == true
  ✓ assert deploy("r1").ok == true
  ✓ assert deploys_of("r1") == 1
  ✓ assert roll_back("r1").ok == true
  ✓ assert_fails transition("r1", "live") — raised invalid transition: rolled_back → live. Current state: 'rolled_back'. Valid targets: []
  ✓ assert_fails transition("r1", "committed") — raised invalid transition: rolled_back → committed. Current state: 'rolled_back'. Valid targets: []
  ✓ assert_fails prod_pushes.set("r1", 2) — raised memory invariant violated on 'prod_pushes': prod_pushes >= 0 && prod_pushes <= 1 — rejected write of 2 (key "r1"); the slot is unchanged
  ✓ assert deploy("r1").ok == false

19 tests: 19 passed, 0 failed
$ soma verify release_pipeline.cell
Verifying release_pipeline.cell...

State machine 'release': 7 states, initial 'committed'
  States: [approved, built, committed, live, red, rolled_back, tested]

  ✓ 7 states, 7 transitions
  ✓ all states reachable from 'committed'
  ✓ terminal states: [rolled_back]
  ✓ no deadlocks
  ✓ liveness: every state can eventually reach a terminal state
  ✓ think-isolated: CTL safety properties hold regardless of LLM output (5 handlers, 6 literal transitions, 0 dynamic)
  ✓ termination: all 6 handlers structurally terminate
  ⚠ refinement: declared transition `red → committed` is never reached by any handler — spec may be aspirational or stale
  ✓ refinement: handler `record_build` ⟶ {built}
  ✓ refinement: handler `report_tests` ⟶ {built → tested [if failures == 0], built → red [if failures > 0]}
  ✓ refinement: handler `sign_off` ⟶ {approved}
  ✓ refinement: handler `deploy` ⟶ {approved → live}
  ✓ refinement: handler `roll_back` ⟶ {rolled_back}
  · transition sources: `record_build`, `sign_off`, `roll_back` call transition(id, target) without declaring the source — the runtime refuses a move the graph forbids, but the proof does not see where the handler starts; `transition(id, from, to)` names the edge (checked by `soma check`, refused at run time from any other state)

Pipeline/invariants:

  ✓ invariant prod_pushes >= 0 — writer 'deploy' proven by induction (writes (prod_pushes.get(rel) ?? 0) + 1)
  ⚠ invariant prod_pushes >= 0 && prod_pushes <= 1 — runtime-checked (computed values): deploy → prod_pushes [prod_pushes <= 1] because `prod_pushes` has no known range
  ⚠ invariant test_failures >= 0 — runtime-checked (computed values): report_tests → test_failures because `failures` is a parameter (narrow it: `require failures >= 0 else …`)

13 passed, 3 warnings, 0 failures

  Temporal properties for 'release':
  ✓ deadlock_free — no deadlocks in any reachable state
Temporal: 1 passed, 0 failed
VERIFY OK

Builtins used

get_status, map, transition

Source file: /corpus/governance/release_pipeline.cell · examples/corpus/governance/release_pipeline.cell