​

BenchFlow — Architecture

​

What BenchFlow is

​

Grounding

• “Environment 总是要 roll out, roll back” — roll-out and roll-back are definitional for an environment. Roll-back = “snapshot environment and go back to its different stage.”
• Branching: an ask_user-type interaction “literally is a checkpoint … to different type of rollout”, and from it — “From reward function to a value function … of the current state.” Branching is “very important for large horizon tasks.”
• “eval = monitoring = reward” — one activity, observed across five spaces (output, action, reasoning, memory, latent).
• “The harness is not meant to be intelligent” — self-improvement targets the model and skills, never the harness; “skill 是属于 memory” (skills are memory).
• ACP is the mechanism for modelling human interaction inside a rollout.

​

Design principles

1. The kernel depends only on contracts. The call graph is the source of truth; anything exported with no live caller is wired in or deleted.
2. Four planes, each swappable, each managed + BYO — Sandbox, Agent, Environment, Reward.
3. The Rollout is a tree. An RL episode is a tree of states; a linear rollout is the degenerate degree-1 case. Branch, snapshot, and restore are first-class — they are how a reward function becomes a value function.
4. Roll-back is definitional. An environment that cannot snapshot and restore its state is incomplete (Han). snapshot/restore are real methods, not stubs.
5. Zero-modification adoption. A benchmark brings a self-describing package + a manifest; it never subclasses BenchFlow or touches private APIs.
6. Eval = monitoring = reward — one activity, scored on the same trajectory, across five spaces.
7. The environment is a stateful state machine the framework provisions, snapshots, restores, and tears down.
8. BenchFlow is the ACP Client — the “user” is a pluggable policy, not a special actor.
9. The harness is not intelligent — its only job is to extract the most from the model; self-improvement targets the model and skills.
10. Readiness and teardown are framework guarantees — never the benchmark’s burden.
11. Ship beats design — a better design that doesn’t run loses to an adequate one that does.

​

The conceptual model — the planes

        bench CLI · bf.run() · the environment manifest
                          │
        ┌─────────────────▼───────────────────┐
        │  KERNEL                               │
        │  Rollout lifecycle · reward · trajectory │
        │  depends ONLY on contracts/           │
        └──┬──────────┬───────────┬──────────┬──┘
           ▼          ▼           ▼          ▼
       Sandbox      Agent     Environment   Reward
       (where)     (who)      (the world)  (how scored)

​

The execution model — tree-native

Job         — a set of Rollouts run together (an eval sweep · a GRPO group · a CL sequence)
Rollout     — one RL episode = a TREE of states (sₜ)                          PRIMITIVE
  • Step    — one edge of the tree: (reason → act) → (tool-in → tool-out)      PRIMITIVE
  • Branch  — the snapshot-and-fork operation; a node with >1 child            PRIMITIVE
Trajectory  — one root-to-leaf path. Computed from the tree, never declared    DERIVED VIEW
Scene       — a declared span carrying a role/skill configuration             AUTHORING SUGAR

• The primitives are irreducible. The tree (Rollout), its edges (Step — Han’s atomic unit; one Step is one “turn”), and the Branch operation (snapshot + fork). Branch is the credit-assignment engine: it evaluates one state across N continuations — averaging the children’s returns estimates V(s). That is Han’s “from reward function to a value function of the current state” and Tree-GRPO’s peer-reviewed result that a tree yields process supervision from a single outcome reward. A GRPO group run as a shared-prefix tree beats N independent rollouts (more rollouts per token/tool budget). Branches occur at ask_user-style interaction checkpoints (one child per option), at GRPO group points, and at value-estimation points.
• Trajectory is a derived view — a pure function of the tree (a path), never declared. It is what serialises out: a linear prompt / completion / reward / metrics / info record, the Verifiers/ORS training unit.
• Scene is authoring sugar — the declaration form for multi-phase / multi-agent rollouts (RolloutConfig.scenes). It desugars completely to per-Step role/skill attribution plus config that changes along the tree, and adds no expressive power. It has no runtime object and no lifecycle of its own — RolloutConfig.scenes is a desugaring pass that lowers to per-Step config. Kept only as a convenient authoring affordance. (The original RFC’s instinct — “a phase is just state” — was correct.)

​

Lifecycles

​

The four planes

​

Skill loading

​

The four contracts

class Sandbox(Protocol):            # where it runs — container level
    async def exec(cmd, *, user, timeout) -> ExecResult: ...
    async def upload(local, remote) -> None: ...
    async def download(remote, local) -> None: ...
    async def expose_port(port) -> Endpoint: ...
    async def snapshot() -> SandboxImage: ...
    async def restore(image: SandboxImage) -> None: ...
    async def teardown() -> None: ...

class Agent(Protocol):              # who acts — Han's harness
    async def connect(sandbox, role) -> Session: ...
    def capabilities() -> AgentCapabilities: ...

class Session(Protocol):            # a LIVE agent session — the Agent plane's real surface
    async def prompt(text: str) -> StopReason: ...   # the task instruction, or a nudge
    async def cancel() -> None: ...
    def on_ask_user(handler: AskUserHandler) -> None: ...  # agent-initiated; the branch hook
    @property
    def steps(self) -> list[Step]: ...               # the session's contribution to the tree

class Environment(Protocol):        # the world — Han's S
    async def provision(ctx) -> EnvHandle: ...
    async def readiness() -> ReadinessProbe: ...
    async def query() -> EnvState: ...               # for the verifier
    async def snapshot() -> StateSnapshot: ...       # roll-back: definitional
    async def restore(snap: StateSnapshot) -> None: ...
    async def reset() -> None: ...
    async def teardown() -> None: ...

class Reward(Protocol):             # how it's scored — Han's V
    async def score(node: RolloutNode) -> VerifyResult: ...

​

The Environment plane & the manifest

[environment]
name           = "chi-bench"
image          = "chi-bench:latest"   # OR base_image + [[services]] (framework-started)
owns_lifecycle = true                 # the image's entrypoint starts the services
isolation      = "per_task"           # OR "persistent" (cross-episode state)

[environment.task_selection]
mechanism   = "env_var"               # OR "image" (per-task images, smolclaws-style)
key         = "CHI_BENCH_TASK_ID"
inject_into = "entrypoint"            # reaches PID 1, not just exec()

[environment.readiness]               # the framework gates on this before the agent runs
http        = ["http://localhost:8023/health"]
timeout_sec = 120

[verifier]
kind              = "agent"
hidden_from_agent = ["expectations.json", "tasks/*/fixtures"]

​

Evaluation — the five spaces

​

The interaction model — ACP

• session/prompt (Client → Agent) — the task instruction and every nudge (user-initiated follow-up).
• request_permission / ask_user (Agent → Client, with enumerated options) — agent-initiated, surfaced through Session.on_ask_user.

​

The edges — adapters & trainers

​

How a Task flows through the architecture

Task ─┬─→ selects the Environment package + manifest  ───→ Environment plane provisions S
      ├─→ carries the instruction / prompt            ───→ Agent plane (H) receives it
      ├─→ names the verifier + hidden fixtures         ───→ Reward plane (V) scores
      └─→ carries config (turn limits, budgets)        ───→ RolloutConfig (C)
                                   │
                                   ▼
              one Rollout (a tree) runs in a Sandbox
                                   │
                                   ▼
          Trajectory(s) + reward  ───→  export  ───→  trainer

​

The eight capabilities — how each fits

​

The hard part — honest risk

1. Environment-state snapshot — DB dump/restore, copy-on-write volumes, fork-able service processes. Designed deliberately, environment-class by environment-class — not one generic call.
2. Agent-session snapshot — freezing and restoring a live ACP session (and the running agent process and its context behind it) is the same class of hard problem. Inspect can only deep-copy conversation state precisely because it cannot snapshot the process. The architecture does not get this for free.
3. Cross-layer consistency — the three snapshot layers (container / environment-state / agent-session) have different consistency models; a naïve capture can produce a container snapshot and a DB snapshot that disagree about a write in flight. The Branch lifecycle therefore quiesces the agent first, then snapshots environment → container → session in order.

​

Adaptations — the decision log

​

Appendix — research validation

• Agentic RL = a temporally-extended POMDP — definitionally a branching structure. (The Landscape of Agentic RL, 2509.02547)
• Tree-structured rollouts yield process supervision from a single outcome reward and more rollouts per token/tool budget. (Tree Search for LLM Agent RL / Tree-GRPO, ICLR 2026, 2509.21240)
• All 13 surveyed RL libraries model rollouts linearly with no environment snapshot/restore — tree-native + heavy-environment snapshot is real novelty, not a reinvention.
• Rollout-as-a-service decoupled from training; the trajectory is the seam. (ProRL Agent; PrimeIntellect Environments Hub)
• Continual learning = a base policy + a persistent, evolving skill library; version and roll back the store. (MetaClaw, MemSkill, SkillLearnBench)