Orchestration

1–4. Goals, constraints, context and strategy

Use this page to understand how the Orchestrator turns confirmed intent into controlled execution. The HTTP host owns durable state, including Microsoft Agent Framework (MAF) checkpoints, and calls other owners. Its Core library validates domain rules without owning the database or starting an agent.

Keep three questions separate:

1. What work is authorized? Typed outcome, workflow, and WorkPlan decisions establish confirmed scope.
2. What work is ready? The planner combines dependencies, checkpoint progress, joined results, and capacity limits.
3. What execution is current? Owner fences, runtime registrations, placement, and action grants identify the active attempt.

A model can propose a decision. It cannot replace typed approval, create its own resource pin, or mark a platform gate complete.

5. Building blocks and state ownership

Project backlog records carry task content, an open/closed lifecycle, a typed repository issue link, and optimistic task and graph revisions. Content writes use the existing Orchestrator backlog project lock and do not require or reserve a run. Atomic execution claims continue to use accepted run selection and prerequisite evidence. See Project backlog content for the authorized routes, bounds, and lifecycle rules.

Periodic repository issue sync

The optional IssueSyncWorker polls due project settings every five seconds. It runs at most 16 due projects per poll, serially, with a database advisory lock per project across service instances. Unconfigured and disabled projects are untouched. Each project cycle has its own persisted interval, item bound and deadline. It reads one bounded page of repository issues and reconciles at most maxItems backlog tasks; cursor progress survives restarts. A page includes closed issues for mapping recovery, but only previously unmapped open issues are imported. Pull requests are excluded.

sequenceDiagram
    participant Worker as Orchestrator issue-sync worker
    participant Identity as Identity Broker
    participant Projects as Projects owner
    participant GitHub
    Worker->>Identity: Dedicated mTLS grant token request
    Identity->>Projects: Dedicated mTLS current creator/tenant/project proof
    Projects-->>Identity: Current Owner and selected repository
    Identity-->>Worker: Single-cycle repository installation token
    Worker->>GitHub: Bounded list, read, create or patch issues
    Worker->>Identity: Revalidate grant before protected writes
    Identity->>Projects: Recheck Owner and repository
    Worker->>Worker: Revision-checked backlog write and durable baseline

The new issue-sync authority is not a run grant. Identity owns a revocable grant with one creator, tenant, project and selected GitHub App repository. It rechecks that the grant is active, the creator is still a current project Owner, and the repository/connection is still selected. The worker validates this evidence before effects and inside revision-checked backlog writes. GitHub credentials live only for a single cycle and are invalidated afterward; no user bearer token is saved. Existing run-token issuance is unchanged. The creator or a current tenant administrator can revoke the grant. Identity persists credential-free, append-only requested and minted audit evidence with grant ID and owner before returning any minted credential.

Mappings live in the backlog's unique repository issue link and in sync receipts. Baselines support field-by-field three-way reconciliation: an unchanged local field follows GitHub; a changed local field wins, including simultaneous edits. Both sides can close and reopen work. This is a deterministic policy, not a conflict prompt. Archived tasks retain their mapping and are not exported or edited.

Before creating an issue the worker persists a pending receipt and includes a stable HTML comment in the remote body. A lost response is recovered by finding that marker in subsequent bounded pages, including closed issues. It never blindly repeats an uncertain create. If a remote issue cannot be found after an uncertain write, status remains issue_sync_create_outcome_pending; it does not claim successful completion. Definitive rejections, transient reads and updates are retried on the next cycle. Partial writes are not rolled back remotely or labelled successful. Changing a project's connected repository fails closed rather than remapping existing work to another repository.

The owner routes and exact settings bounds are documented in the API reference. Migration 027, the Identity grant migration, runtime grants and dedicated service TLS profiles are separate prerequisites. The source contract does not establish that those prerequisites have been applied or that a cycle has run in a deployment.

Source	Responsibility
CoordinationEndpoints.cs and CoordinationOwnerStore.cs	Session, decision, gate, message, turn, roots, fences, and outbox
WorkflowGenerationEndpoints.cs and WorkflowGenerationJobStore.cs	Durable workflow-generation jobs and typed draft results
CoordinatorDecisionOwnerStore.cs	Atomic decision, gate, grant, and outbox writes
MafExecutionPlanner.cs	Deterministic associations and ready-frontier calculation
MafExecutionDispatchEndpoints.cs	Explicit dispatch, durable intents, host sends, and result joining
MafExecutionCheckpointStore.cs and PostgresMafCheckpointStore.cs	MAF checkpoint binding and persistence
SessionSuspendResumeCoordinator.cs	Ordered suspend and resume phases
RuntimeRegistrationOwner.cs and RuntimeUsageSourceOwner.cs	Current runtime identity and SDK evidence
WorkflowContracts.cs, WorkflowDefinitionValidator.cs, and WorkPlanValidator.cs	Workflow and plan types and validation
ExecutableActionGuard.cs and readiness evaluators	Current grants, durable receipts, prerequisites, and schedules

The endpoint and store files are under services/orchestrator/Agentweaver.Orchestrator/. Domain contracts and validation rules are under services/orchestrator/Agentweaver.Orchestrator.Core/.

List stored project issues and pull requests

The Orchestrator serves two project lists from stored data, without a provider call. ProjectIssueReadStore reads backlog tasks that issue sync linked to a repository issue. ProjectPullRequestReadStore reads the pull requests that the project's runs created or reused. ProjectSourceControlEndpoints checks Projects read authority before and after each read, so a caller that loses permission during the read gets 403 and no items.

Pull request records require migration 032. The reviewed maintenance plan applies it with its exact runtime grant; startup also accepts a complete pre-032 schema during rollout. Before maintenance applies it, the list returns 503 pull_request_records_unavailable. After that, it lists only pull requests recorded from that time on. See the API reference.

6. Runtime view

Start the root AgentHost

After Orchestrator commits an accepted root and admits the first message, it saves a runtime-start intent and queues a bounded background continuation. Direct root acceptance uses the same path. Invocation-bound root admission reaches it only after the setup receipt, accepted run selection, root, and first message pass their current owner checks. Startup does not create a model turn.

The continuation uses the accepted run selection and its execution fence. It asks Environment to register the run's Environment, apply the selected egress policy, create a private 1 GiB ReadWriteMany workspace at /workspace, and provision the first Sandbox placement. The workspace uses Strict consistency and Retain for both reclaim and owner deletion. Environment returns the exact lease, placement, and selected default source profile.

Orchestrator then creates the current runtime registration, requests Broker bootstrap, and calls Environment's authenticated POST /internal/runtime/v1/activate route. Ready is stored only after Environment validates the current registration, placement, AgentHost receipt, and readiness evidence. The response and session status report pending, ready, or blocked; pending is not proof that the Host is ready.

After startup reaches ready, a session-status read rechecks current readiness and returns the verified runtime instance, registration revision, accepted selection, execution fence, model selection and source mode, image digest and platform, and Pod UID. This readback works for an ordinary run without a P1 setup claim. P1 invocation IDs remain optional and do not stand in for runtime evidence.

The startup intent stores operation IDs, owner fences, and non-credential receipts. It does not store bearer credentials. A retry needs fresh authenticated run authority. Replaying a start request can queue recovery without sending the first message again. Terminal cleanup releases only owned placements; it does not delete retained workspace data or start a replacement Host. First-start provisioning rechecks current authority before it completes the Sandbox lease. Resume provisioning records a provider effect that already returned before the final authority check. Neither path reports readiness after authority is lost; both preserve the exact resource for cleanup.

The continuation and its durable state live in RuntimeRunStartOrchestration.cs and CoordinationRuntimeRunStartStore.cs. Environment owns first placement in EnvironmentRuntimeRunStartManager.cs. The real activation route is in EnvironmentRuntimeBootstrapEndpoints.cs.

Start child runtimes

Dispatch saves a child-start intent in the same transaction as the committed spawn. The intent binds the child to its parent, work-plan item, selected agent and model, accepted selection, execution fence, and current owner. The intent and its stable operation IDs remain available if a reply is lost. No bearer credential is saved. Each intent is also a durable cleanup obligation before placement starts. Completion or an authorized terminal cleanup request moves the intent to pending cleanup, even when startup stopped before a reply.

After commit, dispatch queues a child-specific continuation. A later authorized dispatch can resume the same intent; the continuation does not depend on an outbox consumer. Environment provisions the child's placement. Orchestrator creates the matching registration, obtains Broker bootstrap, and asks Environment to activate the Host. Each stage records its intent and receipt before the next stage. Current owner, selection, registration, image, model, and placement checks must pass before dispatch sends a task message.

childRuntimeStartup reports the bounded startup state for each association. pending or blocked does not mean that a model turn was sent. A missing or stale readiness receipt stops the send. Retrying requires fresh run authority and reuses the saved operation IDs; an expired queued credential cannot authorize another resource or turn.

Dispatch returns while another owner holds the child-start lease. If the response reports pending, retry the same dispatch request with the same decision version and WorkPlan ID. The retry resumes the saved child-start intent; it does not create another child or placement.

If Broker delivery fails after the child placement and registration are saved, the child intent becomes blocked at retirement. Environment has no child retirement route for this case. Do not retry the used bootstrap operation. Keep the placement and registration as cleanup obligations until authorized terminal cleanup retires the Environment resource and revokes the registration. If Environment cannot confirm retirement, cleanup remains pending.

Clean terminal runs

Cleanup applies to completed, failed, indeterminate, and cancelled runs without changing their terminal outcome. Completion records the startup obligations. An authorized cancellation request can also request or resume cleanup for an already-terminal run. You can supply expectedTerminalState and expectedTerminalFence together to require an exact match with the current run. Cancellation, including an exact retry, requires a fence later than the startup fence.

The version 2 cleanup projection binds the current terminal state and fence to one startup intent, selection, and resource receipts. Environment rechecks that projection and the exact Sandbox resource and provider generation before retirement. If a placement reply was lost, it recovers the lease by the saved operation key and checks any saved registration against that resource. Orchestrator rechecks authority and resource identity before it revokes a registration or commits cleanup. An unknown or conflicting resource remains pending.

Cleanup processes children before the root. Untracked children, unmatched active registrations, unresolved native work, or an unfinished child keep the aggregate pending. runtimeCleanup on run and session status reports state, total, pending, and complete; an unknown resource or unresolved work contributes a pending aggregate entry, not an exact resource count. Workspace data remains retained. Projects setup cleanup already retains a project with any run selection, so it does not archive a project while its owned resources are unresolved.

Terminal cleanup version 2 requires Orchestrator migration 047 and its enabled recovery guard. The guard prevents recovery from advancing a failed or indeterminate run after cleanup has been admitted. Until the guard is active, version 2 cleanup fails closed; the legacy version 1 root-cancellation path still requires the advanced cancellation fence. See the cleanup contract for the coordinated rollout.

Replace a coordinator root Host

The run-bound replacement request names the current runtime instance, registration revision, execution fence, accepted-selection hash, and one operation ID. Orchestrator accepts it only for the current coordinator root and after it verifies a completed native-turn receipt. The request ID stays the same when the caller checks or retries an uncertain operation.

Orchestrator records the replacement operation and cleanup obligation before provider effects. Environment retires the old Host placement before it creates the new Pod and lease. The new placement uses the same workspace volume and resource/data generations. Orchestrator creates a new runtime registration without changing the accepted selection or execution fence, then uses the normal Broker bootstrap and Environment activation paths.

The replacement reaches ready only after the new Host restores the exact acknowledged SDK cache and reports a complete reconstruction interval with no native events or tool effects. The status route returns the saved operation. pending or cleanup_pending means the caller must keep checking that operation; it must not start another replacement. Cancellation follows the saved attempt and keeps unresolved resource cleanup visible.

After replacement, an exact retry of the original casting request is checked against the completion receipt and the saved replacement lineage. Orchestrator returns the original completion with current-registration replay evidence. It does not send another native request or repeat a tool effect. These operations are implemented in RuntimeRunStartOrchestration.cs, CoordinationRuntimeRunStartStore.cs, and ProjectRunEndpoints.cs.

sequenceDiagram
    actor CLI as Native CLI
    participant Gateway
    participant Core as Orchestrator
    participant Environment
    participant Broker
    participant Host as AgentHost
    participant Events
    Note over CLI,Events: Source flow; it does not prove a deployed acceptance run.
    CLI->>Gateway: Submit one approved read-only casting request
    Gateway->>Core: Forward current run-bound request
    Core->>Host: Admit native root turn
    Host->>Events: Append user, assistant and cache material
    Events-->>Host: Return typed material acknowledgments
    Host-->>Core: Return native completion evidence
    Core-->>Gateway: Return proposal and stored completion receipt
    Gateway-->>CLI: Return nativeCompletion
    CLI->>Gateway: Read user and assistant turn text
    Gateway->>Events: Check current read authority and material binding
    Events-->>Gateway: Return the bound turn text
    Gateway-->>CLI: Return authorized text
    CLI->>Gateway: Request Host replacement with current binding
    Gateway->>Core: Forward stable operation ID and expected values
    Core->>Environment: Retire the exact old placement
    Environment-->>Core: Return release receipt
    Core->>Environment: Provision a new placement on the retained workspace
    Environment-->>Core: Return new lease and Pod identity
    Core->>Broker: Bootstrap the new runtime registration
    Broker-->>Core: Return current bootstrap receipt
    Core->>Environment: Activate the new Host
    Environment->>Host: Configure and resume the saved SDK cache
    Host->>Events: Read the exact acknowledged cache reference
    Events-->>Host: Return cache bytes
    Host-->>Environment: Return readiness and empty reconstruction evidence
    Environment-->>Core: Return current readiness
    Core-->>Gateway: Return ready replacement summary
    Gateway-->>CLI: Return the saved operation state
    CLI->>Gateway: Retry the exact casting request
    Gateway->>Core: Forward the original request identity
    Core->>Core: Validate completion and replacement lineage
    Core-->>Gateway: Return original completion and no-call replay evidence
    Gateway-->>CLI: Return nativeReplay without a new native send

Generate a workflow draft

The Orchestrator owner provides these routes:

• POST /api/projects/{projectId}/workflows/generate
• GET /api/projects/{projectId}/workflows/generation-jobs/{jobId}
• GET /api/projects/{projectId}/workflows/generation-jobs/{jobId}/result

The owner requires an Idempotency-Key, the projects.orchestrator scope, current project write and Orchestrator authority, and the exact accepted coordinator root. It checks the current run selection, model registration, execution fence, and decision state before it sends work. The request names only the run, session, expected decision version, and prompt. The owner derives the tenant, actor, selection, model, fence, and native request ID.

The owner saves the job and native request ID before it sends a model turn. An identical authenticated POST returns the same job. A changed request with the same key returns 409 idempotency_key_conflict. The owner sends the saved prompt to the accepted root Host in read-only mode. It validates the native completion and the acknowledged user, assistant, and cache material before it stores a typed WorkflowDefinition. A completed job without a valid stored definition and persisted native completion evidence is an error.

Acceptance returns 202 with separate status and result URLs. The status route returns a job envelope. The result route returns 409 workflow_generation_not_complete until the job completes. Both read routes recheck the owning actor, project, and current accepted selection. They do not start model work. Invalid definitions and roles outside the accepted team fail with typed errors. A generated definition remains a draft; it does not select or confirm a workflow or plan.

If the work queue is full when a job needs scheduling, POST returns 503 workflow_generation_queue_full. The job can already be stored. Retry the identical request with the same Idempotency-Key and current authority to schedule or reconcile that job. If the run becomes terminal while the job is being saved, POST can return 409 workflow_generation_run_unavailable.

Migration 046 adds the job table and closes queued work when a run is cancelled. It preserves an admitted native turn as unresolved until completion is verified. Cleanup stays pending without holding a runtime-start worker. An authorized retry of the cancellation can schedule cleanup again after an in-flight worker verifies completion. Cancellation retries do not independently verify stored proof. If no worker verifies the turn, cleanup remains pending, even when a receipt was stored. If proof is lost before it is stored, the current contract cannot recover it after cancellation. Cleanup does not start another turn.

For an active run, an identical POST can attempt to reconcile the saved native request after a lost reply or a temporary storage failure. The result can remain indeterminate if completion evidence is unavailable. The owner does not create a second request ID or make an automatic paid retry. The runtime database role needs SELECT, INSERT, and UPDATE on the job table. It has no DELETE grant. The migration is optional during rollout; readiness rejects an applied but incomplete schema.

These are Orchestrator owner routes. The read-only CLI API case does not exercise them. The diagram shows the owner source flow, not a deployed or live-model test.

sequenceDiagram
    actor Caller
    participant Core as Orchestrator
    participant Projects
    participant DB as Orchestrator PostgreSQL
    participant Host as Accepted root AgentHost
    participant Events
    Note over Caller,Events: Gateway admission remains pending PR 4.
    Caller->>Core: POST workflow request with Idempotency-Key
    Core->>Projects: Check current authority and accepted selection
    Core->>DB: Save job and stable native request ID
    alt Work queue is full
        Core-->>Caller: 503, retry the same request and key
    else Job is queued
        Core-->>Caller: 202 with status and result URLs
        Core->>Projects: Recheck authority, root, selection, fence and decision
        Core->>DB: Persist send-admitted phase
        Core->>Host: Send the same read-only native request
        Host->>Events: Append user, assistant and cache material
        Events-->>Host: Return material acknowledgments
        Host-->>Core: Return native completion evidence
        Core->>Events: Read and verify acknowledged material
        Core->>DB: Store validated typed draft and completion evidence
        Caller->>Core: GET status or result
        Core->>Projects: Recheck actor and project authority
        Core-->>Caller: Return job status or completed result
    end

Understand workflows, plans, and gates

A workflow is a versioned catalog of steps. Its step mode defines who creates the work:

Mode	Meaning
Fixed	The workflow prescribes task, role, phase, isolation, and outputs.
Open	The coordinator proposes bounded WorkPlan items for the step.
Platform	A named platform executor owns the gate; ordinary agent work cannot replace it.

The gate enum describes domain vocabulary, not a promise that an executor is available. Dispatch reports steps that have no available executor.

Scribe is available when its Knowledge owner is configured. The selected gate snapshots successful accepted child results into actor- and producer-bound checkpoint intents before Knowledge I/O. It records compact Memory revision receipts under the same execution fence and only succeeds after all receipts are durable. Restart replay rechecks current authority and exact persisted content; it does not classify results, create Decisions, or auto-approve context. See Scribe workflow-output.

WorkPlanValidator checks the exact workflow revision, step identity, cardinality, role, selected agent and model, isolation, capabilities, output paths, and dependency graph. Model-proposed items belong only to open steps. Validation returns either a snapshot or structured issues, never a partial snapshot. For conflicting proposed outputs, the validator adds deterministic dependency edges so writes do not race.

Nonempty work, fixed work, and command-backed BuildTest steps require the appropriate accepted isolation context. The owner resolves an existing resource through a registered Sandbox adapter; it does not invent or provision a resource at this boundary.

OpenPullRequest is available when the private authenticated Source Control owner is configured. Dispatch accepts an exact workspace locator for each such step. It saves the request and source binding before calling the owner. Retries reuse the saved request and operation identity, and dependent work waits until a validated pull request receipt is stored. The owner records the operation before contacting the provider. Recovery reuses the exact pull request across open and closed states; a missing or ambiguous result never triggers another create. Managed workspace changes wait while the owner holds the revision lease for the write. A changed source, workspace, or decision leaves the gate incomplete. Merge is available through the same authenticated Source Control owner. Dispatch persists its idempotency key, repository pin, workflow binding, PR number, and expected head before calling /merge-intents. It derives the PR only from one completed OpenPullRequest ancestor or accepts an explicit number and head pair. If the owner creates an approval gate, dispatch returns with the Merge checkpoint still running; dependents remain blocked until an exact approval receipt is current. A later dispatch reuses the saved intent, calls /merge-intents/{id}/execute, and stores only the owner-verified merge SHA. Changed authority or head, ambiguous results, and interrupted execution fail closed without repeating a merge effect.

RubberDuck is available only when the current coordinator-root AgentHost registration and action-grant owner are available. It is a platform executor, not an ordinary work-plan child. The dispatcher stores an immutable intent before sending the exact recorded ancestor results and platform receipts to that AgentHost in read-only mode. The intent binds the accepted workflow revision, actor, selection, execution fence, and current runtime registration. The owner turn uses the checkpointed operation ID, and its completed response uses a stable journal event ID so a restarted host can return the recorded answer without running another review. If only part of the turn was recorded, the host refuses to send it again; the gate stays incomplete.

Only a response whose first nonempty line is PASS can release dependent work. REQUEST_CHANGES, a malformed verdict, stale evidence, a changed owner, or unavailable owner leaves the gate failed or incomplete and keeps dependents blocked. Recovery reports the operation, evidence hash, verdict, and any stable failure code. Other platform gates still require their own registered executors.

Candidate and confirmed plans are separate states. Pending gates prevent decomposition or dispatch according to transition rules. Gate answers bind a request ID, actor, and fence; exact replay returns the saved result. Acknowledging a message records receipt. It does not approve an outcome, confirm a workflow, or complete work.

Create and confirm a plan

Before proposing work, the run needs an accepted Projects selection and current authority. The selected workflow and plan must match that run context.

For a generated workflow and a claimed task, use this order:

1. Generate the workflow draft. Finish the generation job before you propose a WorkPlan. A generated definition is still a draft.
2. Capture and claim the task. Claim it on the run's existing accepted coordinator root. An exact retry reuses that claim and root.
3. Propose and confirm the outcome. Core validates the typed outcome specification. Clarifying questions and outcome confirmation become explicit gate state.
4. Select and confirm the workflow. Selection uses the authorized workflow catalog. A generated definition requires first-use confirmation.
5. Propose and confirm the WorkPlan. Core validates the full plan against the selected workflow and run context. The owner saves the candidate and its confirmation gate. Scope-changing revisions add another confirmation boundary and consume the configured revision-attempt allowance.
6. Dispatch the accepted plan. Dispatch provisions ready child runtimes, verifies readiness, and sends the model-backed work.

Do not propose the outcome again after workflow selection. A new outcome proposal replaces the outcome and clears the selected workflow and WorkPlan; the run must pass through workflow selection and plan confirmation again.

CoordinatorDecisionOwnerStore serializes transitions against current owner state. The winning transaction commits the decision version with its gate, action grants, and decision outbox. The owner rechecks authority around waits and before protected writes.

Owner routes are rooted at /api/projects/{projectId}/runs/{runId}/coordination; Gateway exposes versioned public routes. Use contracts and endpoints for request shapes.

sequenceDiagram
    actor Caller
    participant O as Orchestrator
    participant DB as Orchestrator PostgreSQL
    participant Env as Environment
    participant Host as Child AgentHost
    participant Events as Events and Sessions
    Caller->>O: Generate workflow draft
    O->>DB: Save native request and typed draft
    Caller->>O: Capture and claim task on run B
    O->>DB: Reuse accepted root; save claim and idempotency
    O-->>Caller: Same claim and root on exact retry
    Caller->>O: Propose and confirm outcome
    Caller->>O: Select and confirm generated workflow
    Caller->>O: Propose and confirm WorkPlan
    Caller->>O: Dispatch accepted WorkPlan
    O->>DB: Save child-start intent and checkpoint
    O->>Env: Provision child placement
    Env-->>O: Placement and readiness evidence
    O-->>Caller: Pending while startup lease is held
    Caller->>O: Retry the same dispatch request
    O->>Host: Send associated child task
    Host->>Events: Append native material and usage events
    Events-->>Host: Acknowledge native events and material
    Host-->>O: Return child-turn completion receipt and item-binding hash
    O->>DB: Store receipt with checkpoint result
    O-->>Caller: Return bounded completionEvidence
    O->>DB: Join claim, current item hash, checkpoint and output witness
    O-->>Caller: Return taskCompletion only when every join passes

Dispatch ready work

Before dispatch, provide an explicit actions/dispatch request with the expected decision-state version and WorkPlan ID. The plan must still be confirmed.

1. Restore the bound MAF checkpoint and validate the plan again.
2. Reconcile child lifecycle records with saved dispatch intents and joined results.
3. Compute the ready frontier from step and item dependencies.
4. Apply child, concurrency, wall-time, and runtime admission limits.
5. Commit child records, stable dispatch identities, pending intents, and checkpoint progress before sending work.
6. Recheck authority, runtime registration, and host readiness; then send the ready batch.
7. Validate each host response against its native context and persist the joined result.

Fixed work gets an immutable activation association. Open work uses its confirmed WorkPlan item. Both resolve to deterministic child-session IDs. Dispatch messages also have stable IDs, so retry does not create a new logical send.

Later explicit dispatch requests reconcile saved progress and expose newly ready dependencies. This is request-driven execution, not an automatic scheduler. Command-backed BuildTest delegates resource work to Environment while Orchestrator retains the accepted command and checkpoint. Source Control output and merge evidence use separate owner paths.

For a completed native child turn, the runtime returns a receipt that binds request, prompt, and output hashes to the plan association, child session, runtime registration, accepted selection, execution fence, source and image, model selection and actual model, native completion events, usage events, and acknowledged user and assistant material. For an open WorkPlan item, the receipt also carries a hash of the full item that was dispatched. Orchestrator compares that hash with the current confirmed item before it derives task completion, so a later reassignment or item change cannot reuse the old result. Orchestrator joins each field to the persisted source, native observation, and accounting status before it stores the receipt with the checkpoint result. Task completion requires priced accounting. The dispatch response returns bounded completionEvidence entries; it does not expose prompt or output text through the receipt.

If the first response is lost after the child result is saved and an independent-review gate is pending, the same dispatch request reads the saved result and receipt without sending another child turn.

The optional taskCompletion projection appears only when the owner joins a unique task claim on the root to the confirmed one-item WorkPlan, completed checkpoint, child receipt, and output witness. No request field can set task state to done. A pending startup, incomplete checkpoint, missing or stale receipt, or failed join does not produce task-completion evidence. See dispatch completion and startup readback for response fields and limits.

Promote a plan into backlog tasks

Instead of dispatching a confirmed plan, the run owner can promote it into project backlog tasks with POST .../backlog/promotions. Promotion is allowed only before the run starts execution: the run must have no registered children and no MAF dispatch records.

The Board first reads a preview of the confirmed plan. This read does not change the backlog. The user must confirm before the Board sends the promotion request with the graph revision it currently displays.

1. Lock the project backlog graph rows.
2. Lock the run's owner binding row and validate the confirmed plan, decision version, execution fence, and selection again.
3. Add one task per plan item and one edge per item dependency.
4. Write the promotion record and one origin record per task.
5. Recheck authority and commit.

Dispatch takes the same owner binding lock and rejects a promoted plan, so a plan is either promoted or dispatched, not both. The decision state is not changed. A retry with the same idempotency key returns the stored promotion. For a completed identical retry, the owner checks current authority but does not need to read current decision or selection context again. Tasks captured directly have no origin record. For request and response shapes, see Plan promotion and task origin.

Enforce policy and runtime authority

Core's ExecutableActionGuard checks a current owner-issued grant before calling a protected operation. It evaluates policy, stores source evidence, waits for durable Events acknowledgment, and rechecks current authority before the effect. A policy result is evidence, not a grant.

Runtime registration is a separate boundary. Orchestrator compares the accepted selection, active session and work association, execution fence, and Environment placement before returning current runtime context. An earlier registration record alone does not authorize a turn.

Recover from uncertain execution

• A stale decision version or execution fence conflicts instead of applying work to a newer attempt.
• Missing Sandbox negotiation or runtime prerequisites block dispatch; they do not select a synthetic resource.
• A pending dispatch intent keeps its identity across retries. Missing or uncertain result evidence is not success.
• Owner-reported failure or indeterminate execution advances the run fence. Explicit recovery checks the expected fence and run version before advancing again and restoring eligible logical turns.
• Suspend and resume use a persisted phase machine. Suspend orders fencing, draining, checkpointing, journal and workspace flushes, manifest commit, and placement release. Resume orders manifest validation, environment readiness, egress verification, fence advancement, and dispatch. A retry cannot skip a required phase.

A workflow checkpoint and native SDK cache are different recovery inputs. Restoring conversation context does not replay external effects.

The opt-in checkpoint-recovery path accepts two boundaries. For a pending gate, it requires the exact gate and no unresolved child dispatch, child association, or platform-effect intent. It advances the fence once, keeps the same gate request, and replaces the Host through Environment. The replacement must load authenticated native-cache or journal material before Orchestrator reports ready.

For a saved child join, Orchestrator checks the saved child boundary, records a frozen manifest, advances the fence, and writes a checkpoint with recovery lineage. It keeps dispatchHeld true while it restores saved children with proven-unsent native work. A pending child with no native row follows the existing path. A prepared native origin is eligible only when its immutable row still has owner revision 1, no report, and no accounting progress, and the owner verifies any prior effective binding. The continuation registers the root and every required child before it bootstraps any child. It does not send a new task message during recovery. AgentHost must restore a compatible native cache or saved turn history before it can report readiness. Configuration-only, zero-turn child state is not supported by this recovery path, so the owner keeps the hold active.

If a child completed native work before the checkpoint joined its result, Orchestrator reads the original prompt and result material and checks their session, event, and content bindings. It also checks the saved registration, execution fence, and native accounting against the frozen manifest. The final owner transaction adds the result and completion receipt to one successor checkpoint, completes the child session, and releases the hold only after all required readiness checks pass. A missing or mismatched result rolls back that transaction and keeps the hold active. Recovery does not run the completed child again.

For a prepared pending child, the final ready transaction also appends one immutable native-child-rebind receipt before it releases the hold. The receipt links the original dispatch to the ready replacement and the recovery checkpoint. It does not update the origin row or create another dispatch. Checkpoint reads accept a later current leaf only when its lineage proves descent from the receipt anchor and the saved child work remains unchanged. After the hold is released, ordinary dispatch uses the same saved message, association, and prompt through the effective binding, then follows the usual native report and accounting path. ready still means that the Host restored the boundary; it does not mean that the child task or run is complete.

One committed OpenPullRequest effect is also recoverable at a saved child join. Before advancing the fence, the owner requires exactly one running intent with no receipt and an exact matching committed Source Control operation and stored pull request. Missing, pending, uncertain, or mismatched operation evidence blocks recovery before the fence changes. After recovery is admitted, the owner reads the stored pull request under current authority, verifies its result fingerprint, and does not create another pull request or rewrite the original operation. Orchestrator then records the receipt from the original intent. Until that receipt is saved, recovery reports reconcileCommitted; the dispatch hold remains active through the normal readiness checks. Other platform-effect history remains unsupported at this boundary. See the committed pull request recovery flow.

If a child-join recovery is blocked at Host activation after the replacement registration set is complete, a new operation can supersede it. The owner requires exact retirement and registration receipts, the same saved child and plan identity, and owner evidence that each pending dispatch has either no native row or the same prepared, revision-1 origin with no report or accounting progress. The owner verifies the prior effective binding and includes the origin fingerprint and no-send state in its supersession proof. It records the link with the new fence, checkpoint, and hold in one transaction. The old operation stays blocked and cannot release the new hold. The new operation restores the root and required children again. It appends any rebind only after the replacement is ready. If the owner cannot verify the saved no-send and replacement evidence, it rejects the new operation and keeps the hold active.

Guarded owner writes fail while the hold is active. The owner rejects unsupported platform-effect history instead of replaying it. Retry a pending operation with the same recovery request and idempotency key. Neither boundary causes the owner to answer the gate or start a fresh execution.

The legacy owner-state recovery path and ordinary completed-turn Host replacement remain separate. readRunStatus and readSessionStatus show the recovery step and result; replayRunEvents reports committed progress. Reads do not continue a pending recovery.

Understand readiness evaluation

The Core backlog evaluator checks exact graph and task revisions against owner-projected prerequisite execution and output evidence. A dependency label alone does not make a task ready.

The schedule evaluator computes daily, weekly, and monthly occurrences in UTC. It is a pure function, not a hosted scheduling loop.

7–9. Deployment, shared concepts and decisions

The HTTP owner runs as a shared service over its PostgreSQL schema. Core is a library in that host; MAF checkpoints are owner state, not an SDK cache. AgentHost placement belongs to Environment and explicit dispatch reaches the registered runtime. See the deployment view.

Cross-owner calls preserve current Projects authority, Identity purposes and Events evidence. Local checkpoints and stable operation IDs reduce duplicate logical work, but do not make external effects transactional.

ADR 0002 records accepted/source-implemented issue-sync delegation, not deployment. The platform ADR remains Proposed. Neither status is inferred from this runtime inventory.

10–12. Quality, risks and glossary

Context and stimulus	Response and measure	Verification state
Dispatch restarts with a saved pending intent	Restore the same child/operation identity and reconcile result evidence	Checkpoint/owner restart tests; deployed recovery is separate
Plan promotion races dispatch	Shared owner binding lock admits promotion or execution, not both	Backlog promotion/store tests
RubberDuck returns REQUEST_CHANGES or malformed output	Keep dependents blocked; only first nonempty PASS can release them	Executor/dispatch tests
Issue create response is lost	Recover the stable marker; unresolved outcome stays issue_sync_create_outcome_pending	Controlled issue-sync/store tests

A domain gate name is not an available executor. The dispatch owner must keep unsupported or incomplete gates visible rather than replace them with model work. External response loss requires reconciliation; an automatic scheduler or blind retry would not resolve it.

Terms: workflow prescribes steps; WorkPlan proposes bounded work for open steps; gate records a typed confirmation; checkpoint retains progress; fence rejects an older attempt.

Related content

• Events and sessions explains journal and message ownership.
• Agent runtime explains bootstrap and guarded SDK work.
• Environment and sandbox and Source Control explain resource and repository effects.
• Contracts and endpoints records the API surface.
• Orchestration design records broader design decisions.