| # Harness Flow |
| |
| ## Configuration |
| |
| All model parameters can be configured in [model_config.dart](../packages/frontend/lib/features/agent/config/model_config.dart) and all harness parameters can be configured in [harness_config.dart](../packages/frontend/lib/features/agent/config/harness_config.dart). |
| |
| ## Project Structure & Terminology |
| |
| All agent code lives under `packages/frontend/lib/features/agent/`. |
| |
| ### Folder Overview |
| |
| | Folder | Purpose | |
| | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | |
| | `config/` | Static constants (`HarnessConfig`, `ModelConfig`) — run-loop limits, timeouts, model name, thinking budget. | |
| | `data/` | Repository interfaces (`AgentChatRepository`) and their implementations (`GenkitAgentChatRepository`), plus the conversation model abstraction. | |
| | `logic/coordination/` | **Coordinators** — stateful classes that serialize, deduplicate, and orchestrate async workspace operations. See below. | |
| | `logic/policy/` | **Policies** — mostly stateless rules and evaluation logic (validation prompts, path sandboxing, runtime observation evaluation). | |
| | `logic/run/` | **Run lifecycle** — the turn loop (`AgentRunController`), session state (`AgentSession`), system prompt, and bootstrap context builder. | |
| | `models/` | Immutable value types, enums, and extensions shared across layers (`AgentAutomationResult`, `AgentToolEvent`, `AgentValidationResult`, etc.). | |
| | `tools/` | Tool definitions: the `AgentToolRegistry` that builds Genkit tools, tool I/O schemas, and individual tool implementations grouped into `search/` and `workspace/`. | |
| | `view_models/` | UI-facing view models (`AgentChatViewModel`). | |
| | `views/` | Flutter widgets for the agent chat panel, message bubbles, tool call rows, etc. | |
| |
| ### Coordinator Hierarchy |
| |
| The `coordination/` folder contains one central coordinator and three sub-coordinators: |
| |
| ``` |
| AgentHarnessCoordinator ← central: orchestrates validation gate sequence |
| ├── AgentPubGetCoordinator ← sub: serializes/deduplicates `flutter pub get` |
| ├── AgentDiagnosticsCoordinator← sub: bridges mutation generations with LSP diagnostics |
| └── AgentPreviewCoordinator ← sub: idempotent preview start/reload/rebuild |
| ``` |
| |
| The sub-coordinators don't know about each other. Only the `AgentHarnessCoordinator` knows all three and runs them in the correct order (dependency → analyzer → preview → runtime). |
| |
| ### Key Terminology |
| |
| | Term | Meaning | |
| | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
| | **Run** | One user message → agent processes it (potentially multiple model turns) → result. Managed by `AgentRunController`. | |
| | **Model Turn** | One call to `generateTurn()` producing new messages. A run can have many turns (tool calls, repairs). | |
| | **Validation Gate** | A checkpoint before the agent's final response is accepted. Gates run in order: dependency resolution, analyzer, preview load, runtime observation, preview start. | |
| | **Repair Attempt** | When a validation gate fails, the harness injects a repair prompt and gives the model another chance. Bounded by `maxRepairAttempts`. | |
| | **Mutation Generation** | A monotonically increasing counter that tracks workspace changes (pubspec edits, Dart file writes). Used by coordinators to deduplicate and sequence work. | |
| | **Coordinator** | A stateful class that serializes async operations, deduplicates in-flight work by generation, and tracks pending/completed state. | |
| | **Policy** | A (mostly) stateless class that encapsulates decision rules — e.g. whether a diagnostic is blocking, how to format a repair prompt, whether a file path is allowed. | |
| |
| ## Flow until final candidate |
| |
| The harness drives the model through a bounded validation loop. It provides tools for the model, environment observations and enforces validation. The model is free to generate tool calls and edits for up to `maxModelTurnsPerRun` turns. A turn is one call to `generateTurn(...)` that produces new messages. One turn can have tool calls, tool responses, and a final plain-text answer. When the model produces a final answer, the harness checks if code was modified. If the code was modified, the harness pipes messages from the analyzer to the model so it can validate the code. If problems are reported from the analyzer, the harness feeds a focused repair prompt to the model. To not get stuck in a loop the model is only allowed to make `maxRepairAttempts` repair attempts before the harness gives up and fails the run. Context grows incrementally — after each completed tool roundtrip and validation result, messages are appended directly to the session's model context. There is no separate batch-commit step. |
| Changes to the pubspec.yaml are treated as a special case and trigger an immediate async `pub get` call while the agent can modify other files in the meantime. The harness waits for the `pub get` to complete before it sends the analyzer messages to the model for validation. After a successful validation the harness enters the next phase where it loads the code changes into the preview and observes for runtime errors. |
| |
| ## Preview Observation and Runtime Error Detection |
| |
| After the analyzer passes, the harness loads pending code changes into the running preview and actively monitors for runtime errors. This happens inside `validateCandidateFinal()` as part of the same validation pipeline — analyzer errors, compile errors, and runtime errors all count against the same `maxRepairAttempts budget. |
| |
| ### Preview operation lifecycle |
| |
| Every preview operation (start, rebuild, hot reload) follows the same three-phase sequence inside `AgentPreviewAutomation._runPreviewOperation()` |
| |
| 1. **Serialization gate** — `waitUntilAvailable()` polls `isBusy` at 100ms intervals for up to 10 seconds. This prevents concurrent preview operations from racing (e.g. a rebuild still running when the next validation triggers a hot reload). If the timeout expires, the operation fails with a "preview is currently busy" error. |
| |
| 2. **Await the operation** — `await operation(preview)` runs the actual start/rebuild/hot reload. This has no timeout; the harness waits as long as the compiler needs. Compile errors surface as exceptions and are caught and reported as blocking preview errors. |
| |
| 3. **Runtime observation window** — After the operation completes successfully, `AgentPreviewRuntimeEvaluator.waitForNewBlockingObservation()` checks for runtime errors. It first checks immediately (the error may already be present), then waits `previewObservationWindow` (currently 1s) and checks again. This gives the Flutter framework time to build and lay out the widget tree, which is when runtime errors like `RenderFlex overflowed` occur. |
| |
| ## Flowchart |
| |
| <details> |
| <summary>Pipeline Flowchart</summary> |
| |
| ```mermaid |
| flowchart TB |
| Start(["User request"]) --> AppendUser["AgentSession.appendUserMessage()"] |
| AppendUser --> FirstRun{"First user message in session?"} |
| FirstRun -- Yes --> Tree["Append workspace tree bootstrap context"] |
| FirstRun -- No --> Step["AgentRunController loop (maxModelTurnsPerRun)"] |
| Tree --> Step |
| Step --> Generate["conversationModel.generateTurn(...)"] |
| Generate --> AppendModel["session.appendMessages(newMessages)"] |
| AppendModel --> ToolReq{"Tool requests returned?"} |
| ToolReq -- Yes --> RunTools["Execute requested tools"] |
| RunTools --> AppendToolResp["session.appendMessages(toolResponse)"] |
| AppendToolResp --> RecordEffects["Harness coordinator records mutations, pubspec changes, pending preview work"] |
| RecordEffects --> Step |
| ToolReq -- No --> Candidate{"Non-empty final text?"} |
| Candidate -- No --> Fail["Fail run"] |
| Candidate -- Yes --> Validate["Run validation gates"] |
| Validate --> Deps["ensureDependencyResolution()"] |
| Deps --> DepOk{"Dependency blocker?"} |
| DepOk -- Yes --> Repair["Inject focused repair prompt"] |
| DepOk -- No --> Dirty{"Unvalidated mutations?"} |
| Dirty -- Yes --> Analyzer["Read fresh diagnostics snapshot"] |
| Analyzer --> AnalyzerOk{"Analyzer blocker?"} |
| AnalyzerOk -- Yes --> Repair |
| AnalyzerOk -- No --> PreviewLoad["Load pending preview generation if preview is running"] |
| Dirty -- No --> PreviewLoad |
| PreviewLoad --> PreviewLoadOk{"Preview load/rebuild/hot reload blocker?"} |
| PreviewLoadOk -- Yes --> Repair |
| PreviewLoadOk -- No --> Runtime["Check runtime observations"] |
| Runtime --> RuntimeOk{"Runtime blocker?"} |
| RuntimeOk -- Yes --> Repair |
| RuntimeOk -- No --> AutoStart{"Validated mutations and preview not running?"} |
| AutoStart -- Yes --> StartPreview["Start preview"] |
| StartPreview --> StartPreviewOk{"Preview start blocker?"} |
| StartPreviewOk -- Yes --> Repair |
| StartPreviewOk -- No --> Finish["session.finishRun(assistantMessage)"] |
| AutoStart -- No --> Finish |
| Finish --> Done(["Run complete"]) |
| Repair --> RepairBudget{"Repair attempts left?"} |
| RepairBudget -- Yes --> Step |
| RepairBudget -- No --> Fail |
| ``` |
| |
| </details> |
