Skip to content

Dispatcher & Subagent Orchestration Spec

This document details the mechanics of the "Watcher" (Dispatcher) service, designed to bridge passive Obsidian editing with the multi-agent system securely and efficiently.

Core Problem

A naïve file-watcher (like the basic pi-piqo implementation) triggering a new LLM session for every @agent marker can quickly exhaust API rate limits, crash the local machine (if using local models), and pollute context.

Solution: Grouped Forking & Dispatch

The Dispatcher monitors vault changes but applies a grouping algorithm before spawning subagents via Pi's subagent tool.

1. Marker Syntax & Routing

Markers in markdown files follow this format: > @agent-[Role]:[OptionalTaskID] [Instruction]

Examples: * > @agent-Miner Summarize this section. -> Routes to the Miner agent. * > @agent-Broadcaster Create a podcast. -> Routes to NotebookLM controller. * > @agent-Miner:graph Extract entities here. -> Forces an isolated fork for the Miner.

2. Grouping Logic

When a file save event occurs (debounced by 1000ms):

  1. Scan & Extract: The Dispatcher scans the file for all @agent-* markers and extracts the surrounding context (e.g., 30 lines before, 10 lines after).
  2. Group by Target: Markers are grouped by the specific string agent-[Role]:[OptionalTaskID].
    • If a file has 5 @agent-Miner markers, they are grouped into a single execution payload.
    • If a file has @agent-Miner and @agent-Broadcaster, they form two distinct groups.
    • If a file has @agent-Miner:task1 and @agent-Miner:task2, they form two distinct groups.
  3. Compile Context: For grouped markers, the Dispatcher stitches the extracted contexts together, ensuring no duplicate overlap.

3. Execution via pi-subagents

The Dispatcher leverages the pi-subagents extension to launch the work:

  • Subagent Config: The Dispatcher invokes the subagent tool using the tasks (parallel execution) or chain (sequential) parameters.
  • Context Isolation: By default, subagents are spawned with { context: "fork" }. This gives them the historical context of the project up to that point, but isolates their scratchpad thinking from the main session.
  • Concurrency Control: The Dispatcher explicitly sets a concurrency limit (e.g., max 2 or 3 parallel agents) when calling the subagent tool to prevent hardware/API overload.

4. Resolution & File Editing

  1. When a subagent completes its task, it returns the generated output.
  2. The Dispatcher matches the response to the original file location.
  3. It atomically replaces the original > @agent-... marker with the subagent's response, wrapped in a recognizable block (e.g., <!-- agent-response --> ... <!-- /agent-response -->).
  4. If a grouped task handled 5 markers, the subagent is instructed to return a structured JSON response mapping answers to markers, allowing the Dispatcher to update all 5 locations simultaneously.

Example Payload Construction

If a file contains: 

# Section A
> @agent-Miner Find related entities.

# Section B
> @agent-Miner Cross-reference this with the UCL.

The Dispatcher creates one subagent task for the Miner: 

{
  "agent": "Miner",
  "task": "You have 2 requests from the user's document. Answer each individually.\n\nRequest 1 (Line 10): Find related entities.\nContext: # Section A\n\nRequest 2 (Line 20): Cross-reference this with the UCL.\nContext: # Section B",
  "context": "fork"
}

HTTP Bridge (Obsidian → pi)

In addition to filesystem watching, pi-vault-mind runs a lightweight HTTP server on http://127.0.0.1:11435 (configurable via wiki.httpPort).

Endpoints

Method Path Purpose
GET /vault-mind/status Health check: running state, vaults, uptime, dispatch records
POST /vault-mind/scan Scan a file for @agent markers. Body: { "file": "/path/to/note.md" }
POST /vault-mind/dispatch Reserved for manual dispatch (future)

Shellcommands integration

The obsidian-shellcommands plugin can be configured to POST to /vault-mind/scan on every file save. This gives explicit, reliable notification instead of relying solely on fs.watch (which can be unreliable on WSL, network drives, and some Linux filesystems).

See docs/OBSIDIAN_SETUP.md for the shellcommands configuration walkthrough.