Task management and parallelism

When an agent calls multiple tools in a single turn, running them sequentially wastes time. When a shell command takes 30 seconds, the agent shouldn’t be blocked waiting. This chapter covers agentkit-task-manager: how tool calls are scheduled, routed, and delivered.

The problem

The default behavior is sequential: tool calls execute one at a time on the current task. This is correct and simple, but it becomes a bottleneck when:

• The model requests multiple independent tool calls
• A shell command runs for a long time
• You want to start background work while the model continues

TaskManager trait

#![allow(unused)]
fn main() {
#[async_trait]
pub trait TaskManager {
    async fn start_task(&self, request: TaskLaunchRequest, ctx: TaskStartContext)
        -> Result<TaskStartOutcome, TaskManagerError>;

    async fn wait_for_turn(&self, turn_id: &TurnId, cancellation: Option<TurnCancellation>)
        -> Result<Option<TurnTaskUpdate>, TaskManagerError>;

    async fn take_pending_loop_updates(&self)
        -> Result<PendingLoopUpdates, TaskManagerError>;

    async fn on_turn_interrupted(&self, turn_id: &TurnId)
        -> Result<(), TaskManagerError>;

    fn handle(&self) -> TaskManagerHandle;
}
}

SimpleTaskManager (default)

Runs every tool call inline. No Tokio dependency. No concurrency. Returns the result before the driver continues. This is the default when no task manager is configured.

AsyncTaskManager

Spawns each tool call as a Tokio task. Tasks are classified through a TaskRoutingPolicy:

#![allow(unused)]
fn main() {
pub enum RoutingDecision {
    Foreground,
    Background,
    ForegroundThenDetachAfter(Duration),
}
}

• Foreground — blocks the current turn until resolved
• Background — runs independently, results delivered later
• ForegroundThenDetachAfter(Duration) — starts foreground, automatically promotes to background if it hasn’t finished within the timeout

Routing policies

Implement TaskRoutingPolicy or use a closure:

#![allow(unused)]
fn main() {
let task_manager = AsyncTaskManager::new().routing(|req: &ToolRequest| {
    if req.tool_name.0 == "shell.exec" {
        RoutingDecision::ForegroundThenDetachAfter(Duration::from_secs(5))
    } else {
        RoutingDecision::Foreground
    }
});
}

This lets you make filesystem tools synchronous (fast, no overhead) while giving shell commands a timeout before they detach.

Task lifecycle events

The TaskManagerHandle provides an event stream:

#![allow(unused)]
fn main() {
pub enum TaskEvent {
    Started(TaskSnapshot),
    Detached(TaskSnapshot),
    Completed(TaskSnapshot, ToolResultPart),
    Cancelled(TaskSnapshot),
    Failed(TaskSnapshot, ToolError),
    ContinueRequested,
}
}

Host code can subscribe to these events for progress reporting, UI updates, or manual task management:

#![allow(unused)]
fn main() {
let handle = task_manager.handle();
// List running tasks, cancel tasks, drain results, subscribe to events
}

Integration with the loop

The loop driver integrates with the task manager transparently:

1. Tool call arrives from the model
2. Driver asks the task manager to start a task
3. If TaskStartOutcome::Ready — result is immediately available
4. If TaskStartOutcome::Pending — driver waits for foreground tasks via wait_for_turn()
5. Background task results are picked up via take_pending_loop_updates() on the next iteration
6. Background results are injected into the transcript as tool results

The detach-after-timeout pattern

ForegroundThenDetachAfter deserves special attention. It solves a common problem: you want the model to wait for a command’s output, but you don’t want a slow command to block the entire turn.

ForegroundThenDetachAfter(5s) — two possible outcomes:

Fast command (< 5s):

  t=0s  Task starts (foreground)
  t=3s  Command finishes → result returned immediately
        └── Model sees output, continues normally
        └── Identical to pure Foreground routing


Slow command (> 5s):

  t=0s  Task starts (foreground)
  t=5s  Timeout expires → task promoted to background
        └── Model receives: "Task detached (still running)"
        └── Model continues its turn (reads files, etc.)
  t=30s Command finishes → result stored
        └── On next turn, driver picks up the result
        └── Result injected into transcript as a tool result

This is the right default for shell commands in a coding agent:

• cargo check (2 seconds) → foreground, model sees the output immediately
• cargo test (30 seconds) → detaches after 5s, model continues working
• ls (instant) → foreground, practically no delay

How background results re-enter the loop

When the driver starts a new turn, it calls task_manager.take_pending_loop_updates(). Any completed background tasks have their results injected into the transcript before the model sees it:

Turn N:
  Model: ToolCall(shell.exec, "cargo test")
  Task manager: starts foreground, detaches after 5s
  Model receives: "task detached"
  Model: ToolCall(fs.read_file, "src/test_results.rs")  ← continues working
  Turn ends

Turn N+1:
  take_pending_loop_updates() → cargo test finished: "3 tests passed"
  Result injected into transcript
  Model sees: tool result from cargo test + new user message
  Model: "All 3 tests pass. Here's what I changed..."

Task lifecycle

              ┌─────────────────┐
              │     Started     │
              └────────┬────────┘
                       │
        ┌──────────────┼─────────────┐
        │              │             │
  Foreground    FG then detach   Background
        │              │             │
        │         ┌────▼─────┐       │
        │         │ timeout? │       │
        │         └──┬───┬───┘       │
        │         no │   │ yes       │
        │            │   │           │
  ┌─────▼────────────▼┐  │  ┌────────▼──────┐
  │    Foreground     │  │  │  Background   │
  │    (blocks turn)  │  │  │  (async)      │
  └─────────┬─────────┘  │  └──────┬────────┘
            │            │         │
            │     ┌──────▼─────┐   │
            │     │  Detached  │   │
            │     │  (async)   │   │
            │     └──────┬─────┘   │
            │            │         │
       ┌────▼────────────▼─────────▼────┐
       │       Completed / Failed       │
       └────────────────────────────────┘

Choosing a routing strategy

Example: openrouter-parallel-agent uses AsyncTaskManager with ForegroundThenDetachAfter routing for shell tools and foreground routing for filesystem tools. The TaskManagerHandle event stream is printed to stderr.

Crate: agentkit-task-manager — depends on agentkit-tools-core, agentkit-core, and tokio.