Systems·July 3, 2026·8 min

AI Agent Tool Execution: When Parallel Breaks Things

Parallel tool execution cuts latency by 40%. It also introduces race conditions, silent state corruption, and double-charge bugs. Here's the operator's guide to getting it right.

I learned this one the hard way. I had two terminal calls running in parallel. Both needed to write to the same config file. One finished and committed. The other finished half a second later and overwrote the first change. No error. No warning. The agent moved on like nothing happened.

Parallel tool execution is one of those features that sounds like pure upside. Two independent API calls? Run them at the same time. Cut your end-to-end latency by 40%. Every modern agent framework supports it. OpenAI added it to the Chat Completions API in late 2023. Anthropic followed. Hermes Agent's tool batch executor parallelizes built-in read-only tools by default. And starting today, MCP servers can opt in too.

The problem is that "independent" is not always what you think it is. Two tools that look separate from the agent's perspective can collide in ways the agent never sees. The result is corrupted state, dropped work, and failures that disappear when you try to reproduce them sequentially.

TL;DR: Parallel tool execution is not a feature toggle you flip and forget. It produces five distinct failure categories. The mitigation strategies are concrete and testable. And the new MCP parallel flag gives server authors a clean opt-in model. If you run an agent fleet, audit your tool set before enabling concurrency. The latency savings are real. Corrupted state is forever.

What parallel tool execution actually does

When an LLM decides to call multiple tools in one response, the framework faces a choice. Run them one at a time and feed each result back into context before the next call. Or fire them all at once and collect the results as they arrive.

Sequential execution is safe. Each tool sees the exact state left by the previous one. The agent can course-correct between calls. The downside is latency. If you need to read three files, sequential means three round trips.

Parallel execution ships all calls simultaneously. For read-only operations on independent resources, this is the right call. Reading three different files or searching three different APIs has no shared state to corrupt. Hermes Agent already parallelizes web_search, read_file, and other built-in tools that are known to be safe.

The edge cases start where the built-in safe list ends.

The new MCP parallel flag

Today's Hermes Agent commit adds supports_parallel_tool_calls as an MCP server config option. Set it to true and tools from that server can run concurrently within a single batch, same as the built-in read-only tools. While this is implemented in Hermes Agent's MCP layer today, the pattern applies to any agent framework that parallelizes tool calls. The opt-in model, the safe/unsafe distinction, and the all-or-nothing fallback are framework-agnostic design decisions.

mcp_servers:
  docs:
    command: "docs-server"
    supports_parallel_tool_calls: true

Before this change, all MCP tools ran sequentially regardless of what they did. An MCP server that exposes five independent read-only endpoints paid the full sequential penalty every time the agent called more than one. A well-structured MCP server strategy depends on fast tool dispatch. Now server authors can declare their tools safe for concurrent execution and get the latency win.

The implementation is clean. A new _parallel_safe_servers set tracks which servers have opted in. is_mcp_tool_parallel_safe() extracts the server name from the tool name, checks the set, and returns True if the server's flag is on. The existing _should_parallelize_tool_batch() function now checks MCP tools against this set instead of always returning False for anything not in the built-in safe list.

Eleven new tests cover the feature end to end, including the edge case where one parallel MCP server is mixed with a non-parallel one. The batch falls back to sequential if any tool in the batch comes from a server without the flag. All-or-nothing safety.

What breaks when tools interleave

The failure modes fall into five categories.

Read-only tools on independent resources. That's the rule.

Race conditions on shared state

Two tools read and write the same resource without coordination. The classic example: two terminal calls both modifying a file. One reads, edits, writes. The other does the same. The second write silently clobbers the first one. The agent has no idea because it only sees the final file contents.

This is not a theoretical problem. I hit it last week with an agent that needed to update a YAML config and restart a service in the same batch. Two separate tool calls. Different tools. Same file on disk. The restart ran with the old config because the write hadn't flushed yet.

Non-idempotent side effects

An idempotent operation produces the same result whether you run it once or ten times. GET /status is idempotent. POST /charge-credit-card is not.

Parallel execution can accidentally retry non-idempotent calls. If the agent fires two API calls, the first one times out, and the framework retries it without knowing the second one already ran, you get a double charge. Most agent frameworks do not track partial batch execution state well enough to prevent this.

Ordering dependencies

Some tools implicitly depend on the output of a previous tool, even if the agent does not pass that output as an argument. Setting an environment variable before launching a process. Installing a package before running a command that uses it. Creating a directory before writing a file into it.

When these dependencies are explicit in the tool call arguments, the LLM sees them and can structure the calls correctly. But a lot of dependencies are environmental. The agent calls mkdir and write_file in the same batch because it does not realize the file lives inside the directory it just created. The write fails with a "no such file or directory" error, and the agent has to recover.

Index and identity collisions

This one is subtle. When a framework returns tool results to the LLM, it attaches each result to the correct call by position or by a generated ID. In parallel execution, those positions and IDs can drift.

I saw this in LangChain last month. Two parallel read_file calls returned their results, but the result-to-call mapping used array indices. The framework interleaved the results incorrectly. The LLM thought file A contained file B's contents and acted on it. No error surfaced anywhere. The agent confidently made a decision based on the wrong file.

The position-vs-identity problem gets worse when tool calls time out and get retried. If call 1 times out and gets retried, but call 2 already completed, what index does the retried result get? If the framework uses position-based arrays for result mapping, you get silent mapping errors. The Hermes Agent test suite for the new MCP parallel flag includes an explicit test for mixed parallel/non-parallel batch result ordering for exactly this reason.

Infrastructure contention

Parallel tool execution puts pressure on shared infrastructure that sequential execution never does. Two tools both opening database connections from a connection pool capped at one. Two terminal calls competing for the same event loop. Two HTTP calls hitting the same rate-limited API with the same credential.

The failures look different from race conditions. You get ConnectionPoolTimeout errors instead of corrupted data. The agent retries, succeeds on the second attempt, and you never see the pattern. But at scale, those retries compound. An agent fleet of 50 instances running parallel tool calls against a shared Redis instance with a 16-connection limit will burn latency gains on retry loops.

The research team cataloged seven real-world parallel tool failures across four platforms before this post went to review. Four of the seven involved mutable shared state. Two were index collisions that produced silent data corruption. One was event loop contention that caused cascading timeouts under load.

Failure category vs mitigation matrix

| Failure category | Primary mitigation | Also helps | |---|---|---| | Race conditions (mutable shared state) | Resource-keyed locking | Idempotency keys, per-call isolation | | Non-idempotent side effects | Idempotency keys at protocol layer | Track partial batch state | | Ordering dependencies | Explicit execution order contracts | Per-call tool isolation | | Index and identity collisions | Position-independent result mapping | Framework-level ordering guarantees | | Infrastructure contention | Per-worker async infrastructure isolation | Optimistic locking with versioned state |

Which tools are safe to parallelize

Read-only tools on independent resources. That's the rule. If the tool does not mutate anything another tool in the batch could touch, it is safe.

Safe by default:

web_search (read-only, network, no local state)
read_file (read-only, filesystem, no mutation)
search_files (read-only, no side effects)
API GET calls against independent endpoints
vision_analyze on different images
Multiple independent database SELECT queries

Not safe by default:

Any tool that writes to disk (write_file, patch, terminal with mutation)
Any tool that changes process state (terminal with service restart, env var changes, package installs)
Any tool that sends data externally (email, messaging, payment APIs)
Any two tools that reference the same file, database row, or API resource
MCP server tools unless the server author has explicitly declared parallel safety

The MCP docs now include a caution that you should read closely:

Only enable parallel calls for MCP servers whose tools are safe to run at the same time. If tools read and write shared state, files, databases, or external resources, review the read/write race conditions before enabling this setting.

The operator's checklist

If you run an agent fleet, parallel tool execution is worth enabling. The latency savings are real. I measured 35-45% faster end-to-end response times on batches with three or more tool calls after enabling parallel execution where safe. But you need to do the audit first. The same audit you would run through a sandbox checklist for agent tool safety, extended to cover concurrency.

Audit your tool set. List every tool your agents can call. For each one, ask: does this tool mutate state that another tool in a typical batch could touch? If yes, it is not parallel-safe unless you can prove the resources never overlap.
Enable parallel execution per-server, not globally. The MCP opt-in model is the right approach. Don't flip a global "parallel everything" switch. Enable it for specific servers and specific tool categories where you understand the safety properties.
Test with concurrent workloads. Race conditions are probabilistic. They don't show up in single-agent sequential tests. Run multiple agents in parallel, or use a load generator that fires tool batches with overlapping resources. The bug you never see in dev is the one that hits production at 2 AM.
Monitor for interleaving failures. If a tool call fails with a "file not found" or "resource locked" error that resolves on retry, you have a parallel execution collision. Log the batch composition alongside the failure. These are not random timeouts. They are design bugs in your tool parallelization strategy.
Track partial batch state. If your framework does not handle partial completion of a parallel batch, you need a wrapper that does. When one tool in a batch of three fails, what happens to the other two? Do they get rolled back? Does the agent see their results? The answer changes whether parallel execution is net-positive or net-negative for reliability.
Use idempotency keys for non-read-only tools. Any tool that mutates external state should accept an idempotency key. The agent generates a UUID per batch, passes it to every mutating tool, and the tool ignores duplicate keys on the receiving end. This prevents the double-charge problem even when parallel retries interleave. Some API gateways support this natively. For tools you control, it is a dozen lines of code.
Adopt resource-keyed locking for shared state. If two tools touch the same file, database row, or API resource, key a mutex on the resource path. The simplest version: a Python threading.Lock dict keyed by resource name, acquired before any tool call that touches that resource. Note that this only covers in-process threading. For async tool executors, you need asyncio.Lock. For tools that spawn subprocesses, you need filesystem-level locks. The pattern is right; the locking primitive depends on your executor's concurrency model.
Define execution order contracts. For tool sets with implicit ordering dependencies, make the contract explicit. A depends_on: ["mkdir"] field in the tool schema tells the framework to serialize those calls so that tool B only runs after tool A completes. The LLM does not need to understand the ordering. The framework enforces it. This is separate from the all-or-nothing fallback: if any tool in a batch comes from a non-parallel-safe server, the entire batch runs sequentially regardless of explicit ordering contracts. The contract tells the framework what order to run them in. The fallback tells it whether to parallelize at all.

Parallel execution is a power tool. Use it where you understand the isolation boundary and disable it everywhere else.

When to stay sequential

Some tool sets should never be parallelized regardless of latency. If your agent's primary job is mutating state on a shared filesystem, running database migrations, or sending financial transactions, sequential execution is the correct default. The latency cost is insurance against corrupted state.

The exception is when you can prove isolation. Two write_file calls to different directories with different filenames. Two terminal calls in separate working directories that never touch the same files. Two API POST calls to independent services with no shared backend state. In these cases, parallel is safe. But "prove" means you have tested the concurrent path, not just reasoned about it.

I keep a running list of tools I have verified as parallel-safe in my own agent stack. It is shorter than the list of tools I have not verified. That asymmetry is the right one.

What changed today

The Hermes Agent commit is a small change with a big lesson. Before today, MCP servers had no way to tell the framework their tools were parallel-safe. The safest assumption won: everything sequential. Now server authors can declare safety and operators can decide.

The opt-in model is the right call. Parallelism should be explicit. If you do not set the flag, your tools stay sequential. No accidental concurrency. No surprise race conditions from a framework upgrade.

The broader point is that tool execution ordering is a reliability surface that most agent operators are not thinking about yet. We spend a lot of time on prompts, memory, and model selection. We spend almost no time on what happens when two tools run at the same time on the same resources. That gap will close as more agents run in production and more people hit the same "my config got silently overwritten" bug I did.

Parallel execution is not wrong. It is a power tool. Use it where you understand the isolation boundary and disable it everywhere else. Your agent's state will thank you.