Build Async, Durable AI Agent Workflows with Unreal Agent

What Is Unreal Agent?

Unreal Agent is an async-first agent harness from Unreal Labs. It provides composable infrastructure for accepting inputs, maintaining session history, preparing model context, coordinating LLM turns, translating tool calls, and dispatching durable asynchronous operations.

The project separates synchronous coordination from asynchronous execution. Tool translators validate model-generated requests and produce serializable operation descriptions without performing I/O. An operation manager then executes the committed operations outside the coordinator’s event loop.

Repository Structure

The repository is organized into three primary areas:

• harness/: The reusable agent harness library.
• cmd/: Executables built with the library.
• benchmarks/: Benchmark runners for evaluating the project.

Start with harness/ when integrating the framework into another application. Review cmd/ for concrete executable entry points, and use benchmarks/ when you need to inspect or measure runtime behavior.

Core Concepts

Inputs and the Inbox

An input is an event carrying a globally unique identifier supplied by the caller. That identifier must remain unchanged if the event is delivered again.

The session inbox uses these identifiers for in-memory deduplication of external, control, and crash inputs. This idempotency is session-scoped and volatile, so callers should still provide stable IDs consistently.

{
  "id": "globally-unique-and-stable-id",
  "type": "external-input",
  "payload": "Example request"
}

This shape is conceptual rather than a documented Unreal Agent API type. The important requirement is the stable, globally unique input ID.

Sessions

A session is an append-only, persisted history. Sessions can be resumed and forked, enabling an application to continue existing work or branch from prior history without rewriting the original sequence.

The session store owns the canonical history and operation state. It also atomically records a tool call’s translation status together with any operations produced by that translation.

LLM Turns

An LLM turn is the coordinator-managed sequence surrounding one logical model request. The coordinator persists accepted inputs, prepares the turn, resolves tool translators, records resulting work, and dispatches committed operations.

Tools, Translators, and Operations

A tool combines a schema with a translator. When a model produces a tool call, the translator validates it and converts it into one or more operations.

A tool translator runs synchronously on the coordinator’s event loop. It must not perform I/O or suspend the loop.

The translation result is called the tool call status. It contains either a validation error or references to submitted operations. This status is distinct from the execution state of those operations.

An operation is a serializable, versioned description of asynchronous work. Translators describe work; the operation manager performs it.

Key Components

• Session inbox: Provides volatile, session-scoped input idempotency.
• Coordinator: Persists accepted inputs, manages LLM turns, resolves tool translators, and dispatches committed operations.
• Session store: Persists canonical session history and operation state, supports recovery and forks, and atomically records tool-call status with operations.
• Context builder: Assembles model input in memory and reports omitted, truncated, or compacted content. It performs no I/O and has no persistence dependencies.
• LLM adapter: Sends prepared input to a model provider and returns a normalized completed response. It owns authentication, cancellation, and provider errors.
• Tool registry: Owns the Bash, ViewImage, and skill-use tool definitions and translators, while exposing the set selected by the host.
• Tool translator: Validates calls, produces statuses and operations, and formats recorded statuses and prepared operation output for the model.
• Operation manager: Supplies the actor runtime for durable operations. The local implementation can be replaced.

Installation and Setup

The README does not specify a package manager, language-specific installation command, or stable public API. Setup should therefore begin from a local checkout of the repository and its build metadata rather than from an assumed package command.

1. Clone or download the Unreal Agent repository.
2. Open the repository root and inspect its build and dependency files for the supported toolchain.
3. Build or install the library under harness/ using the repository’s configured tooling.
4. Inspect cmd/ for executable examples that compose the harness components.
5. Use benchmarks/ only after the core project builds successfully.

cd <unreal-agent-repository>

# Inspect the library, executable, and benchmark directories.
ls harness
ls cmd
ls benchmarks

A host application will also need an LLM adapter configured for its chosen provider. Authentication, cancellation behavior, and provider-specific error handling belong in that adapter.

Basic Usage: Following One Agent Request

Because the README does not publish concrete constructor or method names, the safest way to understand usage is to follow the documented component flow.

1. Create or resume a session. The session store provides persisted history and recovery support.
2. Submit an input with a stable ID. Reuse the same ID when redelivering the same logical event.
3. Deduplicate through the session inbox. The inbox rejects or ignores repeated inputs within the active session according to the implementation’s behavior.
4. Persist the accepted input. The coordinator records it in the canonical append-only session history.
5. Build model context. The context builder assembles in-memory model input and reports any content it omitted, truncated, or compacted.
6. Run the LLM turn. The LLM adapter sends the prepared input to the provider and returns a normalized completed response.
7. Resolve tool calls. The coordinator looks up each permitted tool through the registry.
8. Translate synchronously. The selected translator validates the call and produces either a validation error or serializable operations.
9. Commit atomically. The session store records the tool-call status and operations together.
10. Dispatch asynchronously. The operation manager executes committed operations and tracks their execution state separately from translation status.
11. Format results for the model. The translator combines recorded call status with prepared operation output to produce a model-facing result.

Example Tool-Call Lifecycle

Input with stable ID
  -> inbox deduplication
  -> persisted session event
  -> context construction
  -> normalized LLM response
  -> tool registry lookup
  -> synchronous validation and translation
  -> atomic status and operation commit
  -> asynchronous operation execution
  -> model-facing result formatting

If validation fails, the translator records a validation error rather than producing operations. If validation succeeds, the status references the submitted operations, while the operation manager independently tracks whether those operations are pending, running, completed, or otherwise handled by its implementation.

Selecting Tools

The tool registry owns definitions and translators for Bash, ViewImage, and skill use. The host chooses which of these tools to expose.

Expose only the capabilities needed for a particular deployment. The README does not define authorization or sandboxing policies, so those concerns should be handled by the host environment and the operation execution layer.

Extending the Harness

Unreal Agent’s components are intended to be composable. Alternative implementations can replace interface-backed components, provided they preserve the documented responsibilities and invariants.

Implementing a Tool Translator

A custom translator should follow this sequence:

1. Validate the model-produced arguments against the tool’s schema.
2. Return a validation error when the request is invalid.
3. When valid, create one or more serializable, versioned operation descriptions.
4. Return a status that references the submitted operations.
5. Provide formatting logic that converts recorded status and prepared operation output into a model-facing result.

Keep translation deterministic and fast. Do not make network requests, access files, wait on subprocesses, or perform other I/O inside the translator.

Replacing the Operation Manager

The local operation manager is swappable. One documented extension is a proxy manager that serializes operations and sends them to another local operation manager running inside a remote sandbox process.

This design keeps the coordinator and translators focused on validation and durable intent while allowing execution to occur in an isolated environment.

Implementing Storage

Alternative session stores must preserve serializability, versioning, recovery behavior, and the atomic relationship between tool-call status and generated operations.

The project aims to maintain backward compatibility for sessions. If a session version is unsupported, resuming it must fail with an explicit error rather than silently misreading the data.

Advanced Tips

• Generate input IDs at the caller boundary. Store the ID with the originating event so retries and redeliveries retain the same identity.
• Do not confuse inbox deduplication with durable history. The inbox is volatile and session-scoped, while the session store persists canonical events.
• Keep translators pure and synchronous. Translate requests into operations instead of carrying out the work directly.
• Version every operation format. Operations may outlive the process that created them or cross process and sandbox boundaries.
• Preserve atomic commits. Recording a successful tool-call status without its operations could leave a session inconsistent after a crash.
• Track translation and execution independently. A valid call may produce a committed operation that has not yet completed.
• Use context-builder metadata. Monitor what was omitted, truncated, or compacted so model behavior can be diagnosed.
• Centralize provider concerns. Keep credentials, cancellation, and provider errors inside the LLM adapter.
• Fork sessions for experimentation. A fork can explore a different continuation while preserving the source session’s append-only history.
• Use a proxy operation manager for isolation. Serialized operations can be forwarded to a runtime inside a remote sandbox without moving synchronous translation there.

Important Invariants

Extensions should preserve the following guarantees:

• Session-store items remain serializable.
• The storage format remains versioned.
• Unsupported session versions produce explicit resume errors.
• Operations remain serializable and versioned.
• Context construction performs no I/O and accepts no persistence dependencies.
• Tool translation performs no I/O and never suspends the coordinator’s event loop.

Conclusion

Unreal Agent provides a modular foundation for agents that need persistent sessions, deduplicated inputs, coordinated LLM turns, validated tool calls, and durable asynchronous work. Its central design principle is a clean boundary between synchronous translation and asynchronous execution. Start with the existing executables, follow the documented event flow, and preserve serialization, versioning, and atomicity when replacing or extending components.