IvyClaw in Practice: Building a Multi-Agent Development System for Planning, Coding, Testing, and Review

I. What Is IvyClaw?

IvyClaw is a multi-agent AI Agent engineering system for software development tasks, built on DeepAgents and LangGraph. It does not simply make a single large-model call. Instead, it organizes requirement understanding, task planning, information retrieval, code implementation, automated testing, code review, and result delivery into a continuously running Agent Workflow.

The project uses technologies including Python, FastAPI, PostgreSQL, Redis, ARQ, Docker, Prometheus, and Grafana. It is suitable for learning and practicing the engineering implementation of AI Agents.

Note that some capabilities in the repository are still engineering prototypes or demonstrations. Further work is required for production use, including authentication, access control, security isolation, and infrastructure configuration.

Project URL: IvyClaw GitHub Repository.

II. Core Workflow and System Architecture

IvyClaw divides each development request among multiple Agents with clearly defined responsibilities. A typical flow is as follows:

User Request
      ↓
   Gateway
      ↓
  Dispatcher
      ↓
   Planner
      ↓
Researcher（as needed）
      ↓
    Coder
      ↓
   Tester
      ↓
  Reviewer
      ↓
Final Response

Planner understands requirements, plans tasks, and makes decisions; Researcher retrieves and organizes information when needed; Coder implements and modifies code; Tester runs tests and identifies issues; and Reviewer conducts a code review of the final result.

Requests typically first enter the API Gateway, then are passed by channel adapters and the Dispatcher to the Agent Runtime. PostgreSQL persists session state as well as LangGraph Checkpointer and Store data. Redis is used for caching, rate limiting, runtime state, and task queues. Time-consuming tasks are delegated to ARQ Workers for asynchronous execution.

III. Prepare the Runtime Environment

Before starting, prepare a Python project environment, available PostgreSQL and Redis services, and model APIs that meet the project's configuration requirements. The project uses uv to manage Python dependencies.

First, clone the repository and enter the project directory:

git clone https://github.com/ivyfan-toowell/IvyClaw.git
cd IvyClaw

Then create a local environment configuration from the example file. On Linux, macOS, or Unix-like environments, run:

cp .env.example .env

In Windows PowerShell, run:

Copy-Item .env.example .env

Fill in the required settings in .env according to your local environment, for example:

IVC_API_KEY=your_api_key_here
IVC_POSTGRES_URL=postgresql://user:password@localhost:5432/ivyclaw
IVC_DATABASE_URL=postgresql://user:password@localhost:5432/ivyclaw
IVC_REDIS_URL=redis://localhost:6379/0

Do not commit a real .env file to GitHub. Sensitive information such as LLM API keys, LangSmith API keys, Feishu App Secrets, database passwords, and real gateway API keys should be stored locally or in a secure deployment environment.

IV. Install Dependencies and Start Services

Install Python Dependencies

Run the following command in the project root:

uv sync

This command synchronizes the Python environment and dependencies according to the project configuration.

Start the FastAPI API

Start the development server with:

uv run uvicorn api.app:app --reload --port 8000

After startup, open http://localhost:8000/docs to view the Swagger API documentation automatically generated by FastAPI.

Start the ARQ Worker

To process time-consuming Agent tasks, start an ARQ Worker in another terminal:

uv run arq tasks.worker.WorkerSettings

Requests can be submitted to the Worker through a Redis queue. The Worker then invokes the Agent Runtime and persists the task result. Asynchronous task flows typically include a Job ID, task status queries, background execution, idempotent submission, and result retrieval.

Start the Feishu Long Connection

To connect through Feishu WebSocket, run:

uv run python -m channels.feishu_ws

IvyClaw also supports Feishu Webhook, Generic Webhook, and CLI entry points. Different channels are uniformly converted into InboundMessage, after which they reuse the same handle_message() and Agent processing logic.

V. Understanding Agent Tool Calls

IvyClaw assigns each Agent a minimal toolset matched to its responsibilities, reducing unnecessary calls and the risk of incorrect operations. Supported tool types include:

• Git: View and manage code versions.
• pytest: Run automated tests.
• Python and Shell: Run development commands.
• Web Search: Retrieve information through Tavily or Exa.
• MCP: Connect to external tools and services.
• File Tools: Read and modify workspace files.
• Sandbox: Isolate execution of Agent-generated code.

The project also uses skills/ to consolidate reusable development workflows, including requirement clarification, systematic debugging, test-driven development, unit testing, and verification before completion. Typical directories include:

skills/
├── fastapi-endpoint/
├── requirements-clarification/
├── systematic-debugging/
├── test-driven-development/
├── unit-test/
└── verification-before-completion/

VI. Code Sandboxing and Secure Execution

Code generated by Agents is not executed directly in the main service process. Instead, it can be run in an isolated environment. IvyClaw supports Docker Sandbox and Daytona Sandbox, along with commands such as Python, pytest, and Git.

Sandbox capabilities include an independent working directory, sandbox reuse and lifecycle management, and resource limits for CPU, Memory, and PID. Production environments use Docker Sandbox by default, while leaving room for stronger isolation runtimes such as gVisor or Kata.

When configuring a sandbox, assess network permissions, file access scope, resource limits, and concurrency based on actual risks. The README clearly states that production security isolation still requires further strengthening; the current demonstration configuration should not be treated as a complete security solution.

VII. State Persistence and Asynchronous Tasks

Persistence Components

PostgreSQL primarily stores LangGraph Checkpointer, Store, session state, and other persistent data. Redis handles caching, rate limiting, task queues, and runtime state. With Checkpointer and Store, the Agent execution context can continue across requests.

Asynchronous Task Flow

For development tasks with longer execution times, it is recommended that FastAPI create the task and Redis and an ARQ Worker process it asynchronously:

Client
  ↓
FastAPI
  ↓
Create Job
  ↓
Redis Queue
  ↓
ARQ Worker
  ↓
Agent Runtime
  ↓
Persist Result
  ↓
Query Job Status

This approach avoids occupying synchronous HTTP requests for extended periods and provides a foundation for task status queries, retries, and result persistence.

VIII. Human-in-the-Loop: Approval for High-Risk Operations

Operations such as deleting files and making destructive changes are not suitable for complete automation. IvyClaw supports Human-in-the-Loop workflows. When an Agent requests a high-risk tool call, execution can be interrupted while awaiting a human decision.

Agent requests risky action
          ↓
       Interrupt
          ↓
    Human Approval
       ↓       ↓
    Approve   Reject
       ↓
 Command(resume)
       ↓
Agent continues

After human approval, execution continues through a resume command. If the request is rejected, the current flow is terminated or changed. This mechanism establishes a control point between automation efficiency and operational safety.

IX. Gateway, Multi-Tenancy, and Reliability Governance

Before requests enter the Agent Runtime, they pass through the gateway layer for identity recognition and basic governance. The current repository provides:

• API Key authentication.
• Tenant identification and isolation.
• Rate limiting.
• Idempotency control.
• Request Context.
• Audit Log operations auditing.

The README specifically notes that the current Key and Tenant implementations are primarily demonstration configurations. Production environments should integrate a database or dedicated identity authentication system and further improve tenant permissions and data isolation.

The infrastructure layer also provides Retry, Timeout, Fallback, Concurrency Control, PostgreSQL Connection Pool, Redis Connection Pool, and Sandbox Pool mechanisms to handle high concurrency, model failures, and infrastructure fluctuations.

X. Observability, Testing, and Evaluation

IvyClaw integrates multiple observability components. LangSmith tracks Agent and LLM call chains, Prometheus collects metrics, and Grafana provides visual monitoring. The project also provides structured logs and audit logs.

At runtime, focus on the following metrics:

• Request latency and error rate.
• Concurrency.
• Number of LLM calls.
• Token usage and cost.
• ARQ Worker status.
• Sandbox status.

The project also includes tests/, eval/, and loadtest/ directories, covering pytest automated tests, Agent Runtime tests, Code Cases, Execution Scorer, LLM Judge, Baseline Evaluation, Reviewer Flow Test, and Locust load testing.

It is recommended to establish a validation loop in the order of development, automated testing, Agent Evaluation, load testing, metric observation, and issue optimization, rather than judging Agent reliability from a single manual conversation.

XI. Docker Production Deployment

To start the complete service stack, use the production Compose configuration provided by the repository:

docker compose -f docker-compose.prod.yml up -d --build

Production deployment involves the FastAPI web service, ARQ Worker, Feishu service, PostgreSQL, Redis, Prometheus, and Grafana. The API and Worker can share access to the Agent Runtime, which executes generated code through the Docker Sandbox.

Before going live, check database and Redis connections, secret management, API authentication, tenant isolation, rate-limiting policies, sandbox resource limits, log auditing, and monitoring and alerting configuration.

XII. Locating Code from the Project Structure

Familiarity with the directory structure helps you extend functionality quickly:

IvyClaw/
├── agent/          # Main Agent and task scheduling
├── api/            # FastAPI API
├── channels/       # CLI / Feishu / Webhook integrations
├── gateway/        # API authentication, Tenant, rate limiting
├── infra/          # PostgreSQL, Redis, LLM Router
├── middleware/     # Request context, concurrency, auditing, cost tracking
├── obs/            # Metrics and observability
├── profiles/       # Model and Agent Profiles
├── sandbox/        # Docker / Daytona sandboxes
├── skills/         # Agent Skills
├── subagents/      # Planner / Researcher / Coder / Tester / Reviewer
├── tasks/          # ARQ asynchronous tasks
├── tools/          # Git / pytest / Search / MCP tools
├── eval/           # Agent Evaluation
├── loadtest/       # Locust load testing
├── tests/          # Automated tests
├── scripts/        # Development, testing, and demonstration scripts
├── web/            # Web UI
├── Dockerfile
├── docker-compose.prod.yml
├── pyproject.toml
└── .env.example

XIII. Practical Usage Recommendations

1. First configure .env, PostgreSQL, and Redis, then confirm that the API is accessible through the Swagger documentation.
2. First validate the basic Agent Workflow with the synchronous API or CLI, then start the ARQ Worker to process long-running tasks.
3. Check tool permissions for the Coder, Tester, and Reviewer separately to avoid assigning unnecessary high-risk capabilities to any single Agent.
4. Enable sandbox execution for generated code and set reasonable limits for CPU, memory, process count, and the working directory.
5. Enable Human-in-the-Loop for deletions or destructive modifications; do not approve them automatically by default.
6. Use LangSmith, Prometheus, Grafana, and structured logs to observe request flows, costs, latency, and errors.
7. Before going live, run pytest, Agent Evaluation, and Locust load tests, then adjust model routing, concurrency, and retry policies based on the results.

Conclusion

The focus of IvyClaw is not a single model call, but combining multi-agent collaboration, tool calling, code sandboxing, state persistence, asynchronous execution, human approval, gateway governance, and observability into a software engineering Agent system.

After completing the local setup, start with the existing subagents/, tools/, and skills/ to gradually understand how Planner, Researcher, Coder, Tester, and Reviewer collaborate, then extend channels, tools, or evaluation workflows according to your business needs.