How to Run Verification-Driven GitHub Issue and PR Repairs with RepoPilot

What RepoPilot Does

RepoPilot is a self-hosted controller for verification-driven software iteration. It turns selected GitHub Issues and same-repository pull requests into a bounded workflow covering rule review, test generation, failure reproduction, code repair, independent verification, and optional publication of a repair branch and draft pull request.

The project uses the OpenAI Codex SDK to understand repository rules, design requirement-driven tests, and propose production-code changes. A separate Docker runner executes tests against pinned repository snapshots. The controller publishes a result only when the configured evidence and policy checks pass.

RepoPilot does not automatically merge or deploy changes. Maintainers select the work, review the evidence, and retain the final merge decision.

How the workflow is bounded

1. RepoPilot pins the relevant base and head commits.
2. It loads trusted policy from the base branch, including applicable AGENTS.md guidance.
3. Codex reviews the change and proposes tests or repair candidates.
4. Generated tests are frozen before production code is repaired.
5. A separate runner executes tests against independent snapshots.
6. The controller checks stable test identities, failure fingerprints, protected paths, policy results, and configured limits.
7. An eligible result can be placed on a separate branch and submitted as a draft pull request for human review.

For pull requests, RepoPilot compares base and head behavior. For Issues, it first requires a passing original baseline, then attempts to reproduce the reported defect with a new frozen test before proposing a repair.

Key Features and Current Scope

• Pinned inputs: Base and head SHAs, title and body digests, freshness checks, and cancellation help prevent verification against stale inputs.
• Trusted repository rules: Static policies support literal checks and JavaScript or TypeScript call rules. Nested base-branch AGENTS.md files can guide semantic review.
• Independent test execution: The controller delegates execution to a separate Docker runner rather than trusting the model's assessment.
• Multiple test ecosystems: Version 1.3.0 supports Node's test runner, Vitest, pytest, Go tests, and compatible JUnit XML evidence.
• Monorepo support: Configurations can use working directories, multiple named test commands, and disposable dependency services.
• Bounded repair: Attempts, retries, model calls, tokens, task executions, and overall task time can be limited.
• Traceable reports: Atomic JSON reports and abbreviated Markdown summaries retain findings, test outcomes, patches, attempts, and publication state.
• Operational recovery: Task commands support inspection, cancellation, resumption, and reruns. The recover flow handles owned crash leftovers through preview and validated cleanup.

The current implementation targets public, text-based repositories with up to 10,000 files or 16 MiB. It does not provide fork pull-request execution, browser end-to-end testing, a hosted dashboard, webhooks, a distributed queue, automatic dependency installation, automatic merging, or deployment.

Prerequisites

You can download a RepoPilot 1.3.0 portable release for Linux, Windows, or macOS if you do not want to install Node.js. For a source installation, prepare the following:

• Node.js 22, which is the recommended version
• npm and Git
• Docker configured to run Linux containers
• An OpenAI API key when Codex-backed review, test planning, or repair is enabled
• A GitHub token when GitHub access or publication is required

Use matching 1.3.0 controller and Agent image versions. Docker images and project dependencies must be provisioned in advance because test containers have no network access by default.

Install RepoPilot from Source

1. Install, validate, and build the project

From the cloned RepoPilot repository, install the locked dependencies, run project checks and tests, and build the controller:

npm ci
npm run check
npm test
npm run build

RepoPilot's own development tests use mocked APIs, agents, runners, and synthetic fixtures. They do not consume model credits, launch Docker, or write GitHub content.

2. Create a local configuration

Copy the supplied example configuration:

cp repopilot.example.json config.local.json

PowerShell users can use Copy-Item instead of cp. Edit config.local.json to specify the target repository and a trusted test command. Keep this local configuration outside the repository snapshots being reviewed.

The example configuration disables agent review, repair, and publishing. Enable these capabilities separately only after local verification is working.

3. Build the Codex Agent image

Build the Agent container when you want Codex-backed semantic review, test planning, or repair:

docker build -f Dockerfile.agent -t repopilot-agent:local .

Set OPENAI_API_KEY in the controller environment. Only the Agent container receives it. Desktop ChatGPT credentials are not reused. A GitHub credential supplied through GITHUB_TOKEN or GH_TOKEN remains in the controller.

Run Your First Local Pull Request Check

Start with a local comparison between a base branch and a feature branch:

npm run dev -- check --config config.local.json --repo /path/to/project --base main --head feature

Replace /path/to/project, main, and feature with your local repository and refs. A local check does not publish changes and does not edit the source checkout.

How RepoPilot classifies generated tests

Generated scenarios are categorized as either regression or new_behavior. Regression is the default.

• Regression evidence: The same executed test case passes on the base revision and fails on the head revision.
• New behavior evidence: The case fails on base and passes on head, and the proposed test includes an exact requirement quote from the pull-request title or body.
• Review required: Both revisions fail, a case is skipped or missing, execution fails, test discovery changes, or original test results change unexpectedly.

A model-provided label cannot override runner evidence. Regressions must repeat with identical failing test identities and fingerprints before they are eligible for automatic repair.

Review Pull Requests Continuously

RepoPilot's watcher polls non-draft, same-repository pull requests and excludes its own autofix branches. Run one polling cycle while validating the configuration:

npm run dev -- watch --config config.local.json --once

After confirming the results, run the watcher continuously:

npm run dev -- watch --config config.local.json

Publishing is opt-in and requires publish=true. The GitHub token must have repository Contents and Pull requests write permissions. Publication creates separate autofix branches and draft pull requests; it never force-pushes or merges automatically.

Before publication, RepoPilot rechecks the commit SHAs and pull-request description. An existing repair branch or pull request can be reused only when its parent and tree match the verified result.

Reproduce and Repair a GitHub Issue

Issue repair is explicitly selected by the maintainer. To process Issue 123, run:

npm run dev -- fix --issue 123 --config config.local.json

The Issue workflow proceeds in three major stages:

1. Establish the baseline: Existing tests must pass on the pinned target revision.
2. Reproduce the bug: RepoPilot generates and freezes a test tied to the Issue description. The new test must reproduce the reported failure.
3. Attempt and verify a repair: Codex proposes code replacements, and the independent runner checks the frozen generated test, the original suite, and repository policy.

Insufficient, unstable, or incomplete evidence stops the workflow for human review. RepoPilot does not autonomously select Issues, define a product roadmap, or merge the resulting repair.

Add Trusted Repository Policy

Commit .repopilot/policy.json to the target base branch. For example, the following rule rejects disabled JavaScript and TypeScript tests:

{
  "rules": [
    {
      "id": "no-disabled-tests",
      "kind": "forbid-call",
      "extensions": [".ts", ".js"],
      "callee": "test.skip",
      "message": "Keep regression tests enabled.",
      "severity": "error"
    }
  ],
  "exceptions": []
}

Supported static rule kinds include forbiddenText, forbid-call, and require-call. Call rules inspect direct calls, qualified calls, and string property access. They do not resolve aliases or perform whole-program analysis. Literal rules may also match comments.

Policy exceptions can identify a ruleId, exact path, reason, UTC expiration timestamp, and optional exact evidence. Exceptions are trusted only when they come from the base policy, and expired exceptions do not suppress findings.

Overlapping required and forbidden rules cause review to stop. Pull requests that modify trusted rule files require maintainer review and cannot authorize their own repairs.

Configure Test Evidence Carefully

The default reporter is node and uses node --test. Build RepoPilot first so the trusted reporter is available. For Vitest, configure reporter=vitest and use an explicit vitest run command in a trusted image with pinned dependencies.

• Reporter flags belong to the controller rather than the untrusted test command.
• reporter=command collects output only and cannot support verification or publication.
• Omitting the runner produces a policy-only review, with tests recorded as not_run.
• Zero tests, all-skipped tests, malformed reports, and missing test identities never count as passing.
• Configured dependency services use a disposable internal Docker network.

For generated tests involving exports that do not exist on the base revision, check for those exports inside executing test cases. A missing export should not crash a top-level import before structured evidence can be collected.

Manage Running and Completed Tasks

RepoPilot stores reports and snapshots under .repopilot-data. Each task receives JSON evidence and a Markdown summary. Retried executions archive previous evidence using names such as TASK.execution-N.json.

Inspect and cancel tasks

npm run dev -- tasks list --config config.local.json --status running --limit 20 --offset 0
npm run dev -- tasks show TASK_ID --config config.local.json --format markdown
npm run dev -- tasks cancel TASK_ID --config config.local.json

Listing, showing, and cancelling tasks remain available while the controller holds its writer lock. Cancellation creates a persistent request checked during verification and publication. It cannot reverse GitHub writes that have already completed.

Resume or rerun work

npm run dev -- tasks resume TASK_ID --config config.local.json
npm run dev -- tasks rerun TASK_ID --config config.local.json

resume keeps the task identity, configuration, and limits. Depending on state, it restarts verification or continues publication for an already verified task. rerun creates a linked task using the original pinned commits and the current configuration while preserving earlier reports and cancellation markers.

Permanent, terminal, or exhausted tasks require a rerun. Replay also requires the recorded local repository or Git cache, and pull-request inputs are checked again during replay.

Advanced Operational Tips

Use conservative limits

Configure bounded values for task timeout, maxCalls, maxAttempts, maxTaskExecutions, and token usage. The maxTokens value is calculated from usage reported after each model call, so a single call can exceed it. It should not be treated as a hard monetary spending cap.

Treat environment failures separately

RepoPilot categorizes failures and applies bounded phase-level retries to eligible environment problems. Unstable tests, changing discovery, or incomplete evidence block automatic repair instead of being treated as confirmed code defects.

Protect the evidence chain

Repair candidates must preserve generated test identities and pass the original base, original head, and generated cases as required by the workflow. Existing tests, manifests, configurations, policies, and hidden paths are protected. Static and semantic policy checks run again after a candidate repair.

Recover cautiously after crashes

Version 1.3.0 includes a recover workflow for previewing and explicitly cleaning resources owned by interrupted tasks. Verify that the old process has stopped before removing stale locks. Legacy locks and resources may require manual inspection.

Understand the trust boundary

The controller, reports, and runner operate on your machine or worker, but Codex model calls use OpenAI services and send selected repository text and task context. Self-hosting RepoPilot does not provide offline model inference.

Test execution supplies engineering evidence, not tamper-proof attestation against malicious code. Symlinks, submodules, binaries, and case-colliding paths fail closed. Review the project's security guidance before executing untrusted repositories.

Conclusion

RepoPilot combines Codex-assisted review and repair with pinned inputs, frozen tests, independent Docker execution, repository policy, bounded retries, and durable evidence. Begin with a non-publishing local check, validate your reporter and policy configuration, then add watching, Issue repair, and draft pull-request publication only when the evidence meets your team's standards.