[DRAFT] How to Set Up an AI Code Review Agent with OpenHands

Written by

OpenHands Team

Published on

Build an AI agent that reviews pull requests, follows your coding standards, and automatically posts feedback to GitHub.

Code review is essential to shipping reliable software, but it can also slow development down. Every pull request needs attention, and reviewers have to understand what changed, identify potential bugs, check for security vulnerabilities, and make sure the code follows established conventions.

AI coding agents can take on much of that initial work.

Instead of manually reviewing every change, you can configure an agent to perform an initial review of pull requests, identify potential issues, and post actionable feedback or propose code changes directly in your version control.

With OpenHands Agent Canvas, you can set up this workflow in minutes. You don’t need to build a custom agent framework, write a GitHub Actions workflow, or manage a collection of scripts.

Let’s walk through how to get started, customize your agent, and automate the review process.

Note: This tutorial uses GitHub. The same agent, skill, and automation pattern can be adapted to other version control systems, although integrations and trigger configuration may differ.

The quick way: Set up code review with one prompt

Already running Agent Canvas? You can skip straight to this step.

Before you begin: Make sure Agent Canvas is installed and running, and that your agent has access to the GitHub repository you want to review. The GitHub connection must have permission to read pull requests and post review comments. You’ll also need a configured model for your agent.

Open Agent Canvas, start a conversation with your agent, and enter:

Set up a PR review automation for Organization/Repo.

Use the OpenHands `code-review` skill to review pull requests for:
- Logic errors and potential bugs
- Security vulnerabilities
- Performance regressions
- Missing edge cases and test coverage

Prioritize actionable findings that could affect production behavior. Avoid minor formatting and style suggestions.

Post review findings directly to the GitHub PR, including the issue, its severity, and a recommended fix.

Run when a pull request is opened, reopened, marked ready for review, or gets new commits. Skip draft pull requests.

Before posting feedback, check existing review comments and avoid repeating finding that have already been reported. When a pull request receives new commits, focus on changes introduced since the previous review.

If this Agent Canvas cannot receive GitHub webhooks (for example, it runs on localhost), check for new or updated pull requests every 10 minutes instead.

Replace Organiztion/Repowith your GitHub organization and repository.

Agent Canvas can help configure the automation using the OpenHands code review skill and your connected GitHub repository. Review the generated configuration, confirm the trigger and repository permissions, and enable the automation.

One important detail: How the automation runs depends on where your agent is hosted.

  • Cloud-hosted: When running in an online environment with GitHub event integrations configured, the automation can respond to pull request events.

  • Local: If you’re running Agent Canvas locally, you can configure the automation to poll GitHub every ten minutes for new or updated pull requests. Your machine and agent environment need to remain running for scheduled reviews to execute.

Once configured and tested, the workflow can review new or updated pull requests without requiring you to manually prompt the agent each time.

But what exactly did we configure, and how can you make the reviews more useful?

How the code review agent works

An AI code review agent does more than evaluate a code snippet. It can inspect changes in the context of a repository, examine related files, trace dependencies, and identify potential issues.

The workflow looks like this:

PR opened → Automation starts → Agent reviews code → Findings posted to GitHub → Developer addresses issues

OpenHands uses three components to make this work:

ComponentWhat it does
AgentInspects code, reasons about changes, and identifies potential issues
SkillDefines how the agent should review code and what standards it should follow
AutomationDetermines when the agent runs, whether triggered by GitHub events or scheduled polling

The agent performs the work. The skill defines how the review should be conducted. The automation makes sure it happens at the right time.

Separating these components makes the workflow reusable. You can update your review criteria, switch models, or change when reviews run without rebuilding the agent.

1. The agent: Inspecting and reasoning about code

The agent is responsible for analyzing the pull request.

Traditional static analysis tools identify issues through predefined rules and program analysis. An AI code review agent takes a different approach, using a language model to evaluate changes in context, explain potential problems, and suggest fixes.

For example, suppose a pull request introduces this function:

def get_user(user_id):
    query = f"SELECT * FROM users WHERE id = {user_id}"
    return db.execute(query)

A useful review might identify that the query directly interpolates user-controlled input, creating a potential SQL injection vulnerability.

The agent could recommend a parameterized query instead:

def get_user(user_id):
    query = "SELECT * FROM users WHERE id = ?"
    return db.execute(query, (user_id,))

The exact parameter syntax depends on the database driver.

The important distinction is that the agent doesn’t simply flag a potential problem. It can explain why the issue matters and recommend a specific fix.

2. The skill: Defining how your agent reviews code

An LLM can review code when prompted, but that doesn’t mean it will consistently produce useful feedback.

Without clear instructions, models may focus too heavily on formatting, suggest unnecessary refactors, or miss important issues.

Skills help make reviews more consistent.

OpenHands provides a reusable code review skill that you can add to your agent:

/add-skill https://github.com/OpenHands/extensions/tree/main/skills/code-review

You can invoke the skill manually using:

/codereview

For example:

/codereview

Review PR #42 in my-org/my-repo.

Focus on correctness, security, performance,
and reliability issues.

Only report findings supported by the code.
Prioritize actionable feedback over style suggestions.

This is useful when you want to test your review process before automating it.

Customizing the skill for your repository

Every engineering team has different standards.

A backend service might prioritize API compatibility, database performance, and error handling. A frontend application might focus on accessibility, state management, and rendering performance.

You can create repository-specific instructions to guide the agent.

For example:

# Code Review Guidelines

Review pull requests for correctness, security,
reliability, and maintainability.

## Priorities

1. Identify logic errors and unintended behavior.
2. Flag security vulnerabilities.
3. Check error handling and edge cases.
4. Identify potential performance regressions.
5. Evaluate test coverage.

## Rules

- Focus on meaningful issues.
- Do not flag formatting handled by linters.
- Do not suggest refactoring without a clear benefit.
- Explain why each issue matters.
- Reference the relevant file and line.
- Recommend a fix when possible.
- Do not invent issues to fill out the review.

You can maintain these guidelines as part of your repository’s agent configuration or in a customized code review skill, depending on how your OpenHands environment is configured. This lets you update review criteria as your engineering standards evolve.

You can also ask your agent to create or modify the skill directly in Agent Canvas.

For example:

Update my code review skill to prioritize
API compatibility, database performance,
and error handling.

Avoid stylistic feedback unless it affects
correctness or maintainability.

Apply these instructions to future PR reviews.

This makes it easier to improve the review process without rebuilding the automation.

3. The automation: Running reviews without prompting

The automation determines when the agent runs.

For code review, that typically means responding to GitHub activity.

You can configure the workflow to review pull requests when they’re opened, updated, or explicitly marked for review.

The execution model depends on where you’re running OpenHands.

If your agent is hosted online, you can use GitHub event integrations to trigger reviews as events occur.

If you’re running locally, you can configure scheduled polling to check for new pull requests every five minutes.

Both approaches use the same underlying agent and review instructions.

The difference is how the automation detects new work.

This also means you can start experimenting locally and move the workflow to a persistent environment when you want it running continuously.

Test your code review agent

Before enabling automatic reviews across your repository, test the agent on an existing pull request.

Ideally, choose one that has already been reviewed by a developer. That gives you a baseline for evaluating its findings.

Ask your agent:

Review PR #42 in Organization/RepoName using
our code review skill.

Identify correctness, security, and reliability
issues.

Only report findings supported by the code.
Include the severity, potential impact, and
recommended fix for each issue.

Then evaluate the results.

Did it identify real issues? False positives quickly undermine developer trust.

Did it miss important problems? Compare the findings against the original human review.

Was the feedback actionable? A good review should explain what’s wrong, why it matters, and how to address it.

If the agent produces too much noise or overlooks important issues, update your skill and test again.

Verify the automation

Once you’re satisfied with the review quality, test the automation itself.

  1. Enable the automation for a test repository.

  2. Open or update a pull request that matches your trigger conditions.

  3. Confirm that the automation starts.

  4. Check that the agent can access the repository and analyze the changes.

  5. Verify that feedback appears on the correct GitHub pull request.

  6. Check the automation’s execution history for errors.

If the automation doesn’t run, verify the trigger configuration, GitHub permissions, and agent environment.

Beyond code review: Automate more of your SDLC

Code review is a useful starting point, but the same approach can support workflows across the software development lifecycle. Once you have an agent, reusable skills, and a way to trigger execution, you can automate other engineering tasks.

  • Issue triage: An agent analyzes incoming issues, categorizes them, and identifies the information needed to reproduce a problem.

  • CI failure remediation: An agent investigates failed checks, identifies likely causes, and proposes fixes.

  • Dependency maintenance: An agent evaluates dependency updates, checks for breaking changes, and prepares pull requests.

  • Documentation updates: An agent identifies code changes that require documentation changes and proposes updates.

These workflows can share skills, tools, and integrations while remaining independently configurable. For example, an automation that investigates a CI failure could propose a fix and open a pull request, which then triggers a separate code review workflow. Developers remain responsible for reviewing and approving changes according to their team’s policies.

The goal isn’t to replace developers or remove human oversight. It’s to automate repetitive engineering work so developers can focus on the decisions that require their expertise.

Build your next agent with OpenHands

You don’t need to build a custom agent framework or replace the tools you already use to get started.

OpenHands provides an open-source foundation for running AI coding agents, defining reusable skills, and building automated engineering workflows.

Start locally with Agent Canvas. Experiment with models and skills, connect your repository, and test the results. Once your workflow produces useful results, turn it into an automation that runs continuously in the cloud or on your own infrastructure.

When you’re ready to expand beyond a single repository, OpenHands gives you a path to running and managing agents across teams and environments.

Ready to get started? Explore OpenHands Agent Canvas and the Automations documentation.

Get useful insights in our blog

Insights and updates from the OpenHands team

Sign up for our newsletter for updates, events, and community insights.

By submitting your email you agree to our Privacy Policy