Skip to content
ARCHYTASAI

TASK-SWE-103-01 · Task family

Non-local bug discovery. Trace defects across files, services and state transitions.

Custom buildenvironment · ENV-SWE-103 ReviewGuardrole · Code Reviewer

Overview. Find the regression the change introduces outside the diff, prove it, and request changes.

A small, clean-looking diff passes every existing test. The defect it introduces only shows up in code the diff doesn't touch.

The agent has to look past the diff to callers and writers of the same data, and back any finding with a reproduction rather than a guess.

Value to the buyer

Non-local regressions are the bugs review agents miss most. Trains them to read beyond the diff.

Specification. The task family, field by field.

spec · task/task-swe-103-01.yamlENV-SWE-103
task_id:
TASK-SWE-103-01
domain:
Code Review & Regression Prevention
parent_environment:
ENV-SWE-103 · ReviewGuard
target_role:
Code Reviewer
objective:
Find the regression the change introduces outside the diff, prove it, and request changes.
starting_context:
A PR that adds caching to the customer account service, its description, the repository and the test suite.
tools:
  • read_pull_request
  • inspect_diff
  • search_repository
  • inspect_callers
  • read_tests
  • run_reproducer
  • submit_review_comment
  • request_changes
required_actions:
  • Find every reader and writer of the cached data
  • Check that writes invalidate the cache
  • Write a minimal reproduction
  • Submit one evidence-backed finding
expected_outputs:
  • Review finding with location, failure, reproduction and fix
  • Decision on the PR
interaction:
Single agent in a review sandbox.
verification:
The finding is matched to the verified defect and the reproduction is executed.
reward:
Credit for the defect, its location, evidence and severity. Unsupported extra comments cost credit.
difficulty:
Calibrated against reference models during acceptance testing
task_horizon:
Medium investigation across the repository. Execution limits are agreed with the buyer.
delivery:
Task instances with PRs, verified defects, reproducers and verifier code.
commercial_unit:
Per validated task instance

Sample prompt. What the agent is asked to do, as a buyer would see it.

prompt · task-swe-103-01Example
Review a pull request that adds a 5-minute in-process cache to AccountService.get_account(). The change passes all existing unit tests. Evaluate whether it introduces a regression in downstream consumers, whether the tests cover the changed behaviour, and whether any review comments are warranted. Only report defects supported by code evidence, reproducible behaviour or a documented contract.

Verification criteria. What the verifier checks, and how.

  • Writer found

    billing/contacts.py:update_billing_email writes the account without invalidating the cache.

  • Impact

    Invoices can go to the old billing address for up to 5 minutes after a change.

  • Reproduction

    A test updates the email, generates an invoice and shows the old address.

  • Fix

    Invalidate on write, or route writes through AccountService.

  • Precision

    No speculative comments on the TTL value or naming.

Key failure modes. Where agents go wrong on this task.

  • Approving because the tests pass
  • Commenting only on the diff's style
  • Flagging "possible stale data" with no affected caller or reproduction

Other task families. The rest of ReviewGuard, on the same EngineeringWorld state.

Every environment is built to order. Tell us which workflow you need to train or evaluate, and where you are in the process.

Request sample tasksDiscuss custom developmentRequest technical integration detailsSee ways to buy