Skip to content
ARCHYTASAI

TASK-SWE-101-01 · Task family

Specification-faithful code generation. Implement the stated behaviour without adding assumptions the contract doesn't make.

Custom buildenvironment · ENV-SWE-101 SpecForgerole · Software Engineer

Overview. Implement the specification exactly: every stated behaviour, and no unstated restriction.

The agent implements a function or endpoint from a written specification. The specification is complete for the behaviour it covers and deliberately silent on some details.

Agents often fill silences with their own assumptions: an upper bound, a format, an error. A hidden oracle checks the stated behaviour and also checks that the implementation doesn't reject inputs the contract allows.

Value to the buyer

Trains agents to implement the contract, not their own idea of it, which is where reviewers catch most generated-code defects.

Specification. The task family, field by field.

spec · task/task-swe-101-01.yamlENV-SWE-101
task_id:
TASK-SWE-101-01
domain:
Coding & Test Engineering
parent_environment:
ENV-SWE-101 · SpecForge
target_role:
Software Engineer
objective:
Implement the specification exactly: every stated behaviour, and no unstated restriction.
starting_context:
A repository with a stub, a written specification and a small set of example tests.
tools:
  • read_specification
  • inspect_repository
  • write_patch
  • run_tests
  • submit_task
required_actions:
  • Read the specification and list what it requires and what it leaves open
  • Implement the stated behaviour
  • Test the examples and the edge cases the specification mentions
expected_outputs:
  • Implementation
  • Notes on behaviour the specification leaves unspecified
interaction:
Single agent in a repository sandbox.
verification:
Hidden behavioural checks, including inputs the specification allows but the examples don't show.
reward:
Credit per behaviour implemented. Rejecting valid input scores zero on that check.
difficulty:
Calibrated against reference models during acceptance testing
task_horizon:
Short to medium coding task. Execution limits are agreed with the buyer.
delivery:
Task instances with specifications, reference implementations, hidden checks and verifier code.
commercial_unit:
Per validated task instance

Sample prompt. What the agent is asked to do, as a buyer would see it.

prompt · task-swe-101-01Example
Implement parse_duration(text) in the scheduler package. The specification says a duration is one or more number-and-unit pairs using h, m and s, optionally separated by spaces, and returns total seconds: "1h30m" is 5,400. Empty strings and negative numbers raise ValueError. The specification sets no limit on the size of each part. Implement it and explain any choice the specification leaves open.

Verification criteria. What the verifier checks, and how.

  • Stated examples

    "1h30m" returns 5,400 and "45s" returns 45.

  • Spacing

    "1h 30m" is accepted, as the specification allows.

  • No invented limit

    "90m" returns 5,400. Rejecting minutes above 59 adds a rule the contract doesn't have.

  • Errors

    Empty strings and negative numbers raise ValueError.

  • Open choices noted

    The notes say the contract doesn't cover repeated units such as "1h1h".

Key failure modes. Where agents go wrong on this task.

  • Rejecting "90m" because minutes normally stop at 59
  • Returning a timedelta when the contract says seconds
  • Silently accepting negative numbers

Other task families. The rest of SpecForge, on the same EngineeringWorld state.

Every environment is built to order. Tell us which workflow you need to train or evaluate, and where you are in the process.

Request sample tasksDiscuss custom developmentRequest technical integration detailsSee ways to buy