TASK-SWE-101-04 · Task family
Requirement evolution & regression preservation. Update code and tests as requirements change without breaking existing contracts.
Overview. Implement the new requirement, update only the tests it makes stale, and keep every unchanged contract protected.
A specification changes. Some existing tests now encode a requirement that was explicitly replaced, while others still protect contracts that haven't changed.
The agent has to change the code, update the stale tests, and leave the rest alone. Deleting failing tests until the suite is green fails the task.
Value to the buyer
Real codebases change. Trains agents to separate stale tests from tests that still matter.
Specification. The task family, field by field.
- task_id:
- TASK-SWE-101-04
- domain:
- Coding & Test Engineering
- parent_environment:
- ENV-SWE-101 · SpecForge
- target_role:
- Software Engineer
- objective:
- Implement the new requirement, update only the tests it makes stale, and keep every unchanged contract protected.
- starting_context:
- An invoicing package, specification v1 and v2 with a change log, and the current test suite.
- tools:
read_specificationinspect_repositoryread_test_suitewrite_patchwrite_testrun_testssubmit_task
- required_actions:
- Diff the specification versions
- Identify tests that encode replaced requirements
- Change the implementation and those tests
- Confirm unchanged contracts still pass
- expected_outputs:
- Updated implementation and tests
- List of tests changed, each tied to a specification change
- interaction:
- Single agent in a repository sandbox.
- verification:
- Hidden checks for the new behaviour and for every unchanged contract, plus a check that no protecting test was removed.
- reward:
- Credit per updated behaviour and preserved contract. Removing a test that protects an unchanged contract scores zero on that check.
- difficulty:
- Calibrated against reference models during acceptance testing
- task_horizon:
- Medium coding task. Execution limits are agreed with the buyer.
- delivery:
- Task instances with versioned specifications, reference changes and verifier code.
- commercial_unit:
- Per validated task instance
Sample prompt. What the agent is asked to do, as a buyer would see it.
Specification v2 changes how discounts round: v1 rounded each discounted line to the cent, v2 rounds only the invoice total, half up. An invoice with three lines of $3.335 used to total $10.02 and should now total $10.01. Update the implementation and the tests. Currency handling and tax rules haven't changed.
Verification criteria. What the verifier checks, and how.
New rounding
Three lines of $3.335 total $10.01 under v2.
Stale tests updated
Tests asserting per-line rounding are changed, citing the v2 change log.
Contracts preserved
Currency and tax tests are untouched and pass.
No deletions
No test protecting an unchanged contract is removed.
Decimal arithmetic
The implementation uses decimal arithmetic, not floats.
Key failure modes. Where agents go wrong on this task.
- Deleting every failing test
- Rounding with floats, which gets half-up cases wrong
- Changing tax tests that happened to fail for an unrelated reason
Other task families. The rest of SpecForge, on the same EngineeringWorld state.
- TASK-SWE-101-01
Specification-faithful code generation
Implement the stated behaviour without adding assumptions the contract doesn't make.
Software Engineer
- TASK-SWE-101-02
Test suite soundness & overconstraint audit
Detect false failures, repair overly strict tests and keep the suite's fault detection.
SDET
- TASK-SWE-101-03
Red–green–refactor TDD
Write a meaningful failing test, implement the behaviour, then refactor without regressions.
Software Engineer