TASK-FIN-103-01 · Task family
Point-in-time factor reconstruction & leakage audit. Temporal reasoning, data lineage and feature engineering.
Overview. Rebuild the factor so every input was available at its decision date, and report how the results change.
A factor notebook joins fundamentals to prices by fiscal period end, so each rebalance uses figures that weren't published yet. The agent has to find the leak and rebuild the factor on point-in-time inputs.
Restated values add a second leak: the notebook uses the latest value, not the one first reported. The corrected factor must use vintages and filing dates.
Value to the buyer
Look-ahead makes backtests look better than they are. Trains agents to find it in someone else's code.
Specification. The task family, field by field.
- task_id:
- TASK-FIN-103-01
- domain:
- Quantitative Research
- parent_environment:
- ENV-FIN-103 · Systematic Alpha Research
- target_role:
- Quantitative Researcher
- objective:
- Rebuild the factor so every input was available at its decision date, and report how the results change.
- starting_context:
- A research notebook, a fundamentals database with filing dates and vintages, daily prices and the factor specification.
- tools:
inspect_research_notebookget_fundamental_vintagequery_market_dataexecute_pythonrun_backtestsubmit_research_artifacts
- required_actions:
- Trace how the notebook joins fundamentals to prices
- Rebuild the join on filing dates and first-reported vintages
- Recompute the factor and rerun the backtest
- Report the differences
- expected_outputs:
- Corrected factor values
- Revised backtest
- Report on the leak and its effect
- interaction:
- Single agent with sandboxed code execution.
- verification:
- The verifier checks every factor input's availability date and reruns the submitted code.
- reward:
- Any input used before availability scores zero. Credit for correct factor values and a reproducible backtest.
- difficulty:
- Calibrated against reference models during acceptance testing
- task_horizon:
- Long multi-step coding task. Execution limits are agreed with the buyer.
- delivery:
- Task instances with seeded data, reference implementations and verifier code.
- commercial_unit:
- Per validated task instance
Sample prompt. What the agent is asked to do, as a buyer would see it.
A book-to-market factor notebook reports unusually strong returns over its test period. Review how it joins quarterly book equity to monthly rebalance dates, identify any information that would not have been available at each decision date, and reproduce the factor using valid point-in-time inputs. Deliver corrected factor values, a revised backtest and a report explaining material differences from the original results.
Verification criteria. What the verifier checks, and how.
Period-end join found
The notebook uses book equity from the fiscal quarter end, weeks before filing.
Vintage leak found
Restated values replace first-reported ones in history.
Filing-date join
Each value is used only from the trading day after its filing.
Vintages used
First-reported values are used until a restatement is published.
Reproducible
The verifier's rerun matches the submitted factor values and backtest.
Report
Quantifies how much of the original performance came from the leaks.
Key failure modes. Where agents go wrong on this task.
- Adding a fixed lag that is shorter than some filing delays
- Fixing the join but keeping restated values
- Reporting the corrected backtest without explaining the difference
Other task families. The rest of Systematic Alpha Research, on the same FinanceWorld state.
- TASK-FIN-103-02
Strategy implementation & backtest repair
Quantitative coding, financial calculations and translating a strategy into code.
Quantitative Developer
- TASK-FIN-103-03
Market data integrity & corporate action handling
Data validation, adjusted prices and historical consistency.
Data Scientist
- TASK-FIN-103-04
Strategy robustness & overfitting audit
Statistical testing, out-of-sample analysis and methodological validity.
Quantitative Researcher