From Contracts to Guardians: Why Research Agents Need a Second Opinion

Consider a user who asks an agent to build a market-neutral portfolio while keeping every position below a client-approved limit, but never supplies the limit. In one of our tests, the agent inferred 12.5% from the current portfolio and continued. The client later supplied the real number: 15%. The guess sounded conservative, but it changed which portfolios were feasible and which one the agent preferred.

Itoflow is the quantitative-research harness in this account. By harness, we mean the whole system that carries a task from request to answer: the main research agent, the Guardian, their instructions and context, the tools and data they can use, their permissions, the written contract and the deterministic checks that govern progress. The two agents are not visitors to that harness. They are parts of it, and each is a smaller harness of its own: a model coupled to the context, tools, permissions and operating rules needed for its job.

User request ↓

Itoflow quantitative-research harness

Main research-agent harness

LLM

Research instructions · user or mandate context · tools · workspace

work
⇄
review

Guardian-agent harness

LLM

Review instructions · selected evidence · read-only research tools

Harness around the agents · research contract · deterministic checks · data access · permissions · workflow and session continuity

Before research begins, the main agent writes a research contract: a structured plan that records what it believes the user wants, the constraints it will respect, the data and methods it expects to use, and the evidence it promises to produce. Deterministic checks in the surrounding harness validate the exact parts of that plan before research can proceed. This prevents a constraint stated at the start of a conversation from quietly disappearing later.

We described that design in From Conversation to Commitment. The contract solves a real problem, but the 12.5% guess exposed its limit: a deterministic check can confirm that a concentration limit exists without knowing whether the user actually chose it. The same problem appears when an agent tests twenty variations of one model instead of considering different explanations, or ignores evidence discovered after it wrote the plan.

We initially tried to encode more of those judgments as formal rules. The result was longer instructions, more contract fields and more repair loops. Agents spent time rewriting the plan into the required shape without gathering better evidence. The rulebook grew; the research did not improve with it.

What can a research contract check?

A research contract works well for questions with one verifiable answer. Is the plan complete? Does it refer to available data and methods? Do the portfolio weights satisfy the stated limits? Did the agent produce the evidence it promised? The harness's deterministic layer can answer each question the same way every time.

It cannot reliably decide whether the agent asked enough questions, whether “conservative” is a valid interpretation of a missing client limit, whether twenty nearby Ridge regressions constitute a broad enough search, or whether a five-year valuation answer accidentally used six years of earnings. Those judgments depend on meaning and context.

We could write a rule for every failure we had already seen, but it would only recognize familiar wording and familiar methods. The cost was already visible in production. In the 14-day sample we studied, 685 failed plan checks later passed; the median repair took 14.9 seconds, and 96.6% of the intervening tool calls edited, reread or revalidated the plan rather than gathering new research. Some repairs fixed real errors. Many only translated existing prose into the shape expected by the validator.

Why did our previous verifier fall short?

Before the Guardian, Itoflow had an independent verifier agent. It could inspect files, recompute results and reject weak work. It caught real defects, and we retained the strongest parts of its review instructions. The problem was where it sat in the process.

1. The research agent launched it. The agent being reviewed decided when review happened.
2. The research agent framed the assignment. An agent that had already narrowed the problem could ask the verifier to validate that narrower interpretation.
3. It arrived late. By the time review began, the plan, code and analysis already reflected the research agent's assumptions.

Review also became entangled with contract bookkeeping. The agent sometimes spent a large part of a turn packaging claims into machine-shaped evidence records so the workflow could close. That work improved the record of the research more often than the research itself. We reduced the problem over time, but the agent still managed the reviewer it was supposed to satisfy.

What did we borrow from Codex Auto-review?

OpenAI's Codex can automatically pause before a risky computer action and ask a separate model to review it. This feature is called Auto-review. The software—not the coding agent—chooses when review happens and what evidence the reviewer sees. The reviewer can investigate, but it cannot take the proposed action itself.

OpenAI's alignment write-up makes the motivation explicit: the coding agent is trying to finish the task, while the reviewer has the narrower job of challenging a proposed action. Codex's public Guardian policy template asks the reviewer to interpret what the user meant and what they authorized, rather than matching an exact command string.

Quantitative research required four adaptations.

• Review research decisions, not shell commands. Itoflow invokes review when the agent commits to a plan and when it says the answer is ready.
• Give the reviewer more depth. We invoke it far less often than a coding sandbox may review commands, so it can inspect plans, code, data, calculations, results and source evidence.
• Keep the same reviewer. The Guardian follows the work from the first plan through later repairs and the final answer. It remembers its concerns and can distinguish a real fix from an argument.
• Match the instructions to the job. An interactive research chat and a self-directed portfolio strategy should handle missing decisions differently.

When does the Guardian review the research?

The Guardian reviews the work twice: before research begins and before the final answer is released. The Itoflow harness launches both reviews automatically, so the main agent cannot skip them or narrow their scope.

The Guardian either accepts the work or asks for revision. A request for revision blocks progress and tells the research agent what must change. If the reviewer itself fails repeatedly—for example because its model is unavailable—Itoflow records that failure and eventually lets the research continue rather than trapping the user in an endless loop. Exact safety and portfolio constraints still apply.

The Guardian runs as a separate agent with read-only workspace access and approved research tools. It cannot edit files, trade, ask the user or launch more agents. On its first review it receives the user's messages, the research agent's visible reasoning, successful tool calls and results, and the complete current plan. Later reviews add what happened since the previous review while always including the full current plan or answer.

Before release, the Guardian sees the draft answer and can inspect the supporting code, data, calculations and source material itself. The original plan is useful context, but matching the plan is not the goal. The reviewer judges the research as a whole: whether the method is sound, the calculations are correct, the sources support the claims, new evidence was handled properly and the answer is useful.

The interface shows that the Guardian is working during both reviews. When it finishes, the user can see the decision, summary, material concerns and requested changes. The Guardian's private reasoning and internal evidence stay private.

Why do chat and strategy agents need different review?

A chat agent can ask the user to resolve an ambiguity; an autonomous portfolio strategy must often decide without waiting for a reply. Applying one policy to both would make chat too presumptuous or strategy too indecisive. Each therefore receives review instructions suited to its job.

Interactive chat favors asking when a missing user decision changes feasibility, the objective or the meaning of the answer. The Guardian asks the research agent to pose the smallest useful question. It does not demand clarification for harmless ambiguity or follow a fixed questionnaire.

Autonomous strategy is judged against its investment mandate and previous decisions. When the mandate allows discretion, it should make a reasonable choice and continue. When a missing decision would exceed that authority, it should take a narrower action or make no change.

The same principle applies to other agents: keep a common standard for evidence and reasoning, then adapt the review to the decisions that agent is allowed to make.

What changed in the research traces?

We judged the change primarily from traces: what the reviewer challenged, how the research agent repaired the work, and whether the repair improved the research. Scores provide a second check.

An invented client constraint

In a market-neutral allocation task, the user required every position to remain under a client-approved concentration limit but did not disclose the number. The research agent inferred 12.5% from the current portfolio. The plan passed every exact check. The Guardian rejected the unsupported assumption.

The simulated client answered 15%. The Guardian then caught a materially underspecified estimator, incomplete factor-risk accounting, and a historical validation claim that used descriptors not shown to exist at the time. The reviewed run scored 0.915; the run without review kept the 12.5% guess and scored 0.552.

Leakage hidden inside ordinary validation

A historical test should use only information that was available at each point in the past. In a drawdown-aware allocation task, the method chose which assets to include using information from outside the test period. The code ran and the plan passed, but the comparison was not genuinely out of sample. The Guardian required the agent to repeat selection and optimization independently inside each period.

The wrong optimization problem

In a portfolio-transition task, the agent said it could reuse an existing regression function and optimizer. Inspection showed that the regression function did not exist. The optimizer also solved a different problem: it minimized distance from target weights using cash, while the user had asked to maximize expected return relative to a benchmark after asset-specific costs and portfolio limits. The repair removed the nonexistent function and solved the requested objective directly, then recomputed every constraint independently.

Across those seven tasks, the reviewed runs had a mean score of 0.693 versus 0.568 without review; medians were 0.783 and 0.552. The traces show what changed: unsupported user constraints became questions, historical tests stopped using future information, nonexistent library functions were removed, and optimizers were redirected to the user's stated economic objective.

What did this let us remove from the research contract?

The Guardian let us remove contract fields that consumed attention without producing better research. We introduced the reviewer first, then removed one requirement at a time and reran tasks that depended on it.

The first change concerned custom quantitative code. We still require the agent to say whether it reused a tested implementation from our research library or wrote a new one. But it no longer fills out a large structured justification for every custom method. When the agent writes new code, the Guardian checks whether it searched the library seriously and whether the bespoke method is proportionate to the question.

We also stopped requiring every agent to represent its possible approaches as a formal graph. A graph can still help with quantitative model selection, and the harness validates it when the agent uses one. It is no longer the compulsory shape of good thinking. The Guardian instead checks the substantive question: did the agent consider meaningfully different explanations and methods before committing?

Finally, qualitative claims no longer need a separate machine-shaped evidence record merely to close the workflow. The deterministic layer still checks exact identities, portfolio feasibility and quantitative comparisons that can be verified mechanically. The Guardian reviews the quality of sources and reasoning. In one cross-asset rerun, required evidence records fell from twenty to two while the reviewer still caught a horizon mismatch, weak validation and a heuristic described as an optimizer. In a stock analysis, it caught a six-year earnings series described as five years and a rise in valuation described as a fall.

Where did the Guardian make the result worse?

The relative-value task is the counterexample. The Guardian surfaced missing client constraints, required sensitivity tests across recovery, impairment, volatility paths and dependence, and challenged an unsupported claim that the search was exhaustive. The research agent still chose a 100% cash solution. It was feasible but economically weak; the run without review chose a four-name equity sleeve and scored 0.726 higher.

Our claim is therefore limited. The Guardian improved the research process in several observable traces and improved the score on six of seven matched tasks. It did not guarantee a better investment conclusion. Codex Auto-review likewise cannot guarantee that every unsafe action will be stopped. An agentic reviewer can reduce a class of failures; it cannot make a stochastic system deterministic.

What else could an automatic reviewer check?

We started with the plan and the final answer because they mark two commitments: “this is the work I intend to do” and “this work is complete.” Other high-risk moments may benefit from the same pattern without reviewing every routine action.

A substantial Python analysis could trigger a non-blocking quantitative review: did the agent reuse an existing implementation, introduce leakage, confuse gross and net returns, or reinvent a mechanism already present in the library? A portfolio transition could receive a specialized feasibility check before any recommendation reaches the user. A reviewer following a long-running strategy could carry unresolved concerns between executions and distinguish a deliberate mandate revision from silent drift.

The harness now has a clearer division of responsibility. The research contract records stable commitments and exact constraints. The deterministic layer enforces what can be checked exactly. The Guardian reviews whether those commitments fit the request and whether the completed work supports the answer. This leaves the main agent with less paperwork and more capacity for research.

When we published From Conversation to Commitment, the formal contract was our answer to conversational drift. It remains the source of truth for exact commitments. The Guardian now reviews the judgment around it. We will remove further contract rules only when traces show that the Guardian handles them better.

How did we test and source this?

• OpenAI, “Auto-review” — the Codex reviewer design that inspired our automatic review pattern.
• OpenAI Alignment, “Auto-review: A safety layer for agentic systems” — the case for separating the task-completion role from the review role, and the limits of stochastic review.
• OpenAI Codex, public Guardian policy template — review of whether a proposed action matches the user's intent and the available evidence.
• Itoflow, “From Conversation to Commitment” — the written research-plan design that the Guardian now complements.
• Itoflow, “Async Critique Agents” — our earlier work on independent, temporally separated agent review.
• Itoflow, “What Our Quant-Research Agent Actually Does” — the research process whose quality the Guardian is designed to protect.