Hengle JIANG

Hengle JIANG

PhD student @SUSTech

Research essay · Agent architecture

When the Model Changes, What Happens to the Agent?

Model swaps, version upgrades, and the gap between the model we optimize for and the one we deploy.

An agent has been tuned for weeks. Its prompts are stable, its tools work, and its evaluation looks good. Then the production model changes: a cheaper provider, a new version of the same family, or a smaller model chosen to meet latency requirements. The API still accepts the request. The system no longer behaves the same way.

A search that once returned a concise answer becomes an exploratory loop. A model that used to wait for a tool result now emits several calls at once. A summarizer drops an unresolved constraint. A retry that helped the old model causes the new one to repeat a side effect. None of these requires a dramatic drop in general intelligence.

This is the problem I want to understand: what exactly have we optimized when we say that an agent system works? Is the improvement a reusable property of the system, a compensation for one model’s habits, or an interaction that disappears when either component changes?

An agent’s behavior belongs to the combination of model, harness, environment, and budget. Portability means preserving the task contract while measuring and limiting the work required to adapt that combination.

01 / The model is only one part of the deployed policy

I use harness for the model’s surrounding execution system: prompts, context assembly, memory, tool interfaces, routing, validators, retries, and stopping rules. A task outcome depends on all of these, together with the workload and available resources.

J(M, H, D, B, E) = E[task utility | model M, harness H, workload D, budget B, environment E]

Even when code H stays fixed, replacing M changes the distribution of actions. Those actions change the environment and the observations fed into later decisions. A model swap therefore changes both the decision maker and the situations it will encounter.

Four changes deserve separate treatment.

Change Typical reason What needs to be re-established
Same family, new version Upgrade or provider retirement Behavioral compatibility, including previously reliable edge cases
Different model or provider Cost, latency, availability, capability Interface semantics, task quality, operating budget
Tuned on A, deployed on B Strong model used during development; cheaper executor in production Whether the gains survive on B, and the adaptation cost
Model switch during a session Routing, escalation, outage fallback Whether B receives sufficient state and understands pending actions

“A designed the workflow for B” is also different from “the workflow was evaluated and optimized on A, then copied to B.” A strong builder can propose useful code or examples for a weaker executor. The relevant selection signal is still B’s performance. The identity of the designer does not determine the target of optimization.

Historical work comparing March and June 2023 ChatGPT versions documented changes across tasks and instruction following.1 It supports behavioral regression testing. It does not let an external observer attribute a particular regression to a changed training-data mixture or reward function. For a black box, the actionable evidence is the observable contract and its failures.

02 / The dependencies hidden inside a “working” system

Some dependencies are obvious: a parser expects a field name, or a provider represents tool errors differently. Others are behavioral. The system assumes the model will search before answering, preserve an exception in a summary, recover after a failed edit, or stop once a task is complete.

A patch added during development can quietly turn one of these habits into a requirement. The old model ignores a constraint, so the prompt repeats it. The new model follows every repetition, checks the same condition several times, and runs out of time. A local repair has become a source of coupling.

Dependency What can change after a swap Repair at the right layer
Output and tool protocol Missing fields, different stop events, parallel calls Normalize messages and validate schemas; preserve error semantics
Planning and stopping More exploration, premature completion, excessive checking Explicit completion evidence and bounded execution policies
Context and memory Different use of summaries, tool outputs, or examples Store facts and unresolved obligations with provenance; test reconstruction
Recovery Repeating a failed action or treating a business rejection as an outage Classify errors; define which actions can be retried safely
Budget More reasoning, longer tool loops, higher tail latency Measure quality against time and cost; tune operating points
Evaluation A new answer style pleases the judge but violates the task Independent end-state checks and audited judging examples

The loop amplifies these differences. A slightly different first search changes the retrieved documents; that changes the plan, tool calls, accumulated context, and eventual stopping decision. Measuring only the first response misses the main effect.

Figure 1 · A model swap changes the trajectory distribution. Conceptual diagram. The same initial request can lead to different future states; a recorded transcript covers only one path.
Figure 1 · A model swap changes the trajectory distribution. Conceptual diagram. The same initial request can lead to different future states; a recorded transcript covers only one path.

This explains why replay and live execution answer different questions. Feeding both models an identical recorded context is useful for locating a protocol or decision difference. It cannot reveal the states reached by B’s own earlier actions. Closed-loop runs are needed for end-to-end performance; controlled replay is useful for diagnosis.

03 / Portability is possible—and selective

The literature gives evidence in both directions. Sclar et al. found substantial sensitivity to meaning-preserving prompt formatting, with weak correlations in format performance between models.2 A shared prompt can therefore favor one model without being a neutral comparison.

SWE-agent provides a positive counterpart. Its agent-computer interface was developed around GPT-4 Turbo and also worked with Claude 3 Opus; on SWE-bench Lite, its interface improved resolution by 10.7 percentage points over a shell-only baseline.3 Useful interface structure can transfer even when the best wording or operating budget changes.

Figure 2 · A frozen harness can help other models. AHE v3, Terminal-Bench 2, 89 tasks: seed versus evolved harness, with no target-model re-evolution. The harness was evolved using GPT-5.4 high. Source: [4], Section 4.3.
Figure 2 · A frozen harness can help other models. AHE v3, Terminal-Bench 2, 89 tasks: seed versus evolved harness, with no target-model re-evolution. The harness was evolved using GPT-5.4 high. Source: [4], Section 4.3.

AHE’s cross-family gains range from 5.1 to 10.1 percentage points. Its component study also finds that transplanting the system prompt alone can regress. The report notes timeout-budget coupling: a configuration tuned for one reasoning level need not suit another.4

AI4AI at Test-Time studies a different setup: stronger builders construct harnesses for specified target models. In its theory-of-mind experiments, a stronger target regressed in 9 of 20 matched builder-by-benchmark comparisons, despite positive aggregate gains.5 These are target-specific constructions, not a direct test of copying an A-tuned harness to B. They expose the risk of adding structure where a target already performs well.

The useful distinction is between reusable task knowledge and model-specific compensation. An order must not be refunded twice: a durable requirement. “Ask the model to reconsider exactly three times”: a behavioral intervention whose benefit needs to be re-established.

04 / Separate the replacement effect from the adaptation opportunity

Suppose HA was tuned using model A and HB using model B. Evaluate both harnesses with both models on the same held-out workload. The off-diagonal cells are essential.

Figure 3 · A crossed evaluation separates two decisions. Hypothetical success rates, not experimental results. Replacing A with B under H_A loses 20 points; adapting B to H_B gains 25 points. A positive interaction contrasts relative fit, not an intrinsic model ability.
Figure 3 · A crossed evaluation separates two decisions. Hypothetical success rates, not experimental results. Replacing A with B under H_A loses 20 points; adapting B to H_B gains 25 points. A positive interaction contrasts relative fit, not an intrinsic model ability.
Replacement = J(B,HA) − J(A,HA)
Adaptation = J(B,HB) − J(B,HA)
Interaction = [J(B,HB) − J(B,HA)] − [J(A,HB) − J(A,HA)]

In the illustrated matrix, the interaction is +35 percentage points. This tells us that the harness change helps B more than A under the tested conditions. It does not isolate a model’s context-free ability. HB is also merely the best configuration found under its search budget, not a known optimum.

The fixed-harness comparison answers the immediate deployment question: what happens if we replace the model today? The matched-adaptation comparison asks a different question: what could we obtain after giving each model a comparable tuning budget? Both are useful; reporting only one hides part of the decision.

Hold task instances, permissions, scorer, environment resets, and resource policies constant within each comparison. Report equal-dollar and equal-latency comparisons separately from equal-token or equal-turn comparisons. Tokens from different tokenizers, and turns containing different amounts of work, are not interchangeable resources.

This is more than bookkeeping. Anthropic reported a six-point Terminal-Bench 2.0 success difference between strict and uncapped resources while keeping the Claude model, harness, and tasks fixed.6 Infrastructure changes can look like a model effect.

05 / Decouple around contracts you can actually enforce

A single API wrapper solves transport compatibility. Behavioral portability needs a boundary between task requirements and model-specific means of satisfying them.

I would separate three layers: durable application state and requirements; an execution layer that mediates actions and records evidence; and a versioned model adapter for message rendering, tool representations, context packing, and operating settings.

Figure 4 · Separate stable task requirements from replaceable execution choices. Proposed architecture. The adapter may change how a request is expressed; it may not silently weaken permissions, success criteria, or required checks.
Figure 4 · Separate stable task requirements from replaceable execution choices. Proposed architecture. The adapter may change how a request is expressed; it may not silently weaken permissions, success criteria, or required checks.

Consider a support agent that must refund an eligible order exactly once, record the result, and notify the customer only after confirmation. The model may propose the action. The tool layer checks eligibility against authoritative state, uses an idempotency key, and records the transaction identifier. After a timeout, the controller queries the transaction before retrying. Completion depends on the persisted result, not the model saying “done.”

These checks remove specific failure paths if correctly implemented. They do not determine whether the model understood an ambiguous request, selected the right evidence, or produced a useful explanation. Deterministic code can enforce a formalized boundary; semantic judgment remains a measured component of the system.

Keep stable Allow model-specific adaptation Check at the boundary
Business predicates and permissions Wording, examples, tool descriptions Equivalent permitted actions and success conditions
Authoritative facts and event history Context windows, summaries, retrieval order Provenance, unresolved obligations, reconstruction accuracy
Side-effect semantics Serial or parallel scheduling where dependencies permit Idempotency, ordering, committed-state confirmation
Required delivery quality Reasoning effort, retry and escalation thresholds Held-out quality, tail latency, total cost
Evaluation contract Model invocation and protocol conversion No leakage of answers; stable grading criteria

A schema catches a missing refund identifier; it cannot prove that a valid-looking identifier names the intended order. A summarizer can emit a valid object while omitting a crucial exception. Protocol validation must therefore connect to business predicates and evidence, rather than ending at successful parsing.

Anthropic’s Managed Agents separates the model-and-harness “brain,” execution tools, and a durable session event log.7 That is a useful example of lifecycle decoupling: a harness can restart from external state. It does not by itself establish cross-model behavioral equivalence.

Mid-session migration needs the same discipline. Hand over the goal, authoritative facts, completed and pending actions, unresolved obligations, and supporting evidence. Treat free-form summaries as derived views. Two models seeing the same prose is weaker than both reconstructing the same actionable state.

06 / Optimize for the executor you will actually deploy

DSPy separates declarative program structure from prompts and demonstrations optimized against a metric.8 MIPRO jointly searches instructions and examples for multi-stage programs.9 These offer a useful pattern: preserve the program’s intended transformation, then optimize how a target model carries it out.

For an A-to-B migration, keep a neutral baseline, the frozen A-tuned system, and a B-adapted system. Give the adapted baseline a recorded development budget. Otherwise, an elaborate migration method may appear useful only because its competitor received no tuning.

A strong builder can generate candidate demonstrations, tool descriptions, or modules. Select them using B on development tasks, check them on unseen task families, and account for the builder’s cost. If you select every candidate using A, then run only the winner on B, you have tested transfer after source optimization; you have not optimized B.

There is also a capability floor. An adapter cannot conjure missing visual input, unavailable tool permissions, or reliable long-context reasoning by renaming fields. A target may need a smaller task, retrieval, additional verification, or escalation. If these changes still cannot meet the contract at an acceptable cost, reject that model for that workload.

At the other end, a stronger model may need less scaffolding. Test removal as well as addition: fewer repeated reminders, fewer forced planning stages, or a simpler recovery loop. Agentless’s localization–repair–validation pipeline is a useful baseline for asking how much autonomous orchestration a task actually needs.10

07 / The cheapest model is not necessarily the cheapest system

A cheaper call can trigger more calls, more validation, more escalations, and more human repair. Evaluate cost per valid completion alongside total spend and unmet demand; a low cost among the few successful tasks can conceal widespread failure.

For a simple illustration, let the strong route cost 8 normalized units per request. A cheap route costs 1, verification costs 0.2, and a fraction f of requests falls back to the strong route, which costs another 8. These are illustrative costs, not provider prices.

Chybrid = 1 + 0.2 + 8f    Chybrid < 8 ⇔ f < 0.85
Fallback fraction f Expected hybrid cost Share of the direct strong-route cost
10% 2.0 25%
50% 5.2 65%
85% 8.0 100%
100% 9.2 115%

This break-even calculation is useful only with a quality constraint. A verifier that misses bad outputs reduces observed fallback cost while degrading delivery. Measure false acceptance, unnecessary escalation, and the latency of the sequential fallback path. Also confirm that the fallback receives usable state rather than inheriting corrupted assumptions.

For a one-time adaptation cost K and validated recurring savings Δc per request, K/Δc is a simple amortization point when Δc is positive and workload conditions remain comparable. A model that retires before that volume is reached can erase the expected saving. Version lifetime is part of the economics of portability.

08 / Migrate a versioned system, not just a model name

A release should bind the model identifier, prompt and adapter versions, tool schemas, memory format, resource limits, routing policy, and grader version. Record provider revision identifiers where available. Pinning reduces ambiguity; it does not freeze external services or guarantee that the provider will keep a snapshot forever.

I would evaluate migration in layers, moving from cheap diagnostic tests to representative closed-loop execution.

Stage Main question Evidence to retain
Interface checks Can the system exchange valid requests, calls, results, and errors? Parsing failures, unsupported semantics, lost fields
Controlled replay Where do local decisions first diverge? Identical contexts, action validity, matched perturbations
Closed-loop paired runs Does the replacement complete the same work? Reset tasks, repeated trials, outcome transitions, side effects
Held-out adaptation test Did tuning repair the regression beyond development examples? Task-family separation, frozen configurations, tuning cost
Shadow and limited rollout Do offline results survive real traffic? Representative slices, latency tails, escalation, external-state checks
Rollback rehearsal Can the previous system resume safely? Compatible state, versioned memories, reconciliation of committed actions

An unchanged mean can hide different failures. If the old system succeeds on 80 of 100 tasks, and the new one fixes ten old failures but breaks ten old successes, both score 80%. The identities and costs of those regressions may determine whether the migration is acceptable. Repeat trials and report uncertainty rather than treating one stochastic pass/fail transition as a stable property.

τ-bench makes repeated-run reliability explicit through pass^k, the probability of consistent success across k trials.11 For migration, use both task-level transition analysis and reliability measures; distinguish model errors from infrastructure failures and missing evidence.

Shadow execution must isolate writes: running two refund agents against the same live order is not an independent comparison. Likewise, rolling back the model does not reverse an already issued refund or sent message. State reconciliation and compensating actions belong in the migration plan.

The acceptance rule should be chosen before reading target results: tolerated loss on each required workload slice, a bound on unacceptable actions, and cost or latency limits. Tail behavior and critical regressions should not disappear inside an average gain.

09 / What would make this a substantive research contribution?

The interesting object is the dependency that remains after API compatibility is solved. How much of a harness improvement transfers? Which components require retuning? How many target-model observations are needed to recover a specified quality level? Can we predict the failure before a full migration?

Research question Experiment A result that would weaken the claim
Which dependencies are portable? Cross model families with component swaps: prompts, tools, memory, recovery Gains depend mainly on one source family or disappear under matched budgets
Can compatibility probes predict closed-loop regressions? Freeze probes, then test unseen model versions and task families Probe success does not improve prediction beyond a small random end-to-end sample
Does explicit state reduce mid-session migration failures? Switch at matched checkpoints; compare transcript, summary, and structured-state handoffs with matched information Benefits vanish when information content or handoff cost is controlled
Can adaptation be made cheaper? Compare quality-versus-target-evaluation-cost curves against simple retuning Similar quality requires comparable or greater total cost
Does a universal harness beat a small adapter family? Compare a shared harness, lightweight adapters, and separately tuned systems Separate tuning retains a large quality advantage with manageable maintenance

These questions connect architecture to measurable outcomes. The objective need not be a single harness that wins everywhere. A practical result could be a stable task layer plus a small, well-tested set of adapters, with lower adaptation cost and fewer critical regressions than either direct replacement or full redevelopment.

The deepest shift is to treat model compatibility as a maintained empirical property. We already version code and schemas. An agent also needs evidence that a particular model, context policy, tool interface, and budget jointly satisfy its task contract.

A good architecture makes model changes easier to diagnose, cheaper to adapt to, and safer to reverse. Whether it achieves those benefits should be tested as directly as the agent’s task success.

Sources and figure data

Revised October 8, 2026. Figure 2 redraws reported data; Figures 1 and 4 are conceptual, and Figure 3 and the cost table are illustrative calculations. Data and sources.

  1. Chen, Zaharia, and Zou. How is ChatGPT’s behavior changing over time?. 2023 preprint; historical version comparison.
  2. Sclar et al. Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design. ICLR 2024.
  3. Yang et al. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. NeurIPS 2024; v3, interface analysis.
  4. Lin et al. Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses. 2026 preprint, v3; Section 4.3 and Figure 3.
  5. Qian et al. AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses. 2026 preprint, v1; Section 5.6.
  6. Anthropic. Quantifying infrastructure noise in agentic coding evals. Engineering report, February 2026.
  7. Anthropic. Scaling Managed Agents: Decoupling the brain from the hands. Engineering report, April 2026.
  8. Khattab et al. DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. ICLR 2024.
  9. Opsahl-Ong et al. Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs. EMNLP 2024; MIPRO.
  10. Xia et al. Agentless: Demystifying LLM-based Software Engineering Agents. 2024 manuscript, v2; three-stage baseline.
  11. Yao et al. τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. 2024 benchmark paper; repeated-run reliability.