Self-Improving Agents · leejk

TL;DR. Most of what gets called self-improvement is not weight-changing learning but accumulated experience for one environment, and the gains are measured almost entirely within that same environment. Reading the primary sources directly: the "improvement" in EvoTest and SAGE is a same-task measurement, and test-time scaling has a ceiling (give it more turns and past a point it drops). What survives across task boundaries is only the approach that solidifies experience into a reusable skill artifact. Even that transfer crosses models and execution harnesses well but thins along the task axis. The live question is not "does self-improvement work" but which axis you can expect it to transfer along.

One Word Covering Too Much — There's No Consensus

In 2026 "self-improving agent" became a trend word. May alone brought a run of skill-transfer benchmarks — SkillsBench (2602.12670), SkillFlow (2604.17308) — and at the end of the month the RLEval workshop (ACM CAIS, 2026-05-26–29) put how to build RL environments as agent evaluation criteria on the table for the first time. New bragging numbers land every week, yet what the number measures never makes the headline.

The trouble is that "self-improving" names too many mechanisms at once. What changes — weights (RL fine-tuning), persistent memory (Reflexion's verbal self-reflection, Shinn et al. 2023), a skill library (accumulated reusable procedures), or just this session's prompt and context? All four are sold under the same sentence, "the agent gets better by itself," but they differ in what survives after the session ends and in what holds when you move to a new task. So which kind of self-improvement actually leaves something behind is not visible in the headline; you have to read what the paper held out. That is the work of this piece.

How I Dug In

I read and cross-checked three clusters of primary sources directly. ① Paradigm definitions — Reflexion (verbal self-reflection + persistent memory), the definition of test-time learning. ② Measurement of same-environment improvement — the result tables of EvoTest (arXiv 2510.13220, the J-TTL benchmark) and SAGE (2512.17102, Sequential Rollout). ③ Measurement of cross-task transfer — EvoSkill's (2603.02766) zero-shot transfer section, SkillOpt's (2605.23904, Microsoft·SJTU et al.) three-axis transfer table, and a CMU study on the ceiling of test-time scaling. I classified each result on three axes — what changes (weights / memory / skill / prompt), where it's trapped (within-task vs across-task), and whether the measurement is same-environment or held-out.

Two honesty notes. The benchmark numbers I cite are reported figures from 2026 preprints, not values I reproduced. And during this investigation I found a secondary summary that wrongly stated EvoSkill ran "no transfer experiments" — only by pulling the actual PDF and reading its §3.3 "Zero-Shot Skill Transfer" table did I confirm the opposite. In a field where numbers move this fast, that's a live example of why you read the primary source directly.

Most "Improvement" Is Measured Inside the Same Environment

The most common self-improvement number is a curve that gets better while repeating the same task. EvoTest, on the J-TTL benchmark, has an agent play the same game over several consecutive episodes, lifting performance round over round — even being the only system to clear Detective and Library is improvement within those games. SAGE's Sequential Rollout walks a chain of similar tasks, accumulating earlier tasks' skills into a library, and on AppWorld lifts Scenario Goal Completion by 8.9 points while cutting interaction steps 26% and tokens 59% — all numbers inside the same benchmark.

Both are good results; neither is held out. A curve sloping up and a claim of generalization are two different claims. Getting better in the same environment may just mean fitting that environment more tightly — how much of it survives in a new environment has to be measured separately, and most self-improvement headlines skip that measurement.

Test-Time Scaling Has a Ceiling

The intuition that "even in the same environment, more attempts keep helping" also breaks. CMU's Benchmark Test-Time Scaling of General LLM Agents runs 10 leading agents across 4 domains (coding, search, tool-use, reasoning), looking at sequential scaling (turns) and parallel scaling (samples) together. The result is not monotone — performance peaks around 3–7 turns, then yields diminishing returns, and once context runs well past the model's inherent limit the accumulated history interferes with later reasoning (so-called context pollution) and drops below the earlier baseline. A roughly 30% drop is reported when moving across domains.

(This ceiling figure is currently sourced through a secondary write-up, so the absolute values should be read as direction until the primary paper is checked.) The direction alone is enough: even within the same environment, improvement is not unbounded. The place where "improvement" was measured is narrow, and inside it there's a ceiling, not a floor.

What Survives Across Tasks Is Only What's Solidified Into a Skill

Beyond the ceiling — the gain that survives moving to a new task — where does it come from? The two primary sources that explicitly measure held-out transfer point at the same answer: the side that solidifies experience into a self-contained skill artifact (a folder with trigger conditions plus procedure).

EvoSkill (Sentient·Virginia Tech, 2603.02766) — takes the search-persistence-protocol skill evolved on SealQA and applies it with no modification to BrowseComp. Different benchmark, different questions, different difficulty — and accuracy rises 43.5% → 48.8% (+5.3 points) on 128 stratified examples. The paper's claim: "skills discovered by EvoSkill are not overfit to their training task."
SkillOpt (Microsoft·SJTU et al., 2605.23904) — nails the same principle into a three-axis transfer table (Table 4). Cross-model: a SpreadsheetBench skill from GPT-5.4 lifts mini by +9.4, nano by +3.0. Cross-harness (Codex ↔ Claude Code, task fixed): SpreadsheetBench goes Codex→Claude Code +59.7, the reverse +43.6 (this large figure is concentrated on a specific benchmark/harness pairing, so reading it as the size of general transfer overstates it). Cross-benchmark: OlympiadBench→Omni-MATH +1.3 to +3.7. Every row is positive — no transfer falls below the target's no-skill baseline.

Why does only this side hold? As EvoSkill states (4.3), a skill is a self-contained folder of trigger conditions and procedure, decoupled from both model and task. Optimized prompts often collapse when the model or task shifts; structured skills do not. It's the same structural conclusion — when improvement stays in-context inside the model, it evaporates with the session; when it drops into a reusable artifact outside the model, it transfers. (The same arrangement as in the prompt-injection investigation, where the ceiling disappeared only when the guarantee moved out to a deterministic layer outside the model — capability kept inside the model is probabilistic and volatile; what's solidified outside it transfers and holds.)

The Live Question Is "Along Which Axis Does It Transfer"

If that's the consensus-ready conclusion — solidify into a skill and transfer is positive — the frontier's live question is what comes next. Transfer is not uniform. Re-read the numbers above by axis:

Crossing models (cross-model): +3.0 to +9.4 — small but robust.
Crossing harnesses (cross-harness, task fixed): +1.6 to +59.7 — large but with heavy per-benchmark variance.
Crossing tasks/benchmarks (cross-benchmark): +1.3 to +5.3 — the thinnest.

The pattern is assertive — a skill crosses models and harnesses well, but the gain thins the more it crosses task boundaries. So two questions stay open. ① Where is the ceiling of task-axis transfer — whether the +1–4 points cross-benchmark is signal distinguishable from noise, and on which task pairs it bends to zero or negative, no one has yet mapped. ② There is no evaluation standard — this is exactly what RLEval just raised. Without consensus on whether a same-environment curve counts as self-improvement or only held-out transfer does, the competing "we're SOTA" claims can't even be compared (SkillOpt reports +3.2 to +14.0 over EvoSkill, but the setups differ).

Addendum (2026-06-09): A Week Later, Four More of the Same Pattern

A week after this went up, a single day's paper feed (2026-06-09) carried four more papers tagged "self-evolving/self-improving" at once — one more piece of first-hand evidence that the opening line ("a new number every week") wasn't hyperbole. Sorted along the same three axes (what changes · where it's trapped · is the measurement held-out), the four don't produce a new conclusion so much as refill both ends of the existing one.

OpenSkill (Lehigh·UIC·Salesforce et al., 2606.06741) — with no target-task answers (open-world), it solidifies skills from docs and repos into markdown artifacts. It leaves weights untouched, and transferring those skills unmodified to weaker models lifts them +5.5–14.8 pts (Opus 4.6 → Haiku·Qwen·DeepSeek·Mistral) — exactly the EvoSkill·SkillOpt camp. It restamps the body's cross-model conclusion (+3.0–9.4) at a larger magnitude. But — it doesn't measure cross-task·cross-benchmark transfer. The thinnest axis is empty here too.
SIA (2605.27276) — updates harness and weights together (a feedback-agent updates both). LawBench +56.6 pts, GPU-kernel runtime −91.9% — the numbers are big, but all are held-out-split measurements inside the same task/environment, with no cross-task transfer. It's a textbook case of what the body said about SAGE: a rising curve and generalization are different claims. Big numbers + same-env.
Socratic-SWE (2606.07412) — distills failure/repair patterns from execution traces into structured skills used for retraining. 50.40% on SWE-bench Verified after three iterations, reported across Verified·Lite·Pro·Terminal-Bench — nominally cross-benchmark, but all four are variants of one task family (GitHub issue resolution), a different thing from EvoSkill's heterogeneous-task SealQA→BrowseComp transfer. Another face of the body's measurement-standard problem: "crossed a benchmark" is not "crossed a task."
HarnessForge (Beihang·Tsinghua, 2606.01779) — co-evolves harness and policy (LoRA) and measures across five held-out benchmarks with strict train-test splits — the cleanest protocol of the four. Yet the average gain is +3.56% (max 12.0%), the same thin band as the body's cross-benchmark range (+1.3–5.3). And it self-reports "pair-specific compatibility" — a harness-policy pair evolved together substantially beats mismatched combinations. That is, this LoRA is coupled to the harness. The body said "only decoupled skills transfer"; HarnessForge deliberately couples to buy same-env fit and pays for it in thin task-axis gains — not a counterexample but the underside of the same law.

Taken together, none of the four closes the question the body opened — where is the ceiling of task-axis transfer. They split into cross-model (OpenSkill), same-env jackpot (SIA), within-task-family movement (Socratic-SWE), and coupled thin held-out (HarnessForge); not one shows thick transfer across heterogeneous tasks. That the blank persists after four more papers in a single week suggests it's not an accidental omission but the actual boundary of current methods.

(provenance: these four were checked at the level of the HuggingFace paper pages' abstracts and result tables, not the PDF-section depth applied to EvoSkill·SkillOpt in the body. Numbers and transfer labels are authors' reported figures, pre-reproduction and pre-cross-check. The "same task family" and "coupling" judgments in particular lean on the abstracts' wording as structural reading — pulling the actual tables could overturn them.)

What This Investigation Can't Say

Every number cited is a reported figure from a 2026 preprint, not one I reproduced. In particular, a large cross-harness value like +59.7 is concentrated on SpreadsheetBench and a specific harness pairing, so reading it as the size of general transfer overstates it. The CMU ceiling figures come through a secondary source, so the absolutes should be read as direction until the primary paper is checked. And this piece compared weight-changing RL self-improvement (SAGE-style) and weight-frozen skill/memory accumulation in one frame, though the two are different mechanisms for which "transfer" means subtly different things — splitting that further is outside this investigation's scope.

Next Investigation

Run a fixed skill artifact on a fixed model across many task pairs to plot the task-axis transfer curve — where the gain hits zero and where it bends negative. And actually lay an RLEval-style held-out protocol over a single benchmark to measure "in-task improvement" and "transfer" separately. Those two are the next questions this finding opens.

Primary Sources

Addendum (2026-06-09) — abstract / result-table depth, PDF body not pulled: