leejk

leejk/ jk lee

If you're reading this blog, you're already at the front line of the AI trend.

This site is built AI-native. The fastest way to use it: copy any post (bottom-right) → summarize with your LLM.

Featured landscape · 2026.06

Models: lining up the frontier base models

Read full landscape →

01section

Landscapes

2026.06·overview

Models: lining up the frontier base models

Coding, autonomy, hallucination, science reasoning — which model does what well (a 2026-06 ranking plus a 2026-08 generation update)

The frontier base models that coding agents, harnesses, and skills sit on top of — Claude Fable 5, GPT-5.5, Gemini 3.1 Pro — lined up by strengths and weaknesses. The three have converged into the same top band, and the leader differs by domain: coding, long-horizon autonomy, and low hallucination go to Fable 5, science reasoning (GPQA) to Gemini 3.1 Pro. Claude Fable 5's own strengths and weaknesses get a deeper standalone piece.

Read the full overview →

02section

Research

2026.06

When to Fan Out to Subagents

One property — an isolated context — is every benefit and every failure mode; the criterion is not a checklist but a single question

Digging into when to use subagents across three sources — official docs, empirical literature, local skills — they converge on one point: a subagent is defined by a single property — it works in an isolated, fresh context and returns only a summary — and every benefit (parallelism, context protection) and every failure mode (coordination collapse, write-inconsistency) is the two faces of that one property. So the criterion collapses to a single question: does this task want its context isolated, or shared? Reading/gathering/independent search wants isolation, so fan out; writing/synthesis/coordination wants sharing, so keep it in one agent. The cost gate (N×, 15×) and "upgrade the model before adding agents" sit on top. Not a controlled experiment — a synthesis plus a reflexive observation (this very research was itself a read-fan-out).

#research#subagent#multi-agent#orchestration

2026.06

The Collapse of Agent Evaluation

Benchmarks saturate and lose discriminative power; LLM judges get gamed and lose trust — and the escape hatch from both leans on the same judge

Every "+X%" of agent progress is denominated in a benchmark score or an LLM judge's verdict — and that ruler is failing on both sides: benchmarks saturate and lose discriminative power, while judges get gamed or simply waver. From 10 primary sources plus a hands-on reproduction: one-shot surface gaming backfires on a strong judge; the real risks are self-praise bias, scoring instability, and the iterative optimization loop. Measurement resists gaming only when bound to an anchor outside the LLM (execution, compilers, corpus statistics).

#research#evaluation#llm-as-judge#benchmark

2026.06

Memory Corruption

Keep updating and good memory falls below the no-memory baseline — from directly modeling consolidation and invalidation

Agent-memory research optimizes retrieval, but reading the primary sources directly, the corruption enters on the write path (consolidation/update). Three independent works converge: continuous updates push memory utility below the no-memory baseline (Useful Memories), agents fail to retract invalidated beliefs (STALE), and write-stage errors surface much later at read (MemTrace). Modeling consolidation collapse and the stale cascade in deterministic code reproduces the same shape. The fix is not to improve reading but to gate writing.

#research#memory-validity#agent-memory#llm-memory

2026.06

Self-Improving Agents

Most of the "learning" is measured inside the same environment, and transfer thins along the task axis even when it crosses models and harnesses

Most of what gets called self-improvement is not weight-changing learning but accumulated experience for one environment, and the gains are measured almost entirely within that same environment. Reading the primary sources directly: EvoTest and SAGE measure same-task improvement, and test-time scaling has a ceiling. What survives across task boundaries is only the approach that solidifies experience into a reusable skill artifact (EvoSkill, SkillOpt). Even that transfer crosses models and harnesses well but thins to +1–9 points once the task or benchmark changes. The live question is not whether self-improvement works but which axis you can expect it to transfer along.

#research#self-improving-agents#test-time-learning#agent-skill

2026.06

Prompt Injection

The ceiling on probabilistic guards, and why the guarantee moves outside the model

Defenses that make the model itself ignore injections have a ceiling. Reading the primary sources directly: on AgentDojo the best defense only cut attack success to 6.84%, and a guard LLM gets bypassed up to 100% by the same injection. Only one approach holds: take the model out of the trust path and let a deterministic layer outside the model constrain what the model output is allowed to do (CaMeL, dual-LLM, LACUNA, wire guards). The live question is no longer whether you can stop it, but where to place that deterministic boundary.

#research#prompt-injection#agent-security#mcp

03section

Notes

2026.06

A tool needs either a clear name or a clear description — one of the two

Routing that drifts as tools pile up, measured directly, with the fix

The claim that adding tools breaks routing is usually pinned on vague descriptions. Standing up subagents as the router and measuring it, the author found the folklore is only half right — a strong model routes correctly as long as either the name or the description is semantically distinct. The real collapse is when neither disambiguates, and then it does not mis-route by intent but falls back to list position. A destructive tool listed first gets picked even for a read-only request. The fix: make either the name or the description unambiguous, and cut the number of tools so collisions can not form.

#notes#tool-use#mcp#agent-skill

2026.05

Asking "Is This Right?" Is a Waste of Breath

How to move verification to the front

After getting an answer, asking "Is this right?" almost always lands on "yes." Sycophancy and the model's defense of its own output overlap there — the prompt isn't the problem, the order is. Doubt belongs before the answer, and when that misfires, in a separate context. The cheapest way to cut sycophancy short of installing a judge.

#notes#sycophancy#prompting#verification

2026.05

How to Save Even a Handful of Tokens

Six leaks you can plug in five minutes

Token usage has become a productivity signal, but most of the waste is small leaks you can plug in five minutes — once you stop averaging and look at the distribution. Six levers measured on the author's own workflow: trimming the cold-start skill stack, disabling unused MCP servers, keeping cache prefixes stable inside the 5-minute TTL, sending narrower contexts to subagents, defaulting to Sonnet, and using Plan mode + AskUserQuestion to collapse round-trips. Plus the cost each lever pays back.

#notes#token-cost#claude-code#workflow

2026.05

You Need Opposition for Opposition's Sake

How to turn an AI from a flatterer into a loyal critic

An AI assistant defaults to the flatterer, not the loyal critic — preference optimization pushes models toward agreement. So you must engineer opposition by design, but forced opposition over-fires, objecting where no objection is due. Lining up the available subagent-debate skills along that axis points to one conclusion: spinning up disagreement is now cheap; the load-bearing part is the judge.

#notes#multi-agent#subagent#debate

04section

Opinion

2026.06

What to Delegate

If you don't ask whether it's the bottleneck first, you just delegate the wrong work faster

As the cost of execution collapses, anything can be delegated — so skill has moved from "what you delegate" to "what you keep." Do that sorting by intuition and the same technology produces opposite outcomes. Delegability filters in two stages — first diagnose the bottleneck, then filter the bottlenecks by fit. Fit is set by an asymmetry: one benefit (frequency × standardization) against three costs (verification, irreversibility, context). So delegation isn't "is the benefit large" but "is the risk controlled."

#opinion#ai-agents#automation#delegation

2026.06

The Smaller the Team, the More You Need a Spec

Working alone, there's no one to filter the code the AI wrote

I reject the back half of the hybrid consensus — "SDD for big teams and regulated industries, vibe for solos and small projects." Spec-first is not a headcount problem; it is a cost-of-reversal problem. The solo developer is not exempt but the most exposed — least able to tell throwaway code from survivors, and with no team to absorb the ones that survive. The axis is survival, not scale.

#opinion#vibe-coding#spec-driven-development#technical-debt

2026.05

MCP Skepticism, and Why You Still Can't Turn It Off

What burns your tokens isn't MCP — it's how the definitions are loaded

Quandri's "MCP is dead" is the right diagnosis — one tool definition eats 10% of context, and a Linear lookup costs 65x more than the CLI. But the target of the skepticism isn't MCP; it's the design that preloads every definition. That design is already breaking — tool search and code execution cut tokens by 98%. The reason you still can't turn MCP off is separate: the skeptics' prescription (CLI-first) assumes a shell-reachable surface already exists, and a stateful editor like Unity has no such surface. That's why I can't kill Unity MCP.

#opinion#mcp#context-window#progressive-disclosure

2026.05

Where Are We Going

What survives a developer when the measurement collapses

The vibe coding era is real — capital proves it. But the productivity it promised became unmeasurable before it was proven: in a year we went from "AI makes experienced devs 19% slower (they felt 20% faster)" to "we can no longer measure it, because experienced devs refuse to work without AI." What survives the collapse of measurement is judgment — and its erosion, like the +20% illusion, is invisible from the inside. So judgment has to be trained on purpose, as ritual. Though judgment may not be the human's permanent seat either.

#opinion#vibe-coding#ai-adoption#developer-productivity

03section

Projects · Experiences · Profile — expand to read.

If you're reading this blog, you're already at the front line of the AI trend.

Models: lining up the frontier base models

Landscapes

Models: lining up the frontier base models

Qwen3.8-Max

Kimi K3

Grok 4.6

GLM 5.2

GPT-5.5

Gemini 3.1 Pro

Claude Fable 5

Research

When to Fan Out to Subagents

The Collapse of Agent Evaluation

Memory Corruption

Self-Improving Agents

Prompt Injection

Notes

A tool needs either a clear name or a clear description — one of the two

Asking "Is This Right?" Is a Waste of Breath

How to Save Even a Handful of Tokens

You Need Opposition for Opposition's Sake

Opinion

What to Delegate

The Smaller the Team, the More You Need a Spec

MCP Skepticism, and Why You Still Can't Turn It Off

Where Are We Going

About me