Self-Improvement Claims

"Self-improving AI" is one of the most overloaded phrases in the field. In a single sentence it can mean a coding agent that retries until its tests pass, a training pipeline that uses one model's outputs to teach the next, or a machine that autonomously rewrites itself into something categorically smarter overnight. The first two ship in production today. The third has never been demonstrated. As a working developer, your job when you hear the phrase is to figure out which one is on the table, because the engineering, the risks, and the plausibility are completely different.

Two labels, kept separate

KNOWN / EMERGING. These are real, deployed, and bounded:

• Within-task self-refinement. An agent generates output, critiques it against feedback, and revises, reflection loops, tool-use loops, self-critique. The Self-Refine method formalized this: the same frozen model, iterative feedback, measurable gains on the target task.
• Automated prompt and pipeline optimization. Systems that search over prompts, few-shot examples, or pipeline structure against a scored objective (frameworks like DSPy do this).
• Training-time bootstrapping. One model's outputs help train a later model: knowledge distillation, and preference methods like RLAIF where model-generated critiques provide the training signal.

Every one of these is human-orchestrated and bounded, a person defines the metric, the data, the stopping rule, and the compute budget. The improvement is real, but it plateaus.

SPECULATIVE. Open-ended recursive self-improvement: a system that autonomously and repeatedly rewrites its own weights or code into unboundedly greater general capability. There is no empirical demonstration of this. It runs headfirst into hard limits: you cannot optimize past what you can measure, so evaluation caps gains; optimizers exploit flawed metrics (reward hacking); useful new training data is finite; and each capability step tends to cost more compute, not less. For truly unbounded growth you would need capability to satisfy $C_{n+1}$ every cycle with no ceiling. Nothing we can build today sustains that.