Loading lesson…
AI Coding / Working With Coding AI
Trusting output without rubber-stamping it.
Reviewed by Yuvaraj
When an AI assistant writes code, the words on the diff are the model's, but the responsibility for what ships is entirely yours. A model has no accountability, no memory of production incidents, and no stake in the on-call rotation. You do. That single fact reframes the whole activity: your job is not to admire fluent code but to interrogate it. As assistants get faster at producing plausible code, the scarce, valuable work moves from writing to reviewing, and a team that speeds up generation without strengthening review has simply learned to merge bugs faster.
A language model is trained to produce the most probable continuation of your prompt, not the most correct program. Probability rewards surface plausibility: idiomatic names, familiar structure, confident comments, tidy formatting. Correctness is only weakly correlated with those signals. This decoupling is the root of every failure mode below, the code looks like code a competent engineer would write, which is exactly what disarms a hurried reviewer.
Because the model interpolates from patterns it has seen, it also invents details that fit the pattern but not reality: methods that a library plausibly could have but does not, arguments in the wrong order, defaults copied from an older major version. And when asked to "add tests," it frequently observes what the code currently does and encodes that as the expectation, so the tests pass while asserting the bug.
| Failure mode | How to catch it |
|---|---|
| Plausible-but-wrong logic | Trace the algorithm by hand on a real input; state the intended behavior before reading the code |
| Hallucinated / misused APIs | Check every unfamiliar call against current official docs; run it |
| Missing edge cases | Probe empty, zero, negative, duplicate, and boundary inputs |
| Missing error handling | Ask what happens on failure of each I/O, parse, or network call |
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
| Security issues | Look for injection, hardcoded secrets, unsafe defaults, unvalidated input |
| Tests mirroring the bug | Read tests against the spec, not the code; do the assertions match intent? |
| Outdated patterns | Verify APIs, versions, and idioms are current, not deprecated |
This function and its test came from an assistant. Both look reasonable.
def median(values):
values.sort()
n = len(values)
return values[n // 2]
def test_median():
assert median([3, 1, 2]) == 2 # passes
assert median([4, 1, 3, 2]) == 3 # passes, but wrong
The suite is green, so a rushed reviewer approves. Three defects are hiding here. First, the logic is wrong for even-length input: the median of [1, 2, 3, 4] is 2.5, but values[n // 2] returns values[2], which is 3. Second, the test asserts that wrong answer (== 3), it locked in the buggy output instead of the intended definition. Third, values.sort() mutates the caller's list in place (a silent side effect), and an empty input raises IndexError rather than being handled.
You catch all three by reviewing against intent, not output: state the definition of a median first, hand-trace the even case, then notice the test agrees with the code instead of the specification.
The bottleneck moved
Faster generation doesn't reduce total effort, it relocates it. Budget review time as first-class work, and measure your team on defects that reach production, not on lines merged per day.
Common mistakes
Trusting green tests written by the same assistant that wrote the code. Skimming a large diff because it "looks clean." Approving an unfamiliar API call without checking the docs. Letting confident comments substitute for verification. Rubber-stamping code you could not have written and cannot explain, if it breaks at 3 a.m., "the AI wrote it" is not an answer.