AI Systems Engineering: Run, Operate & Evaluate / Operating & Evaluating

MLOps & LLMOps

Ship it, watch it, keep it working.

L3 · AdvancedEvolvingKnown~13 min

Reviewed by YuvarajPublished 24 Aug 2026

Getting a model to work once, on your laptop, is a demo. Keeping a model working, for months, under real traffic, as data drifts and requirements change, is an engineering practice. That practice is MLOps, and its adaptation to large language models is LLMOps. This lesson maps the lifecycle and shows what changes when the model is an LLM you did not train.

The lifecycle, end to end

A production ML system is a loop, not a line:

1DataCollect, clean, version. Bad data is the most common root cause of bad models, and the hardest to notice later
2ExperimentationTry approaches and track every run's data, code, hyperparameters, and metrics so results reproduce

Check your understanding

Answer from memory before revealing, retrieval practice is what builds durable recall.

1.
What is "data drift" in a production ML system?

Ask the tutor

Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.

Finished this lesson?

Mark it complete to earn XP, keep your streak, and schedule a review.

3Model registryA versioned store of trained models with lineage, so deployment is deliberate, not 'the file on someone's machine'

4DeploymentShip behind an API via the same CI/CD discipline as any software, plus model-specific checks

5MonitoringWatch it in production, unlike ordinary software, a model degrades silently while throwing no errors

6Loop backWhat you learn in production feeds the next iteration

Why 'silently' is the key word

A crashed web server pages someone. A model that has quietly become 10% less accurate serves confident, wrong answers and nothing alerts. Monitoring for ML is therefore about quality signals, not just uptime and errors.

Drift: the problem unique to ML

Traditional code does the same thing forever. A model's quality depends on the world matching its training data, and the world moves. Two failure modes:

Data drift, the inputs change distribution (new slang, new product categories, a different user population) while the task stays the same.
Concept drift, the relationship you learned changes (what counted as "fraud" last year does not this year).

You detect drift by monitoring input distributions and, where you can get them, outcomes, comparing live data against the training baseline and alerting when they diverge. The fix is usually retraining on fresh data, which is why the lifecycle is a loop.

What LLMOps changes

With a foundation model you often did not train, the emphasis shifts:

Prompts become artifacts. A prompt is now part of your system's behavior, so it needs versioning, testing, and rollback exactly like code. A prompt change is a deployment.
Evaluation is harder. There is frequently no single correct answer, so accuracy gives way to the evaluation techniques covered in the next lesson (LLM-as-judge, faithfulness, task-specific rubrics).
Cost and latency are first-class. Per-token pricing and generation latency are operational metrics you monitor and budget, not afterthoughts.
New quality risks. Hallucination and prompt injection (covered in the security lesson) are failure modes with no analogue in classical ML, and they need their own monitoring.
Tracing. For multi-step chains and agents, you log the whole trajectory, each prompt, retrieval, and tool call, because a bad final answer is usually caused by one step in the middle.

The mindset shift

Classical MLOps asks "is the model still accurate?" LLMOps also asks "is the system still faithful, safe, fast, and affordable?" You are operating a behavior, not just a classifier.

Classical MLOps

You trained the model on your data
Quality ≈ accuracy on a labelled set
Main risk: data and concept drift
Retrain on fresh data to fix

LLMOps (foundation model)

You often did not train the model
Prompts are versioned artifacts, a change is a deployment
Also watch faithfulness, safety, cost, latency
New risks: hallucination, prompt injection

Evaluation pipelines and gates

The thread tying this together is the evaluation pipeline: an automated suite that scores a candidate model or prompt against a held-out set before it ships, and re-runs on a schedule in production. Treat a regression on that suite the way you treat a failing unit test, a release blocker. Without it, "we improved the prompt" is a hope, not a claim.

Fast-moving layer

The specific platforms for tracking, registries, and observability change constantly, and vendors will tell you their tool is essential. The durable skills are the practices: version everything, monitor quality not just uptime, gate releases on evaluation, and close the loop with retraining.