Day 13. Machine Learning Basics and Human in the Loop

Training something yourself changes how you read every claim.

For twelve days you have used models other people trained. Today you train one, badly, in ten minutes, and that experience is worth more than any explanation of how machine learning works. Tool today: Teachable Machine.

The loop: collect examples, label them, train, test, look at the failures, collect better examples, retrain. Almost all of the quality lives in the data, not the training, which is the opposite of where beginners put their attention.

The written lesson. Read it slowly, it saves you later.

Why your model breaks

Insufficient variety, thirty photos taken in one room in one light teaches the model your room and your light, variety beats volume. Class imbalance, two hundred of one class and twenty of another, and the model learns to guess the common one. Leakage, the model learns a shortcut you did not intend, every photo of class A was taken on your desk, so it learned desk, not the object, invisible in the accuracy number, found only by attacking your own model. Testing on training data, evaluating on the examples you trained on measures memory, not learning. Hold examples back.

Accuracy is a hollow number without context

Ninety four percent is excellent for sorting photos into folders and negligent for anything medical, legal or financial. The right questions: what does a wrong answer cost, what kind of wrong is it, false positive or false negative, which are rarely equally expensive, and what confidence level should trigger a human review. Find, empirically, the confidence score at which your model starts being wrong. That number is your review threshold.

Human in the loop, the deliverable that outlives the course

Today’s Pro exercise is the most commercially valuable thing in the fifteen days, and it produces a document rather than a build. For every AI system you have made, state: the decision being automated, the cost of a wrong output in money, time or reputation, the checkpoint where a person reviews it and on what trigger, who that person is by role, and what evidence gets logged and for how long.

Enterprise clients, procurement teams and regulators ask for exactly this, and most agencies pitching AI in this region cannot produce it. One limit worth stating: if a human reviews every output, the system may not be paying for itself. Find the review rate that keeps risk acceptable without erasing the gain. That is a business calculation, not a technical one.

Two tracks. Pick yours, produce something.