AI can solve the problems now. Is a mistake notebook still worth keeping?

Over the past couple of years Elon Musk has kept handing out a timeline for AI: first it gets smarter than any single person, then, a few years later, around 2030, smarter than all of humanity combined.

Put the way people around me would put it, that becomes a sharper question: if AI is this good at solving problems, are top students still worth anything?

I have no idea whether the timeline holds. That's Musk's to deliver. But the half about top students losing value, I partly buy. I write code with AI every day, and I'm watching one thing get cheaper right in front of me.

What's getting cheaper is "getting it right the first time."

Here's how I use it. After work I tinker with a few small projects of my own: a personal knowledge base and some automation scripts that run on a daily schedule. These days I describe what I want, an AI assistant writes most of the code, and I review it. Its first version usually just runs.

That used to take real effort. It's also what top students compete on at school: the problem is fixed, all the conditions are given, and you write the textbook answer in one go. That is exactly the skill AI picks up fastest, because it's the easiest to set questions for, the easiest to grade, and the easiest to collect practice data for.

But in real work, the nastiest trap is "it runs, but it's wrong," with nothing throwing an error.

Here's one I fell into. On August 3 this year, a script was bulk-writing records into my knowledge base and left out one field in the API call. All 69 records went in. Every single response came back 200, which means "success." In reality all 69 landed in a state I didn't want, and I only found out when I went back to check.

Missing a field was just a slip. The real mistake that time was treating "the call returned success" as "the job got done."

AI can't remember traps like that on its own.

The reason is concrete: the AI assistant I use starts from zero in every new conversation. Where it fell last time, it has no idea. So in the config file it reads every time it starts work, I keep a section called "Mistakes Already Made." The rule: entries get added, never deleted.

Each entry has four things: date, symptom, root cause, and the rule going forward. The rule from the 69-record incident: for anything that writes data, acceptance means you can actually find the result, not just that you got a success code.

That section now has 13 entries. The earliest is from May 17, the latest from August 10.

The cost of this notebook isn't in writing it. It's that every entry exists because I lost something first. One is from when the AI assistant deleted data I had created myself, assuming it was test data, and I only got it back from a backup. Another is from a data migration that went wrong, where patching it in place made things messier and messier until I wiped everything and redid it. AI can help me word the rules neatly. It can't pay that tuition for me.

What a top student really has of value was never just the grades. It's the notebook of wrong answers.

AI can score the grades too. A mistake notebook can only be built one wrong answer at a time, and only out of your own work.

Which brings me to this morning.

October 12, Monday, a little after six. I have a small script that, before I start writing, compares a new topic against what I've written before and warns me if it's too similar. Today it said: similarity 0, good to write.

Similarity 0 felt a little too clean. I scrolled down its output and found one line at the very bottom: embeddings HTTP 404.

Meaning the comparison service it depends on was gone. It hadn't compared anything, and then told me "good to write."

That trap is in the notebook. There's an entry from August 3 that's almost word for word the same: a dedup check whose lookup endpoint returned 404, so it always concluded "safe to write new," and the same batch of data got written again every week without a peep. The rule I set back then: when something breaks, the fallback path must not also be the most permissive conclusion.

So how did I step in it again? I looked at the script. It was written at the end of July, about a week before that rule. The lessons in the notebook kept everything written afterwards in check; nobody ever went back through the older stuff with the notebook in hand. How long it's been running empty like this, I still haven't gone back to find out.

I keep the notebook. I wrote the rules. And the one who walked straight into the hole this morning was me too.