I vibe-coded an optimization model and my colleagues found all the holes

There's a particular kind of confidence that comes right before embarrassment.

You know the one. It's the confidence of the person who watched one YouTube video about plumbing and is now absolutely certain they can replace the pipes under their sink. It's the confidence of someone who bought a guitar and is now absolutely certain they can play Dragonforce. It's the confidence, as it turns out, of someone who used AI to build an optimization model and was very pleased with himself.

That someone was me.

# A brief trip back to the beginning

In one of my very first newsletters, I argued that LLMs can't solve scheduling problems, that's the domain of classical constraint solvers and metaheuristics, the "old school" AI that's been quietly doing serious work for decades. I did add that the future probably lives somewhere in the mix between old-school and new-school AI.

I still stand by that. The prediction aged well, if I say so myself.

What I didn't fully anticipate was how that mix would play out in practice. Specifically, what happens when someone (it’s me 🙂) enthusiastically dives into the GenAI side of that equation, in a domain where their expertise is... let's say, still constantly developing.

# YOLO mode, activated

The experiment started simply enough. Full vibe-coding, no safety net. Just me, an AI, and some guidance files I'd thrown together. No carefully structured prompt engineering session. No domain expert on call.

And honestly? I was impressed. Genuinely. It produced something that looked coherent, spoke the right language, had variables and constraints in all the right places. So I started bragging about it on our internal Slack channel. Very proud of what I’d done, ready to tackle any optimization problem myself!

It took all of 5 minutes until the first feedback knocked that idea out of me. While I thought the model was fine, my expert colleagues found some glaring issues with it.

Now, here's the uncomfortable part: I didn't know what was missing. Not because the AI hid it. Because I didn't have enough experience in this particular area to know what to look for in the first place. The model hadn't failed visibly. It hadn't crashed. It had confidently produced something plausible and I, equally confidently, had accepted it.

# Dunning-Kruger era of AI-assisted development

There's a well-documented cognitive trap (Dunning-Kruger effect) where a little knowledge creates disproportionate confidence. The less you know about a field, the harder it is to recognize what you don't know. Expertise, paradoxically, tends to make people more aware of what they're missing.

AI coding tools are, inadvertently, optimized for exactly this trap. They are extraordinarily good at making things look right, they speak the language, and produce syntactically correct, logically plausible output. In optimization, this is especially treacherous. A model with a subtle error won't necessarily blow up. It'll solve. It'll give you a number. Maybe even a reasonable-looking number. The error only becomes visible when someone who knows the domain asks: "Wait, why didn't you model this constraint?"

Now, could better inputs have helped? Probably yes. Skills files, richer context, more structured guidance could close some of that gap. But that's precisely the trap: we assume the shortcoming is in the prompt or the context, when sometimes it's the prompter.

# What I actually learned

I'm not writing this to argue against using AI for development. Heck, I use it constantly and plan to keep doing so.

But the experience recalibrated something important. The real question isn't "can I build this with AI?", the answer is almost always yes, quickly and impressively. The question is: "do I understand this domain well enough to know if what I built is actually correct?"

In the first newsletter I wrote, I said the future is probably in the mix between old-school and new-school AI. I still believe that. But I'd add a third ingredient to that mix now: old-school human judgment. The kind that knows what it doesn't know.

Because the most dangerous moment isn't when the AI gets something wrong. It's when you don't realize it has.

So the next time you're feeling very good about something you built, like a nice optimization model with Timefold for example ;), find someone who knows the domain and ask them to look at it.

They might confirm you nailed it. Or they might make you realize that you still have much to learn. I speak from experience. 🙂