Programming agents might work, but there are a lot of ways they can go wrong. The amount of guardrails necessary to keep the model in check doesn't scale. The more steps of a process rely on generative output, the higher the potential for error and catastrophic failure. Any process that involves these models must strive to maximize determinism and minimize model variance.

There’s an idea in the science-fiction community called steam-engine time, which is what people call it when suddenly twenty or thirty different writers produce stories about the same idea. It’s called steam-engine time ­because nobody knows why the steam engine happened when it did. Ptolemy demonstrated the mechanics of the steam engine, and there was nothing technically stopping the Romans from building big steam engines. They had little toy steam engines, and they had enough metalworking skill to build big steam tractors. It just never occurred to them to do it.

People aren’t encouraged to think about little further changes they might want to make, or naturally make tweaks as they go; instead, they just adapt their behavior to whatever the app encourages