"Human in the loop" is necessary, but the current process itself makes the loop stultifying, and encourages the human to take themselves out of the loop. That process is straight up dangerous. The temptation to let it rip is always there, and I didn't even have a boss pressuring me to ship code.

It was so tempting to press 2: "Yes, and accept all changes for this session." Why wouldn't you? If you're accepting them all manually, what's the harm? What's the harm? harm harm harm harm Yeah, that's how you get got in this process. Once you stop scrutinizing the model's output, the probability something goes off the rails approaches 1.