"All reported that GPT-2’s outputs were difficult to control
and often strayed off topic or into nonsense. They had to
keep giving GPT-2 “a shove in the right direction,” and then wrestle its outputs back into their contexts"
I find this really interesting. With a lot of the new AI videos, you can see a similar trend. They usually are very good in short form content but tend to lose detail and context of what the story is actually about, like how GPT-2 couldn't write for very long before it lost its way.