Testing and evaluation of AI models becomes more difficult as they increase in capabilities. More intelligent models are more capable of deceiving tests, and thus may appear aligned while having serious problems that go undetected.
Looking through the Common Good Lens, this statement about AI models becoming more difficult to test or evaluate, as they can deceive tests and appear fine and aligned, is a little worrying, to say the least. Amodei has mentioned that AI is progressing fast, and with that, I feel it'd be good for everyone involved, including the community and the systems people use and share, if what he suggests here is accomplished faster than AI's progression.