Hellinger told how one of the trainers asked the group, "What is more important to you, your ideals or people? Which would you sacrifice for the other?"

The second-to-last layer is what Han settled on as a reasonable sweet-spot.

hyper-parameters, i.e., parameters external to the model, such as the learning rate, the batch size, the number of epochs.

BERT engineer is now a full time job. Qualifications include:Some bash scriptingDeep knowledge of pip (starting a new environment is the suckier version of practicing scales)Waiting for new HuggingFace models to be releasedWatching Yannic Kilcher’s new Transformer paper the day it comes outRepeating what Yannic said at your team reading group