For example, a model trained on images in which every man is wearing glasses might be convinced that wearing glasses is a strong signal that someone is a man. While this example might seem obvious, it can be difficult to know in advance if training data consists of poor examples for the task, or if it is unrepresentative.
This example made me think about how easily technology can learn something incorrect if the information used to train it is limited. Wearing glasses has nothing to do with being a man or a woman, but the system might make that assumption. I believe this shows why we should not always trust the answers provided by artificial intelligence. It also makes me wonder how many other incorrect assumptions these systems might make without us noticing. In my opinion, the people who develop these technologies have a responsibility to use diverse information and carefully check the results.