Anyway, what I wanted to talk now is not that much about technical details, but rather about some problems that arise from the deployment of huge language models by big tech companies.
You see, language models are often trained in an unsupervised (or self-supervised) form, fed with massive chunks of text mined from the Internet. This is a very cool idea in principle, because we have access to a vast collection of human language where basically everything we know about can be found. GPT-3 and BERT are just two examples of very different language models trained of huge amounts of text (they are in completely different leagues, though, in terms of training data).
So, if you ask one of these language model the probability of a sentence like “Leonardo da Vinci painted the Mona Lisa” it should give it a very high score. However, if you ask it “Alejandro Piad painted the Mona Lisa”, the score should be close to zero. The reason is very simple, there are far more examples of the first sentence than the second in the Internet. The model doesn’t really know who painted the Mona Lisa, it just knows that many more people think it was Leonardo (keep in mind, though, that both you and me also reason like this a lot of times…)
Now, if only the Internet was a place where all that’s true was massively more common that what’s false. But it isn’t. It is full with conspiracy theories and fake news. So we must be careful in using frequency of something appearing in the Internet as a proxy for truthfulness.
The big problem, though, comes not from purposefully misleading stuff, but from the subtle biases that creep into all of our conversations. For example, what happens when you ask a language model “He is a programmer” vs “She is a programmer”? Naturally, both sentences should be exactly equal in terms of likelihood. But a carelessly trained LM will very likely give a higher score to the first one. Why? Because the Internet has many more examples of programmer boys than girls!
Why does this matter? In some applications this kind of biases pop up immediately. For example, if you Google translate a long paragraph including “she is a programmer” back and forth between English and a language without gender you can get “he is a programmer” back. But these are not the worst cases. You can use a language model with these biases as an internal component of another system, say, to evaluate candidates for job applications, or to assess the reliability of a legal claim, or to estimate if a person will forfeit a mortage, or to pre-screen papers submitted to a research journal. In these cases, you may have no idea how these biases are messing with the final prediction. As a very simplistic example, you could be rejecting women applying for programming jobs more often than men because their profile has less “fit” with the job description.
So here comes the mandatory discussion about “but that’s the real data!”. Yes, it is. And that doesn’t make it right. Reality is full of biases, full of wrong decisions, full of things we want to change. Letting those things creep into our models of reality unnoticed is a recipe for keeping ourselves in the place we are today, not in the place we want to be.
Now, there’s light at the end of the tunnel. Ethics and fairness is a big issue in the AI research community today. The most brilliant minds in our field our working in the detection and mitigation of these problems. The solution is not to vilify and stop using these technologies altogether. Language models are a very powerful tech that can boost some of the most interesting and useful applications of the next decade. The solution is to understand their limitations and deploy them with the necessary care in those scenarios where they’re most likely to cause harm.