NOTE: Article in Progress
Language models1
Understanding the basic functioning of large language models requires an understanding of the following concepts:
- Word vectors
- Transformers
- Feed-forward neural networks
- Attention
Large Language Model (LLM)
Limitations
Word vectors and large language models may perpetuate biases. As they say, 'Garbage in, garbage out.' These models are trained using data generated by humans; hence, the biases of the individuals and society in general may persist in these models, especially through the text generated using these models.
References
- Bender, Emily M., et al. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, 2021, pp. 610–23. (PDF)