Chip Huyen Blog
2023-08-16 00:00 UTC
Score 50.0
USR-0111-20230816-ai-specialis-06d67c0f
[ LinkedIn discussion , Twitter thread ] Never before in my life had I seen so many smart people working on the same goal: making LLMs better. After talking to many people working in both industry and academia, I noticed the 10 major research directions that emerged. The first two directions, hallucinations and context learning, are probably the most talked about today. I’m the most excited about numbers 3 (multimodality), 5 (new architecture), and 6 (GPU alternatives). 1. Reduce and measure hallucinations Hallucination is a heavily discussed topic already so I’ll be quick. Hallucination happens when an AI model makes stuff up. For many creative use cases, hallucination is a feature. However, for most other use cases, hallucination is a bug. I was at a panel on LLM with Dropbox, Langchain, Elastics, and Anthropic recently, and the #1 roadblock they see for companies to adopt LLMs in production is hallucination. Mitigating hallucination and developing metrics to measure hallucination is a blossoming research topic, and I’ve seen many startups focus on this problem. There are also ad-hoc tips to reduce hallucination, such as adding more context to the prompt, chain-of-thought, self-consistency, or asking your model to be concise in its response. To learn more about hallucination: Survey of Hallucination in Natural Language Generation (Ji et al., 2022) How Language Model Hallucinations Can Snowball (Zhang et al., 2023) A Multitask, Multilingual, Multimodal Evaluation of ChatGPT…