Catching up on the weird world of LLMs

I was so compelled by Simon Willison's presentation at the North Bay Python 2023 conference that I decided to write this article to share my thoughts and insights

The talk covers the topic of Large Language Models (LLMs), which are advanced deep learning algorithms, primarily based on transformer networks, designed for a broad range of language understanding and generation tasks. As autoregressive models, they operate by predicting the next word or token in a sequence.

These networks are trained on vast datasets with billions of parameters enabling them to summarize, translate and generate human language content. However, LLMs also inherit complexities, biases and nuances from their training data.

While it is possible to fine-tune these models, they can also be adapted to specific tasks through prompt engineering. Examples of LLMs include OpenAI's GPT series, Google's Bard and Meta's LLaMa, which have become integral in natural language processing and various industry applications.

The speaker, Simon Willison, is the co-creator of the Python framework Django, director of architecture at Eventbrite and developer of various interesting libraries and tools such as Datasette. He was fantastic throughout the presentation, as he seemed genuinely fascinated by the concepts he was explaining as well as their current and future applications.

Willison approached the presentation focusing on practical aspects, such as how to use, fine-tune and build projects using LLMs. He encouraged listeners to play around with the models he was explaining, an idea he embodied himself, having experimented with the systems and built tools on top of them. However, when he needed to, he introduced basic Natural Language Processing concepts in a straightforward manner, allowing attendants to grasp the ideas without requiring extensive prior knowledge of the topic.

Additionally, Willison presented a timeline of recent advancements in LLMs, highlighting key research papers that have significantly influenced the field. This served as a base guide of the LLM landscape for newcomers and reading material for more experienced listeners.

Retrieval Augmented Generation

One of the concepts that stood out the most to me is Retrieval Augmented Generation, a technique in which the model's output depends on some information it requests automatically through an internet search or a database lookup. This ability allows the model to overcome its “knowledge cutoff” — the most recent point in time up to which the model has been trained — by allowing it to access up-to-date information that is beyond its existing knowledge base.

My first experience with a model with these capabilities was through Microsoft’s revamped Bing, which I used to search for influential papers on a project I was working on at the time. It provided me with real paper titles and links to continue my research, in contrast to some other LLMs which could have “hallucinated” them. Although this may seem failproof, it also opens up the possibility of using sources that are not trustworthy.

LLM Security

On the hallucination topic, Willison raised intriguing concerns regarding security and trust. Deploying hallucination-prone models to productions is troublesome, to say the least. Trying to reduce hallucinations and avoiding harmful or unsafe responses from the model through Reinforcement Learning with Human Feedback (RLHF) might backfire, as it may result in a model that is overly cautious with its output. In the following Reddit post, a user shares a conversation with the newly released Claude 2.1, in which it refuses to kill a Python process as it cannot provide a command that may “harm processes or systems”.

Even if we manage to create and launch a correctly aligned and hallucination-free model, malicious users may still try to hijack it to their benefit. This may be carried out using Prompt Injection, a technique that allows to override the model’s intended purpose through a prompt, enabling the user to use the model for a different task or to access confidential information, such as the hidden initial prompt of the model.

In the following tweet, we can see a case in which a researcher manages to use a GPT4-powered chatbot to perform any task, by first providing a question regarding its original intended use. In the replies, another ML practitioner manages to persuade the bot to share its instructions:

Transparency and Open-Source

Willison also covered transparency issues, such as the closed nature of the best-performing models and the reluctance to share the sources of data with which they have been trained (partly fueled by legal concerns). This secrecy is extremely harmful to the LLM landscape as it misdirects researchers, causes resource misallocation and leads to poor regulation due to fears driven by public opinion.

Open source allows wider access to AI technologies, enabling more people to improve and ethically manage these models. With more experts involved, issues like biases can be more effectively identified and addressed. Open source also promotes trust in AI, as everyone can see how the models are built and operate. This approach is vital for making AI safer and more beneficial for a broader audience, ensuring adherence to robust data privacy and security standards.

Conclusion

Willison’s talk was both a condensed way to get up-to-date with the LLM landscape and a compelling call to action for ethical, transparent, and innovative AI development. As a very didactic presentation that covers a widespread technology, I would recommend it to everyone, but especially to those interested in Natural Language Processing and Large Language Models in particular.