Blog
Announcing my new side-project and third non-fiction, technical book: Elixir Chatbot Alchemy !
Following the same engineering ethos and exploratory learning approach that has proven so popular with readers of Northwind Elixir Traders , Elixir Chatbot Alchemy explores the implementation of a chatbot in Elixir. By utilizing the REST API of a self-hosted instance of Ollama and CPU inference on small(-ish) open-weight LLMs, we explore how LLMs operate, the impact of model choice on the quality of inference, and use Phoenix LiveView to build a variety of features, such as document summarization and a chatbot that can be used on a webpage, and an admin UI with reporting of the use of the chatbot.
Note: this is a super-early low-fanfare release, subject to heavy changes of scope and content.
Current content
- Requirements on the Elixir application (what we want to achieve)
- Choosing an LLM suitable for CPU inference
- Calling Ollama from Elixir
- Understanding Ollama’s HTTP response
- Fundamental knowledge of LLMs
- Short conversations and implementing a chat with completion
- Context truncation and periodic summarization
- Failure modes of completion
- Exceeding the context length
- Context compression through summarization
- GPU-based inference, parallel execution, and scaling
- Summarizing the summaries
- Sidequest: bigger models, better summaries?
- Context stuffing
Planned content
The following sections are envisioned within the scope of the book, but not yet set in stone.
- Implementing an actual chat
- Using LLM system prompts
- Storing chats in SQLite
- Tracking chat state with a GenServer
- Extracting and summarizing text from documents
- Categorizing a document
- Extracting structured JSON data from text
“Maybe” content
The following sections are speculative and any of those might be dropped entirely.
- Phoenix LiveView of a chat on a page
- Implementing an admin UI with reporting
Release schedule
This book is currently a work in progress at a very early stage, and new chapters are being written and released infrequently depending on my workload, as I’m currently focused on Phoenix Product Codex .
For early adopters
An early draft has been released on Leanpub –also note that early adopters benefit from lower prices while the book is in development.