In this lesson, I want to cover the fundamentals of working with context and cache. You simply need to know these things to interact effectively with an LLM. Understanding context will allow you to formulate a task with surgical precision. And understanding cache will help you avoid going broke while working—or if you decide to write your own AI agent.
Today we'll talk a little about the limitations of cloud-based models and how to deal with them. This is about censorship built into a model's reasoning block. The biggest problem with censorship is that it develops and spreads endlessly, and every month you're forbidden from talking and thinking about an ever-growing number of things. And it permeates every area of life. The most disgusting kind of censorship arises because someone might take offense at words. Sometimes I rewatch old movies in which people spoke naturally and were capable of delivering great jokes regardless of gender, age...
One token is not one letter. Although sometimes it is. A token is a unit of meaning in an LLM. It can be a whole word, part of a word, or a punctuation mark. Understanding how tokens work makes it possible to use a neural network more effectively. That is what we will talk about today.
I kept talking one of my colleagues’ ears off about trying an LLM to solve a coding task for a fairly large feature. Eventually, he gave in and tried it, after which he spent a long time complaining and wondering how stupid the neural network was. I hadn’t explained or told him anything, thinking he would figure it out himself. As it turned out, he had used the ChatGPT 4-mini model for coding, which is a very small and stupid model for tasks like these. That’s why I decided this article would be worthwhile.
Hi, everyone! I created a new blog and decided to start with the most relevant topic today: product development with AI. This guide is intended for a broad audience: everyday developers, people just starting out in programming, and those who have spent the last few years asleep and paid little attention to AI. I’ll try to explain all the important aspects briefly and clearly, and share recipes and tools for writing maintainable, extensible, production-ready code that people can understand.
