LLM Fundamentals for Beginners: Tokens, Embeddings, Context Windows and Cost

If you want to work with generative AI, the LLM fundamentals in this guide are the vocabulary you will use every day: tokens, embeddings, context windows, temperature, cost, grounding and fine-tuning. None of it needs advanced mathematics. It does need a clear mental model, because most beginner mistakes with large language models come from guessing how they work rather than knowing.

What a large language model is

A large language model is a neural network trained on a very large amount of text to do one job: predict the next token given everything that came before. Everything you see a model do, from answering questions to writing code, is that single prediction step repeated one token at a time, with each new token fed back in as input.

That picture removes a lot of mystery. The model is not looking anything up, and it does not check its own work unless it is prompted or given tools to do so. If the wider terms are still blurry, our guide to the difference between AI and machine learning is a useful first read.

Tokens: how text becomes numbers

Models do not read characters or words. They read tokens, which are chunks of text produced by a tokeniser from a fixed vocabulary. Common words are usually one token; rare words, names and unusual spellings are split into several pieces. Spaces and punctuation count too.

Take a familiar sentence and a typical split:

The quick brown fox jumps over the lazy dog.
["The", " quick", " brown", " fox", " jumps", " over", " the", " lazy", " dog", "."]
44 characters / 10 tokens = 4.4 characters per token

That ratio is where the rule of thumb of roughly four characters per token in English comes from. It is only a rule of thumb: code, numbers, non-English text and unusual formatting usually cost more tokens per character, because the tokeniser falls back to smaller pieces. Before relying on an estimate, run real text through your model's tokeniser and measure.

Token counts govern how much fits in the context window, how long a response takes, and what a request costs.

Embeddings: why king and queen sit close together

An embedding is a list of numbers, a vector, that represents a token, a word, a sentence or a whole document. The model learns these vectors during training so that items used in similar contexts end up pointing in similar directions. Similarity is then measured with simple geometry, most often cosine similarity, which scores how closely two vectors align regardless of their length.

This is why "king" and "queen" sit close: they appear in the same kinds of sentences, next to the same kinds of words, so training pushes their vectors together. The classic illustration is that the vector arithmetic king minus man plus woman lands near queen, because the direction between the two pairs encodes something like gender while the rest of the meaning stays shared. Sentence embeddings work the same way at a larger scale, which is what makes semantic search possible.

You do not need to train embeddings yourself; pre-trained models turn any text into a vector with one call, and the mathematics of similarity is standard material in any machine learning course.

Context windows and what falls out

The context window is the maximum number of tokens a model can consider in a single request, counting everything: the system prompt, the conversation so far, any documents you paste or retrieve, and the answer it is about to write. It is a hard limit, not a suggestion.

The arithmetic is worth doing once. At roughly three-quarters of an English word per token, a 128K-token window holds about 96,000 words in total. That sounds enormous until a long system prompt, twenty turns of chat and several retrieved documents are added, and the space left for the answer shrinks.

When the limit is exceeded, either the request is rejected or the application silently drops content, usually the oldest turns, so the model forgets what you told it earlier. Neither is a model failure; both are decisions your code has to make deliberately. Keeping a running token count, summarising old turns and pinning must-keep facts in the system prompt are the standard remedies.

Temperature and sampling in plain words

At each step the model produces a probability for every token in its vocabulary. Sampling is the rule for picking one. At temperature zero the model takes the most likely token every time, which gives near-repeatable output. Raising the temperature flattens the probabilities so less likely tokens get chosen more often, which reads as more varied and, at high settings, less coherent.

Top-p sampling is the other control you will meet. It restricts choices to the smallest set of tokens whose combined probability reaches a threshold, then samples within that set. In practice, use low temperature for extraction, classification and code, and moderate temperature for brainstorming and drafting. Change one setting at a time while you learn.

Why cost scales with tokens in and out

Usage-based pricing for LLM APIs is almost always per token, with separate rates for input (what you send) and output (what the model generates). The formula is short:

cost = (input_tokens  x input_rate_per_token)
     + (output_tokens x output_rate_per_token)

Two consequences follow. First, in a chat application the whole history is re-sent on every turn, so input tokens grow with each message. If every turn adds 200 tokens, the tenth turn sends 2,000 tokens, and across ten turns you have sent 200 + 400 + ... + 2,000 = 11,000 input tokens for a conversation containing only 2,000 tokens of text. Second, output is generated one token at a time, so long answers cost more money and more waiting time, and output tokens are commonly priced higher than input tokens.

The habits that follow are simple: keep system prompts tight, cap the maximum output length, trim or summarise history, and log token counts for every call. Rates differ by model and change over time, so always calculate with current published figures.

Hallucination and grounding

A hallucination is a fluent, confident answer that is wrong. It is not a bug in the usual sense but the natural result of a system built to produce plausible text rather than verified text. The model has no notion of true and false, only of likely and unlikely.

Grounding fixes this by putting the facts in the prompt, so the model has something to copy from rather than something to guess. Retrieval-augmented generation (RAG) automates that: your documents are split into chunks, each chunk is turned into an embedding and stored, a user question is embedded the same way, the nearest chunks are retrieved by similarity, and those chunks are placed in the context with an instruction to answer only from them and to cite them. Every concept in this article appears in that one pipeline, which is why RAG makes a good first project.

Fine-tuning vs prompting

Prompting changes a model's behaviour by changing its input: instructions, examples and retrieved context. It is cheap to iterate on and reversible in seconds. Fine-tuning changes the model's parameters by training further on your own examples. It suits a consistent output format, a particular tone, or shortening a prompt that has grown very long.

Fine-tuning is a poor tool for adding facts, because the model can still hallucinate over tuned material; use retrieval for knowledge. The working order for almost every project is: prompt first, add RAG when the model needs your data, and fine-tune only for a measured problem that prompting cannot solve.

A short study path for LLM fundamentals

Reading about these ideas is not the same as owning them. The exercises build on each other, and all run on an ordinary laptop with a free API tier or a small local model.

1. Tokenise real text. Run ten sentences, a code snippet and a paragraph in another language through an open-source tokeniser. Record characters per token and note where the four-character rule breaks.
2. Measure similarity. Embed twenty words and five sentences with a pre-trained model, then write a cosine similarity function in Python and print the nearest neighbours for a few queries.
3. Manage a context window. Build a command-line chat loop that keeps a running token count, trims the oldest turns at a budget, and shows what was dropped.
4. Test sampling. Send the same prompt five times at temperature zero and five times at a high temperature. Compare the outputs and note which tasks each setting suits.
5. Write a cost calculator. Given token counts and a rate card, compute the cost of a single call and of a ten-turn conversation, then shorten the system prompt or cap the output and recompute.
6. Build a minimal RAG. Index your own study notes, retrieve chunks for a question, pass them to a model and check whether the answer is grounded in what you retrieved.

A starting point for exercise two:

import math

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    na = math.sqrt(sum(x * x for x in a))
    nb = math.sqrt(sum(x * x for x in b))
    return dot / (na * nb)

If you are new to Python, complete a Python programming course or its equivalent first; every exercise assumes you can write functions, loops and file handling unaided. For a guided route, the AI course at Aiinfox Academy covers tokens, embeddings, RAG and deployment in the same order, and the beginner resource guide lists free material for self-study. Questions about which route fits are welcome through the contact page.