From Neural Networks and Markov Chains to Large Language Models
To understand how a large language model works, it helps to begin with two simpler ideas.
The first is a neural network, which learns to transform inputs into useful predictions by adjusting numerical parameters. The second is a Markov chain, which models how a system moves from one state to another using probabilities.
A modern language model combines neural-network training with probabilistic sequence generation. It is much more powerful than a basic Markov chain because it can use a long section of earlier text instead of looking only at the current state.
What a neural network does
A neural network is a mathematical system that receives numbers, processes them through several layers, and produces an output.
The network contains adjustable numbers called weights and biases. When the network is first created, these values are mostly random. Training gradually changes them so the network produces better answers.
A simple image-recognition task provides a clear example of how this works.
Recognizing handwritten digits
The MNIST dataset contains images of handwritten digits from zero through nine. Each image is 28 pixels wide and 28 pixels tall. That means every image contains 784 pixels.
Each pixel is stored as a brightness or intensity value between 0 and 255. The dataset also includes the correct label for each image. An image of a handwritten seven has the label 7.
The standard MNIST dataset contains 60,000 training images and 10,000 test images. Keras MNIST documentation
The model’s task is to receive the pixel values and predict which digit the image contains. The effect of using labeled examples is that the model has a correct answer against which every prediction can be compared. This is called supervised learning.
Training and test data serve different purposes
The training images are used to adjust the model’s weights. The test images are kept separate and used only after training. They show how well the network performs on images that did not directly change its weights.
Part of the training data can also be held back as validation data. For example, 10 percent of the training images might be reserved for validation.
The effect of this separation is that developers can determine whether the model learned general patterns or merely became too specialized to the examples it repeatedly saw.
Pixel values are normalized
The original pixel values range from 0 to 255. Neural networks usually train more easily when their inputs use smaller and more consistent numerical ranges. Each pixel can therefore be divided by 255.
A pixel value of 0 remains 0. A value of 255 becomes 1. Every value between them becomes a decimal between 0 and 1.
The effect of normalization is more stable optimization. It prevents large input scales from creating unnecessarily large internal values and usually makes learning faster and more predictable.
Labels can be one-hot encoded
The correct label can be stored as the number 7, but some training setups represent it as a ten-position vector. For a seven, the vector contains a one in position seven and zeros everywhere else:
0 0 0 0 0 0 0 1 0 0
This is called one-hot encoding. Each position represents one possible class, from zero through nine.
The effect of one-hot encoding is that the correct answer has the same basic shape as the network’s output. The network produces ten probabilities, and the one-hot label shows which of those ten probabilities should be high.
If labels remain ordinary integers instead, the model can use a sparse version of the same classification loss. The choice changes the label format, not the basic learning goal.
Flattening changes the image’s shape
The original image is a 28-by-28 grid. A basic dense neural network expects one long list of input numbers. A flattening layer rearranges the image into a list of 784 pixel values.
The image changes from 28 rows × 28 columns to 784 pixel values.
Flattening does not remove any pixel values. It only changes how they are arranged. The effect is that the image becomes compatible with an ordinary dense neural-network layer.
Flattening does remove the explicit two-dimensional layout. A dense network can still learn from the pixels, but it is not automatically told which pixels were neighbors in the original image. Convolutional networks preserve and use spatial structure more directly, but a dense model is easier to understand as a first example.
Dense layers connect values to neurons
In an LLM, a neuron is a small mathematical processing unit inside a neural network. It receives numbers from other neurons, multiplies each number by a learned weight, adds them together with a bias, and passes the result through an activation function.
During training, the model adjusts the neuron’s weights and bias so that it responds to useful patterns in text. A single neuron usually does not represent a complete word or idea. Meaning is distributed across many neurons working together.
After flattening, the 784 pixel values can enter a dense layer containing 128 neurons. A dense layer means every neuron receives information from every value in the previous layer.
Each input is multiplied by a weight. Those weighted values are added together with a bias. The result then passes through an activation function.
The weights determine how strongly each input influences the neuron. The bias allows the neuron’s response to shift independently of the inputs. At the beginning, the weights are largely random. During training, they change.
The effect of the first dense layer is that the network can combine pixels into learned features. Some neurons may become sensitive to lines, curves, edges, or other recurring patterns that help distinguish digits.
These interpretations are only a useful intuition. A neuron does not necessarily have one simple, human-readable purpose. Information is often distributed across many neurons.
Activation functions add flexibility
If every layer performed only linear calculations, several layers would behave like one larger linear calculation. The network would be unable to learn many complicated patterns.
An activation function introduces nonlinearity. A common activation is ReLU. ReLU turns negative values into zero and leaves positive values unchanged.
The effect of ReLU is that different neurons can turn on for different kinds of input. This allows the network to form flexible decision boundaries instead of being limited to simple straight-line relationships.
This matters because people can write the same digit in many different ways. A seven might be narrow, wide, tilted, crossed, or curved. The network needs nonlinear transformations to handle these variations.
A second hidden layer combines simpler features
The output of the 128-neuron layer can pass into another dense layer containing 64 neurons. The second layer receives patterns discovered by the first layer and combines them again.
The first layer might respond to relatively simple features. The second layer can combine those features into more useful patterns for distinguishing complete digits.
The effect of adding another hidden layer is greater processing depth. The network can build more complicated representations from simpler ones.
More layers are not automatically better. They increase the number of parameters and can make training more difficult. The architecture must be appropriate for the task and amount of data.
The output layer represents the possible digits
The final dense layer contains ten neurons because there are ten possible digits. Each output neuron corresponds to one digit from zero through nine.
The output layer uses softmax. Softmax turns the ten raw scores into probabilities between zero and one that add up to one.
The model might produce a 0.97 probability for seven, a 0.02 probability for one, and very small probabilities for the remaining digits. The final prediction is usually the digit with the highest probability.
The effect of softmax is that the network’s raw output becomes an understandable probability distribution across the available classes.
The complete digit network
The complete network begins with a 28-by-28 image. It flattens the image into 784 values, sends those values through 128 ReLU neurons, sends the result through 64 more ReLU neurons, and produces ten softmax probabilities.
In compact form, the shape is:
28 × 28 image
784 flattened values
128 ReLU neurons
64 ReLU neurons
10 output probabilities
The effect of this arrangement is a complete path from raw pixel values to a digit prediction.
Loss measures how wrong the prediction is
The model needs a way to compare its output with the correct answer. For one-hot labels and a softmax output, categorical cross-entropy is a common loss function.
If the image contains a seven and the model assigns a high probability to seven, the loss is low. If the model assigns a very low probability to seven, the loss is high.
The effect of the loss function is that the model’s error becomes a single number that training can reduce.
Loss is more informative than simply recording whether the top choice was right or wrong. It also measures how much probability the model gave to the correct answer.
Accuracy provides an easier measurement
Accuracy measures the percentage of images classified correctly. If the model correctly identifies 97 of 100 images, its accuracy is 97 percent.
The effect of accuracy is easy interpretation. It tells us how often the model’s highest-probability choice was correct.
Loss and accuracy measure different things. Accuracy counts correct answers. Loss also measures confidence. Two models can have the same accuracy but different losses if one is more confidently wrong on its mistakes.
The optimizer adjusts the parameters
An optimizer controls how the weights and biases are updated. A common optimizer for a small classification model is Adam.
During training, the network first performs a forward pass. The image moves through all the layers and produces ten probabilities. The loss function compares those probabilities with the correct label.
Backpropagation then works backward through the network and calculates how each weight and bias contributed to the error. Adam uses those gradients to adjust the parameters.
The effect of the optimizer is gradual learning. One update changes the model only slightly. Thousands of updates across many examples create the larger improvement.
Training occurs in batches and epochs
The complete training dataset is normally divided into batches. A batch size of 32 means the model processes 32 images before applying one parameter update.
An epoch means the model has worked through the entire training dataset once. A small digit model might train for five epochs with a batch size of 32 while reserving 10 percent of the training data for validation.
The effect of batching is computational efficiency and a useful estimate of the gradient from several examples. The effect of multiple epochs is repeated practice. The model can revisit the training data and continue improving its parameters.
Too few epochs may leave the model undertrained. Too many may cause overfitting, especially if validation performance begins to worsen.
Testing measures generalization
After training, the model is evaluated on the separate test set. These test images were not used to update the weights.
One representative run of this small dense network produced a test accuracy of about 97.57 percent and a test loss of about 0.0837. Exact results can vary because of random initialization, training order, software settings, and other details.
The effect of final testing is an estimate of how well the trained model generalizes within the MNIST task.
High MNIST accuracy does not prove that the model can recognize every handwritten digit found in the real world. The test set comes from the same general dataset and format as the training set.
Predicting one image
To test one image, the image still needs the shape expected by the model. If the network normally receives batches, one 28-by-28 image is given an extra batch dimension so it represents a batch containing one image.
The image passes through the flattening layer, hidden layers, and softmax output. The model returns ten probabilities. The position with the highest probability becomes the predicted digit.
If the highest probability is at position seven and the actual label is seven, the model classified that image correctly.
The effect of this final step is the use of a trained neural network on new input. Training has ended, so the weights are no longer changed during the prediction.
What the digit example teaches us
The handwritten-digit model demonstrates the basic neural-network process.
Numbers enter the model. Layers transform those numbers. The output is compared with a known answer. Loss measures the error. Backpropagation calculates gradients. An optimizer updates the parameters. Repeating this process teaches the model to recognize patterns.
A large language model uses the same basic learning process. The input data, architecture, and training scale are different, but loss, gradients, parameters, and optimization remain central.
Before moving to language models, it helps to understand another kind of system: a Markov chain.
Markov Chains
A Markov chain models a system that moves between possible states.
Unlike a neural network, a basic Markov chain does not need layers, neurons, backpropagation, or an optimizer. Its behavior is defined by transition probabilities.
The important idea is that the probability of the next state depends only on the current state.
The Markov property
Imagine a traffic light with three possible states: red, green, and yellow. You may not know exactly when the light will change. However, its current color gives you information about what could happen next.
In a first-order Markov chain, the next state depends only on the current state. Once the current state is known, the earlier sequence of states does not provide additional information.
If the light is currently red, the next-state probabilities depend on red. The model does not care whether the light has been red for one step or ten steps. It also does not care what color appeared before red.
This is called the Markov property. Stanford’s course notes describe it as a system in which knowing the current state determines the probabilities of the next state, while older states add no further information. Stanford Markov-chain notes
The effect of this assumption is simplicity. The model needs to store only the current state instead of the complete history.
The limitation is also important. Many real systems depend on duration, history, hidden conditions, or outside information. A basic Markov chain cannot represent those effects unless its state is expanded to include them.
States describe the possible conditions
The traffic-light example has three states: red, green, and yellow. At any step, the system occupies exactly one of these states.
The effect of defining the state space is that the system’s possible conditions become explicit. Every transition must begin in one state and end in one of the allowed states.
Transition probabilities describe possible changes
Each current state has probabilities for every possible next state.
In a simplified example, a red light might have a 60 percent chance of remaining red, a 40 percent chance of changing to green, and no chance of changing directly to yellow.
A green light might have a 70 percent chance of remaining green, a 30 percent chance of changing to yellow, and no chance of changing directly to red.
A yellow light might have an 80 percent chance of changing to red, a 20 percent chance of remaining yellow, and no chance of changing directly to green.
These probabilities are only a teaching example. Actual traffic signals are usually controlled by timers, sensors, programming, and safety rules rather than random Markov transitions.
The effect of transition probabilities is that uncertainty becomes measurable. We may not know the exact next state, but we can describe how likely each possible state is.
The transition matrix stores the probabilities
The transition probabilities can be stored in a transition matrix. Each row represents the current state. Each column represents a possible next state.
For the traffic-light example, the rows can be interpreted like this:
Current red: 0.60 red, 0.40 green, 0.00 yellow
Current green: 0.00 red, 0.70 green, 0.30 yellow
Current yellow: 0.80 red, 0.00 green, 0.20 yellow
Every row must add up to one because one of the possible next states must occur.
The first row answers the question, “If the light is currently red, what are the probabilities of the next color?”
The effect of the transition matrix is that all one-step behavior is stored in one organized numerical structure.
One simulation produces one possible path
Suppose the system begins in the red state. The program looks at the red row of the transition matrix. It randomly chooses the next state using the probabilities 0.60 for red and 0.40 for green.
If green is selected, green becomes the current state. The program then uses the green row to choose the following state.
This process can produce a path such as:
Red → Red → Red → Green → Green → Yellow → Red
This is only one possible path. Running the simulation again can produce a different sequence.
The effect of random selection is that the generated sequence follows the transition probabilities without becoming completely predetermined.
NumPy provides pseudo-random number generators that can sample from probability distributions. NumPy random-sampling documentation
A random seed makes a simulation repeatable
A random seed initializes the pseudo-random number generator. Using the same seed, such as 42, allows the same program to reproduce the same sequence of random choices under the same implementation and settings.
The effect of setting a seed is reproducibility. It makes debugging, comparison, and demonstration easier.
The simulation is still modeling randomness. The seed simply makes that pseudo-random sequence repeatable.
A long simulation reveals overall behavior
One short path does not reveal much about the system. The simulation can be repeated for 10,000 transitions. Every selected state is stored in a history.
Afterward, the program counts how often the light occupied each state.
One 10,000-step simulation using these transition probabilities produced approximately 34.9 percent red, 47.9 percent green, and 17.2 percent yellow.
The exact percentages vary slightly between random runs. Over many steps, they should remain near the chain’s long-run distribution.
Green becomes the most common state because once the system reaches green, it has a 70 percent chance of remaining there on each transition.
The effect of a long simulation is that stable overall patterns can emerge even though the exact path remains unpredictable.
Matrix powers predict several steps ahead
The transition matrix describes one-step probabilities. Multiplying the transition matrix by itself combines two transitions. Raising the matrix to a higher power gives probabilities after several transitions.
If the system begins at red, the appropriate row of the transition matrix raised to the tenth power gives the probabilities of being red, green, or yellow after exactly ten transitions.
In this example, green has the largest probability after ten transitions.
The effect of matrix powers is that future state probabilities can be calculated without simulating every possible path individually.
The exact sequence remains unknown, but the distribution of possible outcomes can be calculated.
Visualizations make the chain easier to inspect
A timeline can display every simulated traffic-light state as a colored segment. A bar chart can show the percentage of time spent in red, green, and yellow.
The effect of the timeline is that the sequence and runs of repeated states become visible. The effect of the bar chart is that long-run behavior can be compared more easily.
What a Markov chain can and cannot tell us
A Markov chain can describe the possible states, the probabilities of moving between them, the probability of future states, and the amount of time the system is likely to spend in each state.
It cannot normally predict the exact path that one random simulation will take. It also assumes that the transition probabilities remain the same unless the model is designed to change them.
Markov chains are useful for weather models, queues, reliability systems, page navigation, biological sequences, games, customer behavior, and many other systems that change over time.
The effect of the Markov model is a compact description of probabilistic movement between states.
Markov chains can generate simple text
A basic text generator can treat each character or word as a state. The program can examine a text dataset and count which characters follow each current character.
If q is usually followed by u, the transition probability from q to u becomes high.
During generation, the program begins with one character, samples the next character from its transition probabilities, adds it to the text, and repeats. This is essentially a bigram language model.
The effect is simple text generation that reproduces local patterns from the training material.
The limitation is memory. A first-order character Markov chain sees only the current character. It cannot use a sentence, paragraph, or conversation to choose the next one.
From Markov Chains to Language Models
A language model and a Markov chain both work with probability distributions over what might happen next.
A basic Markov text generator uses a fixed transition table based only on the current token. A neural language model uses a trainable neural network to calculate the next-token probabilities.
A Transformer goes further by using many earlier tokens, not only the current one.
This distinction is important. A modern LLM is not simply a first-order Markov chain over individual tokens.
A finite-context language model could be described as Markov-like if its complete context window is treated as the current state. However, that state may contain thousands of tokens and is processed by a large neural network.
The effect of the Transformer is that next-token probabilities can change based on a much richer context.
Large Language Models
A large language model is a neural network trained to predict tokens in sequences.
It uses the training process demonstrated by the handwritten-digit network, but its inputs and outputs are pieces of text.
It also performs probabilistic sequence generation like a Markov chain, but it calculates each probability distribution from a much larger context.
The model predicts one token at a time
A token is a small unit of text. It might be a word, part of a word, punctuation, a space, or a single character.
Suppose the model receives:
The robot opened the
The model calculates a score for every token that could come next. It might decide that door is very likely, window is somewhat likely, and sandwich is unlikely.
The model selects one token and adds it:
The robot opened the door
It then predicts again using the updated text:
The robot opened the door and
This continues until the response is complete.
The effect of next-token training is that the model must learn far more than simple token pairs. To make accurate predictions, it needs to recognize spelling, grammar, sentence structure, writing styles, factual associations, programming syntax, and relationships between distant parts of the text.
GPT-3 demonstrated that scaling autoregressive next-token training could produce strong performance across many language tasks. GPT-3 research paper
Language-model training begins with text
Before training, the model’s parameters are mostly random.
Developers collect a large text dataset. A small educational model might use Tiny Shakespeare, which contains about one million characters. A commercial model may use a much larger and more varied mixture of books, articles, websites, conversations, documentation, and code.
The data is normally cleaned, filtered, and deduplicated. Part of it is kept separate for validation.
The effect of the training corpus is similar to the effect of the MNIST images. It provides examples from which the model can learn.
The difference is scale and complexity. Instead of learning ten image categories, the language model learns a probability distribution over possible tokens in many kinds of context.
The tokenizer turns text into units
Text must be converted into numbers before the neural network can process it. A tokenizer divides the text into tokens and gives each token an ID.
A character tokenizer treats every character separately. A word tokenizer tries to keep words whole. Most modern LLMs use subword or byte-based units.
Subword tokenization can keep common words intact while dividing rare words into smaller reusable pieces. Systems such as SentencePiece can learn subword units directly from text. SentencePiece research paper
The token ID is only a label. It does not represent importance or meaning.
The effect of tokenization is that language becomes a sequence of numerical symbols.
Tokens are divided into context windows
The model does not process an entire training corpus in one operation. The token sequence is divided into shorter context windows. A small character model might use 128 tokens.
Inputs and targets are shifted by one position:
Input: h e l l
Target: e l l o
This teaches the model that e follows h, l follows he, another l follows hel, and o follows hell.
The effect is that every position becomes a training example.
The context-window size also limits what the model can directly use. Information outside the window is not available unless an external system supplies it again.
Batches make training efficient
Several token sequences are processed together in a batch. A batch might contain 64 sequences with 128 tokens in each sequence.
The effect is similar to batching MNIST images. Hardware can perform many related calculations in parallel before the optimizer updates the model.
Embeddings replace token IDs with learned vectors
A token ID is not passed through the model as a meaningful scalar number. Instead, the ID selects an embedding vector.
An embedding contains many adjustable values. A small model might use 128 values per token. During training, the embeddings change into representations that help predict text.
The effect is that discrete token labels become flexible numerical representations that later layers can compare and transform.
Position information preserves order
The model must know not only which tokens are present but also where they occur. “Dog bites man” and “Man bites dog” contain the same main words, but their order changes the meaning.
Position information is therefore added or applied to the token representations.
The effect is that the model can distinguish between earlier, later, nearby, and distant tokens.
A causal mask hides future tokens
During training, many positions are processed together. A causal mask prevents an earlier position from seeing future tokens containing the correct answers.
Each position can use itself and earlier tokens, but not later ones.
The effect is that training matches real generation, where future tokens do not exist yet.
Self-attention finds relevant context
A Transformer uses self-attention to decide which earlier positions may be useful for the current prediction.
Each token representation produces a query, a key, and a value. The query represents what the current position is looking for. The key represents what a token can be matched by. The value contains the information that token can contribute.
The query is compared with the available keys. Stronger matches receive larger scores.
The scores are scaled to keep their size manageable. The causal mask removes future positions. Softmax converts the remaining scores into attention weights.
The weights are then used to combine the value vectors.
The effect of self-attention is that every token receives a context-dependent representation. Useful earlier information can have more influence than less relevant information.
Multiple attention heads examine different patterns
One attention calculation is called a head. A complete Transformer uses several heads at the same time.
In a small model with 128-value representations, four heads can each produce 32 values. Their outputs are joined back into 128 values.
Different heads can learn different attention patterns. One may respond to nearby grammar, while another may become useful for punctuation, speaker names, or longer-distance relationships.
The effect is that the model can examine the same context in several ways at once.
Multi-head attention is one of the central ideas in the original Transformer architecture. Attention Is All You Need
The output projection combines the heads
After the head outputs are joined, a learned output projection mixes them together.
The effect is coordination. Information found by one head can be combined with information found by another.
The feed-forward network transforms each position
Attention moves information among token positions. A position-wise feed-forward network then processes the collected information separately at every position.
A small model might expand a 128-value representation to 512 values, apply GELU, and reduce it back to 128. The temporary expansion gives the network more internal space to transform the information.
The effect is that attention decides where information comes from, while the feed-forward network changes how that information is represented.
Dropout discourages fragile dependencies
During training, dropout can randomly turn off a small portion of intermediate values. The model cannot depend on one pathway always being available.
The effect is regularization. Useful information is encouraged to spread across multiple connections.
Dropout is normally disabled during validation and generation.
Layer normalization controls internal values
Layer normalization keeps internal values within a more consistent range.
The effect is more stable training, especially when many transformer blocks are stacked together.
Residual connections preserve information
A residual connection adds a layer’s transformed output back to its original input. The layer keeps the earlier representation and contributes an update instead of replacing it completely.
The effect is better information flow and a more direct path for gradients through deep networks.
Transformer blocks are stacked
Attention, feed-forward processing, normalization, and residual connections form a transformer block. A small model might stack four blocks. Commercial LLMs use much larger architectures.
The effect of stacking is greater processing depth. Each block refines the representations created by the previous block.
The language-model head produces token scores
After the transformer blocks, a final linear layer produces one logit for every token in the vocabulary. A character model with 65 possible characters produces 65 logits at every position.
Softmax can convert those logits into probabilities.
The effect is that the model’s internal representation becomes a distribution over possible next tokens.
The complete model should be tested before training
Before a long training run, the model should receive a test batch. The output shape, parameter count, and initial loss should be checked.
The initial predictions are expected to be poor because the weights are still random.
The effect of this check is to confirm that the architecture is assembled correctly before spending time training it.
A random baseline shows the starting point
The untrained model can generate a short sample before any parameter updates occur. The output will usually be random or nearly random.
The effect of saving this baseline is that the improvement caused by training becomes visible.
Cross-entropy measures language-model error
At every token position, cross-entropy measures how much probability the model assigned to the correct next token.
The effect is the same as in the digit classifier. Prediction error becomes a numerical loss that backpropagation can reduce.
Backpropagation trains every component together
The language model performs a forward pass and produces token logits. Cross-entropy calculates the loss. Backpropagation calculates gradients through the language-model head, transformer blocks, attention projections, feed-forward layers, position system, and token embeddings.
An optimizer such as AdamW updates the parameters. AdamW research paper
The effect is that all parts of the Transformer gradually learn to cooperate.
Gradient clipping can stabilize training
A batch can occasionally produce unusually large gradients. Gradient clipping limits their combined size before the optimizer applies them.
The effect is protection against a single extreme update damaging the model’s earlier progress.
Training and validation loss should be compared
Training loss measures performance on text used for learning. Validation loss measures performance on held-out text.
Both should generally fall during useful training. If training loss continues falling while validation loss rises, the model may be overfitting.
The effect of monitoring both curves is that developers can distinguish general learning from excessive specialization.
Checkpoints preserve learned parameters
After training, the model’s architecture settings and learned parameter values can be saved in a checkpoint.
The effect is that the model can be loaded later without returning to random initialization and repeating all training.
Generation uses the latest context
During generation, the prompt is tokenized and passed through the Transformer. If the prompt is longer than the context window, only the most recent tokens fit directly inside the model.
The model produces logits for the next token, selects one, adds it to the sequence, and repeats. The effect is autoregressive generation.
This resembles a Markov-chain simulation because each new token is sampled from probabilities. The major difference is that the probabilities are calculated by a neural network using the full available context rather than one fixed transition row based on the current token.
Temperature controls randomness
Temperature changes how concentrated the token probabilities become.
A temperature below 1 makes the highest-scoring tokens more dominant. Output becomes more predictable but may become repetitive.
A temperature above 1 spreads probability more widely. Output becomes more varied but may contain more mistakes.
The effect of temperature is to control how cautiously or freely the model samples. It does not change the model’s trained knowledge.
Top-k filtering removes weak choices
Top-k keeps only a selected number of the highest-scoring tokens.
If a character model has 65 possible characters and top_k is 40, only the best 40 remain eligible. The other 25 receive zero probability after softmax.
The effect is that extremely weak choices cannot be selected.
Sampling produces different continuations
After temperature and filtering are applied, the system samples one token from the remaining distribution.
The highest-probability token is most likely to be chosen, but it is not guaranteed.
The effect is variety. The same prompt can produce different continuations on different runs.
This is another connection to Markov-chain simulation. Both systems can follow fixed probability rules while producing different individual paths.
A trained character model learns structure
After training on Shakespeare, a small character model may begin producing speaker names, colons, line breaks, punctuation, and dialogue-like formatting.
Its sentences may still be incorrect or incomplete, but they are far more structured than the random baseline.
The effect shows that the model learned statistical patterns from the dataset.
It does not prove human-like understanding. It shows that the model became better at predicting characters in Shakespeare-like contexts.
A key-value cache makes generation faster
During generation, attention keys and values from earlier tokens can be stored in a key-value cache.
The effect is that the model can reuse earlier calculations instead of recomputing all of them for every new token.
A working Transformer is not automatically large
A small character model can contain the same basic components as a modern LLM. It can have embeddings, position information, causal attention, multiple heads, feed-forward networks, normalization, residual connections, dropout, and autoregressive generation.
A commercial LLM has a much larger vocabulary, more layers, wider representations, longer context windows, more parameters, more data, and much greater computing requirements.
The effect of scaling is a much wider range of learned patterns and capabilities.
Post-training creates an assistant
Next-token training produces a base language model. A base model is trained to continue text. It is not automatically trained to follow instructions safely or helpfully.
Additional training can use examples of desirable responses and preference comparisons. Techniques such as supervised fine-tuning, reinforcement learning from human feedback, and direct preference optimization encourage more useful behavior.
The effect of post-training is behavioral. It changes which responses the model is encouraged to produce without replacing the underlying token-prediction process.
The InstructGPT research demonstrated that increasing model size alone does not guarantee better instruction following. InstructGPT research paper
Why fluent output can still be wrong
The model is trained to produce probable text, not guaranteed truth. A false statement can receive a high probability if it resembles patterns in the training data or fits the current prompt.
The effect is that an LLM can produce a fluent, detailed, and incorrect answer. This is commonly called hallucination.
Important factual claims still require reliable sources and verification.
External tools can provide missing information
An LLM does not automatically have live internet access, current databases, a calculator, or private records.
Applications can connect the model to search, databases, calculators, code execution, and retrieval systems. In retrieval-augmented generation, relevant documents are placed inside the model’s context.
The effect is that the model can generate answers using information that was not contained in its training data or that changed after training.
The Complete Connection
A neural network teaches us how numerical parameters can learn from examples. A Markov chain teaches us how probabilities can describe movement through a sequence of states. A language model combines these ideas.
Its neural network learns the parameters that produce next-token scores. Its generation process repeatedly samples from probability distributions. Unlike a simple Markov chain, a Transformer calculates those probabilities using many earlier tokens and multiple layers of learned processing.
Tokenization converts text into numerical units. Embeddings give those units learned representations. Position information preserves order. Causal masking hides future answers. Self-attention gathers relevant context. Multiple heads examine different relationships. Feed-forward networks transform the results. Residual connections preserve information. Layer normalization stabilizes the values.
Cross-entropy measures error. Backpropagation calculates gradients. The optimizer updates the parameters. Validation measures whether the learning transfers to held-out data.
During generation, temperature changes the sharpness of the distribution, top-k removes weak choices, and sampling selects the next token. The selected token is added to the context, and the process repeats.
This progression, from basic neural-network learning, through probabilistic state transitions, to context-aware Transformer prediction, explains the foundation of a modern GPT-style large language model.
Research acknowledgment: This article is based in significant part on the research and explanations presented by @lerabyte on TikTok. Their work provided the foundation for many of the concepts discussed here. The article was independently written and adapted, and any errors in interpretation are my own.
