Same Skeleton: How GPT, Claude, Gemini, and LLaMA Differ
Next-token prediction, decoding knobs, post-training, and the converging modern Transformer stack — what actually separates familiar model names.
GPT, Claude, Gemini, LLaMA — the names are everywhere. Public detail is limited, especially for closed models, but the broad picture is clear: they largely live in the same Transformer design space. Tokenization, embeddings, position signals, stacked layers of multi-head attention and feed-forward blocks, residual streams, normalization, and a next-token prediction loop form a shared skeleton.
The generation loop
After every layer has processed the sequence, generation usually reads only the final vector of the last token. That vector becomes a score for every vocabulary entry — logits, not yet probabilities. Softmax turns logits into a distribution over the next token. The model rarely always picks the single highest score; temperature, top-k, and top-p shape how sharp or diverse the draw is. One token is appended, KV cache is reused where possible, and the loop repeats until a stop token or the context limit.
The long answer you read is that loop, one token at a time. Speculative decoding is a common efficiency trick behind the scenes: a smaller draft model proposes several tokens, the large model verifies them in parallel, and accepted guesses keep the same distribution as running the large model alone — often faster when the draft is accurate.
Base models only learn “what comes next”
A fact many people overlook: next-token prediction on huge text corpora is the training signal for a base large language model. The base model is not directly optimized for factual accuracy, dialogue manners, coding style, or safety. Instruction following, preference alignment, and safer behavior are largely products of post-training on top of that base — fine-tuning stages that reshape how the same skeleton behaves for users.
Three places models actually diverge
- Weights: the numbers learned from different data mixes and training recipes — still the core difference.
- Configuration: depth, vocabulary size, attention-head layout, total parameters, dense versus mixture-of-experts.
- Post-training: instruction tuning, human-feedback alignment, and safety controls that heavily shape user experience.
A converging toolkit — and what lasts
From roughly 2023 to 2025, many frontier and open-weight models converged on similar choices: pre-norm, RMSNorm, RoPE, SwiGLU, grouped-query attention, and for the largest systems often MoE. No single company invented the whole stack at once; different teams arrived at nearly the same engineering answers after years of polishing the 2017 design.
That convergence is historically unusual — the field once kept vision, language, and audio in separate architectural silos. Transformers now span modalities. Alternatives such as state-space models (for example Mamba) and hybrid designs may matter more for ultra-long sequences. Still, the durable problems remain: turn text into units, give those units meaning and order, exchange information across the sequence, process deeply per position, keep deep stacks numerically stable, and predict the next unit. Understanding those parts makes new papers less like jargon and more like “which component did they change.”