--- title: LLM Evolutionary Tree document: tree canonicalUrl: https://brianletort.ai/industry/tree schemaVersion: 2026.05.02 lastUpdated: '2026-09-05' counts: branches: 7 models: 264 vendors: 49 signals: 33 breakthroughs: 18 branches: - foundational - encoder_only - encoder_decoder - decoder_only - mixture_of_experts - multimodal - reasoning vendors: - 01.AI - AI21 Labs - Alibaba - Allen Institute for AI - Anthropic - Baidu - BigScience - ByteDance - CMU + Google - Cohere - Cohere For AI - Databricks - DeepMind - DeepSeek AI - Google - Google + DeepMind - H Company - HuggingFace - IBM - Inworld - JetBrains - Liquid AI - Meituan - Meta - Meta AI - Microsoft - MiniMax - Mistral AI - Mistral AI / NVIDIA - Moonshot AI - NVIDIA - OpenAI - OpenBMB - Ornith AI - Perceptron - Poolside - Snowflake - SpaceXAI - Stanford NLP - TII - Tencent - Thinking Machines Lab - Thomson Reuters - Tsinghua THUDM - Xiaomi - Z.AI (Zhipu) - Zyphra - fast.ai - xAI --- # The LLM Evolutionary Tree *A living lineage of frontier and open-weight models. Last updated 2026-09-05.* **Counts.** 264 models across 7 branches and 49 vendors. 33 infrastructure signals. 18 architectural breakthroughs. ## Branches ### Foundational (`foundational`) Pre-transformer language representations. Word embeddings and early contextual representations that established distributed semantics and the pretrain-then-finetune paradigm. _5 models._ - `word2vec` — Word2Vec (Google, 2013-01-16) - `glove` — GloVe (Stanford NLP, 2014-10-25) - `fasttext` — FastText (Meta, 2016-07-15) - `ulmfit` — ULMFiT (fast.ai, 2018-01-18) - `elmo` — ELMo (Allen Institute for AI, 2018-02-15) ### Encoder-only (`encoder_only`) Bidirectional transformer encoders trained with masked-language-modeling objectives for understanding tasks (classification, NER, QA, retrieval). _10 models._ - `bert-base` — BERT (base) (Google, 2018-10-11) - `bert-large` — BERT (large) (Google, 2018-10-11) - `xlnet` — XLNet (CMU + Google, 2019-06-19) - `roberta` — RoBERTa (Meta, 2019-07-26) - `albert` — ALBERT (Google, 2019-09-26) - `distilbert` — DistilBERT (HuggingFace, 2019-10-02) - `electra` — ELECTRA (Google, 2020-03-23) - `ernie-3` — ERNIE 3.0 (Baidu, 2021-07-05) - `text-embedding-3` — text-embedding-3 (OpenAI, 2024-01-25) - `openai-privacy-filter` — OpenAI Privacy Filter (OpenAI, 2026-04-28) ### Encoder-decoder (`encoder_decoder`) Sequence-to-sequence transformers with separate encoder and decoder stacks; pretrained with denoising or text-to-text objectives. _6 models._ - `t5` — T5 (Google, 2019-10-23) - `bart` — BART (Meta, 2019-10-29) - `glm` — GLM (original) (Tsinghua THUDM, 2021-03-18) - `t0` — T0 (BigScience, 2021-10-15) - `ul2` — UL2 (Google, 2022-05-10) - `flan-t5` — FLAN-T5 (Google, 2022-10-20) ### Decoder-only (`decoder_only`) Autoregressive transformers that generate text token-by-token using only a decoder stack. The dominant architecture for chat models prior to the test-time-compute era. _75 models._ - `gpt-1` — GPT-1 (OpenAI, 2018-06-11) - `gpt-2` — GPT-2 (OpenAI, 2019-02-14) - `gpt-3` — GPT-3 (OpenAI, 2020-06-11) - `lamda` — LaMDA (Google, 2021-05-18) - `chinchilla` — Chinchilla (DeepMind, 2022-03-29) - `palm` — PaLM (Google, 2022-04-04) - `gpt-3-5-turbo` — GPT-3.5 Turbo (OpenAI, 2022-11-30) - `llama-1` — LLaMA 1 (Meta AI, 2023-02-24) - `jurassic-2` — AI21 Jurassic-2 (AI21 Labs, 2023-03-09) - `gpt-4` — GPT-4 (OpenAI, 2023-03-14) - `claude-1` — Claude 1 (Anthropic, 2023-03-14) - `claude-instant-1` — Claude Instant 1 (Anthropic, 2023-03-14) - `palm-2` — PaLM 2 (Google, 2023-05-10) - `falcon-7b-40b` — Falcon 7B / 40B (TII, 2023-05-25) - `phi-1` — Phi-1 (Microsoft, 2023-06-21) - `claude-2` — Claude 2 (Anthropic, 2023-07-11) - `llama-2` — Llama 2 (Meta AI, 2023-07-18) - `qwen-7b` — Qwen-7B (Alibaba, 2023-08) - `code-llama` — Code Llama (Meta AI, 2023-08-24) - `falcon-180b` — Falcon 180B (TII, 2023-09-06) - `phi-1-5` — Phi-1.5 (Microsoft, 2023-09-11) - `mistral-7b` — Mistral 7B (Mistral AI, 2023-09-27) - `cohere-command` — Cohere Command (Cohere, 2023-09-29) - `ernie-4` — ERNIE 4.0 (Baidu, 2023-10) - `deepseek-coder` — DeepSeek-Coder (DeepSeek AI, 2023-11) - `yi-34b` — Yi-34B (01.AI, 2023-11) - `grok-1` — Grok-1 (xAI, 2023-11-04) - `gpt-4-turbo` — GPT-4 Turbo (OpenAI, 2023-11-06) - `claude-2-1` — Claude 2.1 (Anthropic, 2023-11-21) - `deepseek-llm-67b` — DeepSeek LLM 67B (DeepSeek AI, 2023-11-29) - `qwen-72b` — Qwen 1.8B / 14B / 72B (Alibaba, 2023-12) - `phi-2` — Phi-2 (Microsoft, 2023-12-12) - `glm-4` — GLM-4 (Z.AI (Zhipu), 2024-01) - `qwen-1-5` — Qwen 1.5 (Alibaba, 2024-02) - `deepseek-math` — DeepSeekMath 7B (DeepSeek AI, 2024-02) - `gemma-1` — Gemma 1 (Google, 2024-02-21) - `mistral-large-1` — Mistral Large (Mistral AI, 2024-02-26) - `grok-1-5` — Grok-1.5 (xAI, 2024-03) - `kimi-k1` — Kimi K1 (Moonshot AI, 2024-03) - `claude-3-sonnet` — Claude 3 Sonnet (Anthropic, 2024-03-04) - `claude-3-opus` — Claude 3 Opus (Anthropic, 2024-03-04) - `command-r` — Cohere Command R (Cohere, 2024-03-11) - `claude-3-haiku` — Claude 3 Haiku (Anthropic, 2024-03-13) - `jamba` — AI21 Jamba v0.1 (AI21 Labs, 2024-03-28) - `command-r-plus` — Cohere Command R+ (Cohere, 2024-04-04) - `llama-3` — Llama 3 (Meta AI, 2024-04-18) - `phi-3` — Phi-3 (mini / small / medium) (Microsoft, 2024-04-23) - `yi-1-5` — Yi-1.5 (01.AI, 2024-05-13) - `aya-23` — Cohere Aya 23 (Cohere For AI, 2024-05-23) - `codestral-22b` — Codestral 22B (Mistral AI, 2024-05-29) - `qwen-2` — Qwen2 (Alibaba, 2024-06) - `claude-3-5-sonnet` — Claude 3.5 Sonnet (Anthropic, 2024-06-20) - `gemma-2` — Gemma 2 (Google, 2024-06-27) - `mistral-nemo-12b` — Mistral NeMo 12B (Mistral AI / NVIDIA, 2024-07-18) - `llama-3-1` — Llama 3.1 (Meta AI, 2024-07-23) - `mistral-large-2` — Mistral Large 2 (Mistral AI, 2024-07-24) - `grok-2` — Grok-2 (xAI, 2024-08) - `falcon-mamba-7b` — Falcon Mamba 7B (TII, 2024-08-12) - `jamba-1-5` — AI21 Jamba 1.5 (AI21 Labs, 2024-08-22) - `qwen-2-5` — Qwen2.5 (Alibaba, 2024-09) - `llama-3-2-edge` — Llama 3.2 (1B / 3B edge) (Meta AI, 2024-09-25) - `granite-3-0` — IBM Granite 3.0 (IBM, 2024-10-21) - `aya-expanse` — Cohere Aya Expanse (Cohere For AI, 2024-10-23) - `claude-3-5-haiku` — Claude 3.5 Haiku (Anthropic, 2024-11-04) - `llama-3-3-70b` — Llama 3.3 70B (Meta AI, 2024-12-06) - `phi-4` — Phi-4 (Microsoft, 2024-12-12) - `falcon-3` — Falcon 3 family (TII, 2024-12-17) - `granite-3-1` — IBM Granite 3.1 (IBM, 2025-01-12) - `mistral-small-3` — Mistral Small 3 (Mistral AI, 2025-01-30) - `gpt-4-5` — GPT-4.5 (OpenAI, 2025-02-27) - `command-a` — Cohere Command A (Cohere, 2025-03-11) - `glm-4-32b-0414` — GLM-4-32B-0414 (Z.AI (Zhipu), 2025-04) - `granite-3-3` — IBM Granite 3.3 (IBM, 2025-04-16) - `jamba-2` — AI21 Jamba 2 (AI21 Labs, 2026-01-08) - `lfm2-5-2-6b` — LFM2.5-2.6B (Liquid AI, 2026-08-04) ### Mixture-of-Experts (`mixture_of_experts`) Sparse architectures that route each token to a small subset of expert sub-networks. Lower per-token compute, higher total parameter count, and meaningfully different infrastructure profile (interconnect-heavy). _53 models._ - `switch-transformer` — Switch Transformer (Google, 2021-01-11) - `glam` — GLaM (Google, 2021-12-09) - `st-moe` — ST-MoE (Google, 2022-02-17) - `mixtral-8x7b` — Mixtral 8x7B (Mistral AI, 2023-12-11) - `dbrx` — DBRX (Databricks, 2024-03-27) - `mixtral-8x22b` — Mixtral 8x22B (Mistral AI, 2024-04-10) - `snowflake-arctic` — Snowflake Arctic (Snowflake, 2024-04-24) - `deepseek-v2` — DeepSeek-V2 (DeepSeek AI, 2024-05-06) - `deepseek-coder-v2` — DeepSeek-Coder-V2 (DeepSeek AI, 2024-06) - `phi-3-5-moe` — Phi-3.5 MoE (Microsoft, 2024-08-21) - `hunyuan-large` — Hunyuan-Large (Tencent, 2024-11) - `deepseek-v3` — DeepSeek-V3 (DeepSeek AI, 2024-12-26) - `minimax-text-01` — MiniMax-Text-01 (MiniMax, 2025-01-15) - `hunyuan-turbos` — Hunyuan-TurboS (Tencent, 2025-03) - `deepseek-v3-0324` — DeepSeek-V3-0324 (DeepSeek AI, 2025-03-24) - `qwen-3` — Qwen3 (235B-A22B / 30B-A3B) (Alibaba, 2025-04) - `llama-4-scout` — Llama 4 Scout (Meta AI, 2025-04-05) - `llama-4-maverick` — Llama 4 Maverick (Meta AI, 2025-04-05) - `kimi-k2` — Kimi K2 (Moonshot AI, 2025-07) - `gpt-oss-120b` — gpt-oss-120b (OpenAI, 2025-08-05) - `gpt-oss-20b` — gpt-oss-20b (OpenAI, 2025-08-05) - `granite-4-0` — IBM Granite 4.0 (IBM, 2025-10-02) - `deepseek-v3-2` — DeepSeek-V3.2 (DeepSeek AI, 2025-12) - `hunyuan-2` — Hunyuan 2.0 (Tencent, 2025-12) - `mistral-large-3` — Mistral Large 3 (Mistral AI, 2025-12-08) - `glm-5` — GLM-5 (Z.AI (Zhipu), 2026-02) - `mistral-small-4` — Mistral Small 4 (Mistral AI, 2026-03-16) - `kimi-k2-6` — Kimi K2.6 (Moonshot AI, 2026-04-20) - `deepseek-v4-pro` — DeepSeek-V4 Pro (DeepSeek AI, 2026-04-24) - `deepseek-v4-flash` — DeepSeek-V4 Flash (DeepSeek AI, 2026-04-24) - `mimo-v2-5-pro` — MiMo V2.5 Pro (Xiaomi, 2026-04-27) - `laguna-m-1` — Laguna M.1 (Poolside, 2026-04-28) - `laguna-xs-2` — Laguna XS.2 (Poolside, 2026-04-28) - `emo-1b14b` — EMO (Emergent Mixture-of-Experts) (Allen Institute for AI, 2026-05-08) - `cohere-command-a-plus` — Command A+ (Cohere, 2026-05-20) - `hy-mt2-30b-a3b` — Hunyuan Hy-MT2-30B-A3B (Tencent, 2026-05-21) - `mellum2` — Mellum2 (JetBrains, 2026-06-01) - `north-mini-code` — North Mini Code (Cohere, 2026-06-09) - `diffusiongemma` — DiffusionGemma (Google + DeepMind, 2026-06-10) - `minimax-m3` — MiniMax-M3 (MiniMax, 2026-06-12) - `glm-5-2` — GLM-5.2 (Z.AI (Zhipu), 2026-06-16) - `longcat-2-0` — LongCat-2.0 (Meituan, 2026-06-30) - `nemotron-labs-twotower` — Nemotron-Labs-TwoTower (NVIDIA, 2026-07-01) - `hy3` — Hy3 (Tencent, 2026-07-06) - `inkling` — Inkling (Thinking Machines Lab, 2026-07-15) - `kimi-k3` — Kimi K3 (Moonshot AI, 2026-07-16) - `inkling-small` — Inkling-Small (Thinking Machines Lab, 2026-07-30) - `qwen3-8-max` — Qwen3.8-Max (Alibaba, 2026-08-03) - `glm-5-3` — GLM-5.3 (Z.AI (Zhipu), 2026-08-14) - `ornith-1-5-397b` — Ornith-1.5 (Ornith AI, 2026-08-19) - `thomson-1-0-small` — Thomson-1.0-Small (Thomson Reuters, 2026-08-24) - `qwen3-8-flash-next` — Qwen3.8-Flash-Next (Alibaba, 2026-08-26) - `hy4-preview` — Tencent Hy4 Preview (Tencent, 2026-08-28) ### Multimodal (`multimodal`) Models trained natively on text plus image, audio, or video. The defining shift is unified representation across modalities, not a vision adapter bolted onto a text-only model. _46 models._ - `flamingo` — Flamingo (DeepMind, 2022-04-29) - `kosmos-1` — Kosmos-1 (Microsoft, 2023-02-27) - `kosmos-2` — Kosmos-2 (Microsoft, 2023-06-26) - `gpt-4v` — GPT-4V (Vision) (OpenAI, 2023-09-25) - `gemini-1` — Gemini 1.0 (Google + DeepMind, 2023-12-06) - `gemini-1-5` — Gemini 1.5 (Pro / Flash) (Google + DeepMind, 2024-02-15) - `gpt-4o` — GPT-4o (OpenAI, 2024-05-13) - `falcon-2-11b` — Falcon 2 11B (+ VLM) (TII, 2024-05-13) - `gpt-4o-mini` — GPT-4o mini (OpenAI, 2024-07-18) - `pixtral-12b` — Pixtral 12B (Mistral AI, 2024-09-17) - `llama-3-2-vision` — Llama 3.2 Vision (11B / 90B) (Meta AI, 2024-09-25) - `pixtral-large` — Pixtral Large (124B) (Mistral AI, 2024-11-18) - `gemini-2` — Gemini 2.0 (Flash / Pro / Flash-Lite) (Google + DeepMind, 2024-12-11) - `qwen-2-5-vl` — Qwen2.5-VL (Alibaba, 2025-01) - `minimax-vl-01` — MiniMax-VL-01 (MiniMax, 2025-01-15) - `phi-4-multimodal` — Phi-4-multimodal / Phi-4-mini (Microsoft, 2025-02-26) - `granite-3-2` — IBM Granite 3.2 (incl. Granite Vision 3.2 2B) (IBM, 2025-02-26) - `gemma-3` — Gemma 3 (Google, 2025-03-12) - `ernie-4-5` — ERNIE 4.5 (Baidu, 2025-03-16) - `doubao-1-5` — Doubao 1.5 (Deep Thinking) (ByteDance, 2025-04) - `gpt-4-1` — GPT-4.1 (OpenAI, 2025-04-14) - `gpt-image-1` — gpt-image-1 (OpenAI, 2025-04-23) - `voxtral` — Voxtral 24B / 3B (Mistral AI, 2025-07-15) - `doubao-1-6-vision` — Doubao 1.6-Vision (ByteDance, 2025-09) - `ernie-5` — ERNIE 5.0 (Baidu, 2026-01-22) - `qwen-3-5` — Qwen3.5 (Alibaba, 2026-02-24) - `gemma-4` — Gemma 4 (Google, 2026-03-31) - `qwen-3-6-plus` — Qwen3.6-Plus (Alibaba, 2026-04-02) - `qwen-3-6-35b-a3b` — Qwen3.6-35B-A3B (Alibaba, 2026-04-16) - `nemotron-3-nano-omni` — Nemotron 3 Nano Omni (NVIDIA, 2026-04-28) - `inworld-realtime-tts-2` — Inworld Realtime TTS-2 (Inworld, 2026-05-05) - `gpt-realtime-2` — GPT-Realtime-2 (OpenAI, 2026-05-07) - `gpt-realtime-translate` — GPT-Realtime-Translate (OpenAI, 2026-05-07) - `gpt-realtime-whisper` — GPT-Realtime-Whisper (OpenAI, 2026-05-07) - `minicpm-v-4-6` — MiniCPM-V 4.6 1.3B (OpenBMB, 2026-05-11) - `perceptron-mk1` — Perceptron Mk1 (Perceptron, 2026-05-12) - `sana-wm` — SANA-WM (NVIDIA, 2026-05-15) - `gemini-3-5-flash` — Gemini 3.5 Flash (Google + DeepMind, 2026-05-19) - `gemini-omni-flash` — Gemini Omni Flash (Google + DeepMind, 2026-05-19) - `fara-1-5-27b` — Fara1.5-27B (Microsoft, 2026-05-22) - `cosmos-3-nano` — Cosmos 3 Nano (NVIDIA, 2026-06-01) - `holo-3-1` — Holo3.1 (H Company, 2026-06-02) - `gemini-3-6-flash` — Gemini 3.6 Flash (Google + DeepMind, 2026-07-21) - `muse-glimmer-30b` — Muse Glimmer (Meta, 2026-08-10) - `gemini-3-7-flash` — Gemini 3.7 Flash (Google, 2026-08-13) - `glm-5-3-flash` — GLM-5.3-Flash (Z.AI (Zhipu), 2026-08-26) ### Reasoning (`reasoning`) Models whose defining architectural shift is test-time compute — large-scale RL on chain-of-thought to produce long internal reasoning before a final answer. Often called "thinking models" or "extended-thinking models." _66 models._ - `o1-preview` — OpenAI o1-preview (OpenAI, 2024-09-12) - `o1-mini` — OpenAI o1-mini (OpenAI, 2024-09-12) - `qwq-32b-preview` — QwQ-32B-Preview (Alibaba, 2024-11) - `o1` — OpenAI o1 (OpenAI, 2024-12-05) - `o1-pro` — OpenAI o1-pro (OpenAI, 2024-12-05) - `deepseek-r1` — DeepSeek-R1 (DeepSeek AI, 2025-01-20) - `kimi-k1-5` — Kimi K1.5 (Moonshot AI, 2025-01-20) - `deepseek-r1-distill` — DeepSeek-R1-Distill (Qwen / Llama family) (DeepSeek AI, 2025-01-22) - `o3-mini` — OpenAI o3-mini (OpenAI, 2025-01-31) - `grok-3` — Grok-3 (incl. Think / Big Brain) (xAI, 2025-02-18) - `claude-3-7-sonnet` — Claude 3.7 Sonnet (Anthropic, 2025-02-24) - `ernie-x1` — ERNIE X1 (Baidu, 2025-03-16) - `hunyuan-t1` — Hunyuan T1 (Tencent, 2025-03-21) - `gemini-2-5-pro` — Gemini 2.5 Pro (Google + DeepMind, 2025-03-25) - `glm-z1` — GLM-Z1-32B-0414 (Z.AI (Zhipu), 2025-04) - `o3` — OpenAI o3 (OpenAI, 2025-04-16) - `o4-mini` — OpenAI o4-mini (OpenAI, 2025-04-16) - `phi-4-reasoning` — Phi-4-reasoning / mini-reasoning (Microsoft, 2025-04-30) - `claude-opus-4` — Claude Opus 4 (Anthropic, 2025-05-22) - `claude-sonnet-4` — Claude Sonnet 4 (Anthropic, 2025-05-22) - `o3-pro` — OpenAI o3-pro (OpenAI, 2025-06-10) - `magistral` — Magistral (Mistral AI, 2025-06-10) - `minimax-m1` — MiniMax-M1 (MiniMax, 2025-06-16) - `qwen-3-thinking-2507` — Qwen3-235B-A22B-Thinking-2507 (Alibaba, 2025-07) - `grok-4` — Grok 4 (xAI, 2025-07-09) - `claude-opus-4-1` — Claude Opus 4.1 (Anthropic, 2025-08-05) - `gpt-5` — GPT-5 (OpenAI, 2025-08-07) - `gpt-5-codex` — GPT-5-Codex (OpenAI, 2025-09-15) - `claude-sonnet-4-5` — Claude Sonnet 4.5 (Anthropic, 2025-09-29) - `doubao-1-6` — Doubao 1.6 (ByteDance, 2025-10) - `claude-haiku-4-5` — Claude Haiku 4.5 (Anthropic, 2025-10-15) - `gpt-5-1` — GPT-5.1 (Instant + Thinking + Pro) (OpenAI, 2025-11-13) - `gemini-3-pro` — Gemini 3 Pro (Google + DeepMind, 2025-11-18) - `claude-opus-4-5` — Claude Opus 4.5 (Anthropic, 2025-11-24) - `gpt-5-2` — GPT-5.2 (OpenAI, 2025-12-11) - `kimi-k2-5` — Kimi K2.5 (Moonshot AI, 2026-01-26) - `claude-opus-4-6` — Claude Opus 4.6 (Anthropic, 2026-02-05) - `claude-sonnet-4-6` — Claude Sonnet 4.6 (Anthropic, 2026-02-17) - `grok-4-20` — Grok 4.20 (xAI, 2026-02-17) - `gemini-3-1-pro` — Gemini 3.1 Pro (Google + DeepMind, 2026-02-19) - `gpt-5-3-instant` — GPT-5.3 Instant (OpenAI, 2026-03-03) - `gpt-5-4` — GPT-5.4 (Thinking + Pro + mini + nano) (OpenAI, 2026-03-05) - `deepseek-r2` — DeepSeek-R2 (DeepSeek AI, 2026-04) - `claude-mythos` — Claude Mythos Preview (Anthropic, 2026-04-07) - `glm-5-1` — GLM-5.1 (Z.AI (Zhipu), 2026-04-08) - `claude-opus-4-7` — Claude Opus 4.7 (Anthropic, 2026-04-16) - `gpt-5-5` — GPT-5.5 (OpenAI, 2026-04-23) - `grok-4-3` — Grok 4.3 (xAI, 2026-04-30) - `gpt-5-5-instant` — GPT-5.5 Instant (OpenAI, 2026-05-05) - `zaya1-8b` — ZAYA1-8B (Zyphra, 2026-05-06) - `qwen-3-7-max` — Qwen3.7-Max (Alibaba, 2026-05-20) - `claude-opus-4-8` — Claude Opus 4.8 (Anthropic, 2026-05-28) - `claude-fable-5` — Claude Fable 5 (Anthropic, 2026-06-09) - `gpt-5-6-sol` — GPT-5.6 Sol (OpenAI, 2026-06-26) - `claude-sonnet-5` — Claude Sonnet 5 (Anthropic, 2026-06-30) - `leanstral-1-5` — Leanstral 1.5 (Mistral AI, 2026-06-30) - `grok-4-5` — Grok 4.5 (xAI, 2026-07-08) - `muse-spark-1-1` — Muse Spark 1.1 (Meta, 2026-07-09) - `claude-opus-5` — Claude Opus 5 (Anthropic, 2026-07-24) - `muse-spark-1-2` — Muse Spark 1.2 (Meta, 2026-08-05) - `gpt-5-6-cyber` — GPT-5.6-Cyber (OpenAI, 2026-08-10) - `grok-4-6` — Grok 4.6 (SpaceXAI, 2026-08-12) - `granite-4-2-30b` — IBM Granite 4.2 (IBM, 2026-08-25) - `claude-fable-5-1` — Claude Fable 5.1 (Anthropic, 2026-09-01) - `gpt-6-astra` — GPT-6 Astra (OpenAI, 2026-09-03) - `gemini-3-8-flash` — Gemini 3.8 Flash (Google, 2026-09-03) ## Architectural breakthroughs - **2014 — Seq2Seq with Attention.** Sutskever, Vinyals, and Le introduced the LSTM encoder-decoder for machine translation; Bahdanau, Cho, and Bengio added soft attention to fix the fixed-length context bottleneck. Together these papers are the direct ancestors of the transformer. (Branches: foundational, encoder_decoder) - **2017 — Transformer (Attention Is All You Need).** Vaswani et al. replaced recurrence and convolution with self-attention alone. Achieved 28.4 BLEU on WMT'14 EN-DE in 3.5 days on 8 GPUs and became the shared ancestor of every modern LLM branch. (Branches: encoder_only, encoder_decoder, decoder_only, mixture_of_experts, multimodal, reasoning) - **2018 — Masked Language Modeling Pretraining.** BERT (Devlin et al.) established bidirectional masked-language-modeling pretraining as the foundation of modern NLP understanding. Defined the encoder-only branch. (Branches: encoder_only) - **2019 — Text-to-Text Framing.** T5 (Raffel et al.) reframed every NLP task as text-to-text and established span-corruption denoising on the C4 corpus. Defined the modern encoder-decoder branch. (Branches: encoder_decoder) - **2020 — In-Context Learning at Scale.** GPT-3 (Brown et al.) demonstrated few-shot in-context learning as an emergent capability of scale. The 175B-parameter dense decoder became the scaling-laws moment and launched the OpenAI API. (Branches: decoder_only) - **2021 — Trillion-Parameter Sparse Routing.** Switch Transformer (Fedus, Zoph, Shazeer) pushed mixture-of-experts past one trillion parameters with simplified single-expert routing. 4-7x speedup over T5-XXL. Defined the MoE branch at scale. (Branches: mixture_of_experts) - **2022 — RLHF / InstructGPT.** Reinforcement Learning from Human Feedback applied to GPT-3 produced InstructGPT and then ChatGPT. Made dialogue-format chat tuning the industry default and turned LLMs into products. (Branches: decoder_only) - **2022 — Chinchilla Compute-Optimal Scaling.** Hoffmann et al. (DeepMind) established that tokens and parameters should scale equally for compute-optimal training. Outperformed Gopher 280B and GPT-3 175B on MMLU at 70B parameters and 1.3T tokens. Every subsequent training run respects this law. (Branches: decoder_only, mixture_of_experts, multimodal, reasoning) - **2022 — FlashAttention.** Dao et al. produced an IO-aware exact attention algorithm that reduced the memory footprint of attention from quadratic to linear in sequence length. Made long-context training and inference economically feasible at scale. (Branches: decoder_only, mixture_of_experts, multimodal, reasoning) - **2022 — Native Multimodal Training.** Flamingo (Alayrac et al., DeepMind) demonstrated few-shot vision-language learning by interleaving images, video, and text in a single sequence. Defined the multimodal branch and seeded Gemini's natively-multimodal design. (Branches: multimodal) - **2023 — Open MoE Goes Mainstream.** Mixtral 8x7B (Dec 2023) was the first widely-deployed open-weights mixture-of-experts. Apache 2.0 license; 47B total / 13B active. Triggered an open-source MoE wave (DBRX, Arctic, Jamba, DeepSeek-V2). (Branches: mixture_of_experts) - **2024 — Multi-head Latent Attention (MLA).** DeepSeek-V2 introduced MLA — a low-rank latent compression of the KV cache that reduced memory by 93% versus dense attention. Combined with DeepSeekMoE, this is the architectural innovation that enabled the V3/R1 cost shock in early 2025. (Branches: mixture_of_experts, reasoning) - **2024 — Test-Time Compute (the o1 moment).** OpenAI's o1-preview (September 2024) demonstrated that large-scale RL on chain-of-thought, with the model "thinking" before answering, produces capability gains comparable to a generation of pretraining scale-up. Defined the reasoning branch. (Branches: reasoning) - **2024 — Model Context Protocol (MCP).** Anthropic published the Model Context Protocol — an open spec for how LLMs connect to tools, data sources, and other models. Adopted by every major frontier vendor by mid-2025. Made the agentic stack interoperable. (Branches: decoder_only, multimodal, reasoning) - **2025 — Group Relative Policy Optimization (GRPO).** DeepSeek introduced GRPO in DeepSeek-Math and scaled it in DeepSeek-R1 to skip supervised fine-tuning entirely and train reasoning behavior via pure RL. Lower training cost than RLHF/PPO and competitive results. Now used by Qwen, Microsoft Phi reasoning, Hunyuan, GLM, and others. (Branches: reasoning) - **2025 — Agent-to-Agent (A2A) Handoff.** A2A protocol (Google + Anthropic + OpenAI cross-vendor work) standardized structured handoffs between agents. Enabled native multi-agent orchestration in Kimi K2.6, Grok 4.20, GLM-5.1. (Branches: reasoning) - **2025 — DeepSeek Sparse Attention (DSA).** DeepSeek-V3.2 introduced Sparse Attention — a per-head dynamic sparsity pattern that dropped attention compute by ~73% per token. First cross-vendor architectural pattern to propagate within the Chinese frontier itself; adopted by GLM-5 (Feb 2026) and reused in DeepSeek-V4. (Branches: mixture_of_experts, reasoning) - **2026 — Adaptive Thinking.** Claude Opus 4.6 (Feb 2026) introduced adaptive thinking — the model auto-decides reasoning depth per query rather than requiring an explicit "thinking mode" toggle. Replaces manual extended thinking. Now used by Mistral Small 4, Qwen3.6, Doubao 1.6. (Branches: reasoning) --- Raw YAML (lossless): . Compact JSON: . Canonical HTML: .