Large Language Model

What is LLM

A Large Language Model (LLM) is a language model built from a deep neural network containing tens or even hundreds of billions of parameters. Based on the Transformer architecture, it is trained via self-supervised learning on massive amounts of unlabeled text data and possesses powerful natural-language understanding and generation capabilities. Simply put, after reading huge amounts of text on the internet, an LLM learns to "predict the next word"—and from this seemingly simple ability, complex intelligence such as reasoning, translation, code writing, and conversation emerges.

Representative models: GPT-4/5 (OpenAI), Claude (Anthropic), Gemini (Google), DeepSeek, and Llama (Meta).

Transformer Architecture

Transformer is an architecture proposed by Google in 2017 and is the cornerstone of all modern LLMs. At its core is the "self-attention mechanism"—when processing a word, the model can simultaneously attend to all other words in the sentence and understand their relationships. Compared to earlier RNN/LSTM models that could only read sequentially from left to right, Transformer can process in parallel and capture long-range dependencies, which is the fundamental reason it can "understand context".

# Core idea of Transformer (simplified)
# Self-attention: every word "looks at" all other words and computes association weights
attention("bank") = 0.7 * river + 0.3 * money  # disambiguate meaning from context

# Training objective: predict the next word
Input: "The weather today is really"
Target: predict "nice"

# Parameter scale determines capability
GPT-3:  175 billion parameters
GPT-4:  ~1.8 trillion parameters (MoE architecture)

Parameters & Training

The "large" in LLM shows up in parameter count: from hundreds of millions in early models, to 175 billion in GPT-3, to the trillion level in GPT-4. Parameters are the "knowledge weights" a model learns during training. Training happens in two stages: pretraining (learning general language ability from massive text) + fine-tuning (aligning with human intent using instruction data, e.g. RLHF). The more parameters and the broader the data, the stronger the capabilities that emerge—known as "emergent abilities".

Training costs are extremely high: GPT-4 training cost over $100 million and required thousands of GPUs running for months.

How Developers Use LLMs

Three ways developers use LLMs: 1) Call cloud APIs (OpenAI/Anthropic/DeepSeek)—fastest to get started, pay-as-you-go; 2) Self-host open-source models (Llama, Qwen, DeepSeek)—data stays local, controllable; 3) Fine-tune open-source models for vertical scenarios. In AI coding tools, the LLM is the Agent's "brain", and tool calling (Function Calling / MCP) is its "hands and feet".

# 1. Call a cloud API (Python example)
from anthropic import Anthropic
client = Anthropic()
response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Write a quicksort in Python"}]
)

# 2. Self-host an open-source model
# Install Ollama → ollama pull deepseek-r1 → ollama run deepseek-r1

# 3. Connect via tools like Continue/Aider—no code needed

Selection advice: use Claude/GPT for general tasks; DeepSeek for cost sensitivity; local Ollama for privacy sensitivity.