Llama 4 Open Source 2025.4
The first MoE architecture multi-modal open source model, tens of millions of token contexts, challenging GPT-4o
π¦ What is Llama 4?
Llama 4 is a new generation of open source large model released by Meta on April 5, 2025. It adopts the hybrid expert (MoE) architecture for the first time and supports multi-modal processing of text, image, video, and audio. This is Meta's major breakthrough in the field of AI, marking that open source models have officially entered a new stage of direct competition with closed source models.
MoE Architecture
Hybrid expert architecture, dynamic routing, efficient reasoning
Multimodal native
Unified processing of text/image/video/audio
Very long context
Scout supports 10 million token contexts
π€ Llama 4 model family
Llama 4 Scout Long Text Expert
Designed for long text tasks such as document summarization and code base analysis, and can run on a single H100 GPU.
Llama 4 Maverick Ultimate version
The main flagship version, surpassing GPT-4o and Gemini 2.0 in tasks such as programming, mathematical reasoning, and multi-language processing.
Llama 4 Behemoth Coming soon
Ultimate version, 2 trillion parameters, STEM benchmarks surpassing GPT-4.5 and Claude Sonnet 3.7.
π Performance comparison
| Model | Parameter quantity | Context | Multimodal | Open Source |
|---|---|---|---|---|
| Llama 4 Maverick | 400 billion | 128K | Yes | Yes |
| GPT-4o | Unpublished | 128K | Yes | No |
| Gemini 2.0 | Unpublished | 1M | Yes | No |
| DeepSeek V3 | 685 billion | 128K | Yes | Yes |
π Quick start
Using Hugging Face
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "meta-llama/Llama-4-Maverick"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
inputs = tokenizer("Hello, Llama 4!", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0]))
Using Ollama
# δΈθ½½εΉΆθΏθ‘ Llama 4 ollama pull llama4-maverick ollama run llama4-maverick
π Related Resources
- Hugging Face: huggingface.co/meta-llama
- Official website: llama.meta.com
- GitHub: github.com/meta-llama/llama