Llama 4 Meta Open Source 2025.4

The first MoE architecture multi-modal open source model, tens of millions of token contexts, challenging GPT-4o

2 trillion Behemoth parameters
10 million Scout context
128 Maverick Expert Count
#2 Arena Rankings

πŸ¦™ What is Llama 4?

Llama 4 is a new generation of open source large model released by Meta on April 5, 2025. It adopts the hybrid expert (MoE) architecture for the first time and supports multi-modal processing of text, image, video, and audio. This is Meta's major breakthrough in the field of AI, marking that open source models have officially entered a new stage of direct competition with closed source models.

Architecture innovation: Llama 4 shifts from "parameter size competition" to "architecture efficiency optimization". The MoE architecture allows the model to significantly reduce inference costs while maintaining strong capabilities. Maverick version inference cost is only half that of DeepSeek V3.

MoE Architecture

Hybrid expert architecture, dynamic routing, efficient reasoning

Multimodal native

Unified processing of text/image/video/audio

Very long context

Scout supports 10 million token contexts

πŸ€– Llama 4 model family

Llama 4 Scout Long Text Expert

Designed for long text tasks such as document summarization and code base analysis, and can run on a single H100 GPU.

109 billion
Total parameters
17 billion
Active parameters
10 million
Context
16
Number of experts

Llama 4 Behemoth Coming soon

Ultimate version, 2 trillion parameters, STEM benchmarks surpassing GPT-4.5 and Claude Sonnet 3.7.

2 trillion
Total parameters
STEM #1
Benchmark
Teacher Model
Positioning
2025 Q3
Expected release

πŸ“Š Performance comparison

Model Parameter quantity Context Multimodal Open Source
Llama 4 Maverick 400 billion 128K Yes Yes
GPT-4o Unpublished 128K Yes No
Gemini 2.0 Unpublished 1M Yes No
DeepSeek V3 685 billion 128K Yes Yes

πŸš€ Quick start

Using Hugging Face

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "meta-llama/Llama-4-Maverick"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

inputs = tokenizer("Hello, Llama 4!", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0]))

Using Ollama

# δΈ‹θ½½εΉΆθΏθ‘Œ Llama 4
ollama pull llama4-maverick
ollama run llama4-maverick

πŸ”— Related Resources