Model card
Llama 3 (405B, Instruct).
MetaLarge language model1 current SOTA
Imported from Papers with Code dump
§ 02 · Benchmarks
Every benchmark Llama 3 (405B, Instruct) has a recorded score for.
| # | Benchmark | Area · Task | Metric | Value | Rank | Date | Source |
|---|---|---|---|---|---|---|---|
| 01 | HumanEval | Computer Code · Code Generation | pass-1 | 89.0% | #1 | — | source ↗ |
| 02 | HumanEval+ | Computer Code · Code Generation | pass-1 | 89.0% | #1 | — | source ↗ |
| 03 | MBPP | Computer Code · Code Generation | pass-1 | 78.8% | #3 | — | source ↗ |
| 04 | MBPP+ | Computer Code · Code Generation | pass-1 | 78.8% | #4 | — | source ↗ |
| 05 | GSM8K | Reasoning · Mathematical Reasoning | accuracy | 96.8% | #18 | — | source ↗ |
| 06 | LibriSpeech | Speech · Speech Recognition | wer | 3.1% | #28 | — | source ↗ |
| 07 | MATH | Reasoning · Mathematical Reasoning | accuracy | 73.8% | #31 | — | source ↗ |
| 08 | MMLU | Reasoning · Commonsense Reasoning | accuracy | 87.3% | #31 | — | source ↗ |
| 09 | MMLU-Pro | Reasoning · Commonsense Reasoning | accuracy | 73.3% | #55 | 2024-07-31 | source ↗ |
| 10 | MMLU-Pro | Reasoning · Commonsense Reasoning | accuracy | 73.3% | #55 | — | source ↗ |
| 11 | GPQA Diamond | Reasoning · Multi-step Reasoning | accuracy | 51.1% | #61 | — | source ↗ |
Rank column shows this model’s position vs all other models scored on the same benchmark + metric (competitors after the slash). #1 in red means current SOTA. Sorted by rank, then newest result.
§ 03 · Strengths by area
Where Llama 3 (405B, Instruct) actually performs.
§ 04 · Papers
1 paper with results for Llama 3 (405B, Instruct).
- 2024-07-31· 10 results
The Llama 3 Herd of Models
§ 05 · Related models
Other Meta models scored on Codesota.
§ 06 · Sources & freshness
Where these numbers come from.
pwc-dump
10
results
paperswithcode
1
result
0 of 11 rows marked verified.