Model card
DeepSeek-Coder-V2-Instruct.
DeepSeekopen-sourceUnknown paramsMoE Transformer
236B MoE (21B active). Trained on 6T tokens. Matches/beats GPT-4o on code. June 2024.
§ 01 · Benchmarks
Every benchmark DeepSeek-Coder-V2-Instruct has a recorded score for.
| # | Benchmark | Area · Task | Metric | Value | Rank | Date | Source |
|---|---|---|---|---|---|---|---|
| 01 | Bugs2Fix | Computer Code · Bug Detection | accuracy | 75.3% | #3 | 2024-06-17 | source ↗ |
| 02 | CodeSearchNet | Computer Vision · Optical Character Recognition | bleu-4 | 22.8% | #3 | 2024-06-17 | source ↗ |
| 03 | CrossCodeEval | Computer Code · Code Completion | exact-match | 41.3% | #3 | 2024-06-17 | source ↗ |
| 04 | TransCoder (GeeksForGeeks) | Computer Code · Code Translation | computational-accuracy | 84.6% | #4 | 2024-06-17 | source ↗ |
| 05 | MBPP | Computer Code · Code Generation | pass@1 | 89.4% | #7 | 2024-06-17 | source ↗ |
| 06 | MBPP | Computer Code · Code Generation | pass@1 | 89.4% | #7 | — | source ↗ |
| 07 | HumanEval | Computer Code · Code Generation | pass@1 | 90.2% | #17 | 2024-06-17 | source ↗ |
| 08 | HumanEval | Computer Code · Code Generation | pass@1 | 90.2% | #17 | 2024-06-01 | source ↗ |
| 09 | LiveCodeBench | Computer Code · Code Generation | pass@1 | 43.4% | #23 | 2024-03-12 | source ↗ |
Rank column shows this model’s position vs all other models scored on the same benchmark + metric (competitors after the slash). #1 in red means current SOTA. Sorted by rank, then newest result.
§ 02 · Strengths by area
Where DeepSeek-Coder-V2-Instruct actually performs.
§ 03 · Papers
2 papers with results for DeepSeek-Coder-V2-Instruct.
- 2024-06-17· Computer Code· 6 results
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
- 2024-03-12· Computer Code· 1 result
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
§ 04 · Related models
Other DeepSeek models scored on Codesota.
§ 05 · Sources & freshness
Where these numbers come from.
arxiv
6
results
arxiv-2409.12186
1
result
shadow-page-humaneval
1
result
official-leaderboard
1
result
9 of 9 rows marked verified. · first result 2024-03-12, latest 2024-06-17.