Code Generation2021python
Mostly Basic Python Problems
974 crowd-sourced Python programming problems suitable for beginners. Covers programming fundamentals and standard library.
Current State of the Art
Claude 3.5 Sonnet
Anthropic
89.2
pass@1
pass@1Primary
| # | Model | Score | Paper / Code | Date |
|---|---|---|---|---|
| 1 | Claude 3.5 SonnetAPI Anthropic | 89.2 | Dec 2025 | |
| 2 | GPT-4oAPI OpenAI | 87.8 | Dec 2025 |