Codesota · Benchmark · CrossCodeEvalHome/Leaderboards/Code & Software Engineering/Code Completion/CrossCodeEval
Unknown

CrossCodeEval.

Multilingual benchmark (Python, TypeScript, Java, C#) for cross-file code completion requiring understanding of cross-file context. 1000 examples per language from GitHub repos. Primary metric is Exact Match.

Paper ↗Leaderboard ↓
§ 01 · SOTA history

Year over year.

§ 02 · Leaderboard

Results by metric.

Found a wrong score or missing run?
Use row edits to send a sourced correction into moderation.
Add / edit result ↗Report issue ↗

Exact Match

Exact Match is the reported evaluation metric for CrossCodeEval. Codesota tracks published model scores on this metric so readers can compare state-of-the-art results across sources and model families.

Higher is better

Trust tiers for Exact Matchverifiedpapervendorcommunityunverified
RankModelTrustScoreYearLinksFix
01Claude Sonnet 4
Anthropic model card. Python EM with BM25 retrieval.
verified44.52026Source ↗Looks wrong?
02Qwen2.5-Coder 32B
Qwen2.5-Coder paper. Python EM with BM25 retrieval.
verified43.72024Paper ↗Code ↗Looks wrong?
03DeepSeek-Coder-V2-Instruct
DeepSeek-Coder-V2 paper. Python EM with BM25 retrieval.
verified41.32024Paper ↗Code ↗Looks wrong?
04GPT-4o
CrossCodeEval paper Table 3. Python EM with BM25 cross-file context retrieval.
verified38.22023Paper ↗Code ↗Looks wrong?
05Codestral 22B
Reported in Codestral blog. Python EM with BM25 retrieval.
verified35.62024Source ↗Looks wrong?
06StarCoder2 15B
StarCoder2 paper. Python EM with BM25 retrieval.
verified32.12024Paper ↗Code ↗Looks wrong?
§ 04 · Submit a result

Add to the leaderboard.

← Back to Code Completion