Who leads the codesearchnet---python benchmark?

CodeTrans-MT-Base currently leads codesearchnet---python with a score of 20.39 on Smoothed Bleu 4.

What is the state-of-the-art score on codesearchnet---python?

The state-of-the-art result on codesearchnet---python is 20.39 (Smoothed Bleu 4), achieved by CodeTrans-MT-Base as of 2021.

How many models are tracked on codesearchnet---python?

Codesota tracks 7 models on codesearchnet---python.

When was the codesearchnet---python leaderboard last updated?

The codesearchnet---python leaderboard on Codesota includes results through 2021, with the earliest tracked result from 2020.

Codesota · Benchmark · codesearchnet---pythonHome/Leaderboards/Vision & Documents/Document OCR/codesearchnet---python

Unknown

codesearchnet---python.

Name: codesearchnet---python Benchmark Results
Creator: Unknown
Published: 2020-01-01
License: https://creativecommons.org/licenses/by/4.0/

codesearchnet---python is a state-of-the-art machine learning benchmark indexed on Codesota. This page tracks published model results, top scores per metric, and the SOTA timeline for codesearchnet---python.

Paper ↗Leaderboard ↓

§ 01 · SOTA history

Year over year.

§ 02 · Leaderboard

Results by metric.

Found a wrong score or missing run?

Use row edits to send a sourced correction into moderation.

Add / edit result ↗Report issue ↗

Smoothed Bleu 4

Smoothed Bleu 4 is the reported evaluation metric for codesearchnet---python. Codesota tracks published model scores on this metric so readers can compare state-of-the-art results across sources and model families.

Higher is better

Trust tiers for Smoothed Bleu 4verifiedpapervendorcommunityunverified

Rank	Model	Trust	Score	Year	Links	Fix
01	CodeTrans-MT-Base From paper: CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing	verified	20.39	2021	Paper ↗Code ↗	Looks wrong?
02	CodeBERT (MLM) From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	15.48	2020	Paper ↗Code ↗	Looks wrong?
03	CodeBERT (MLM+RTD) From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	15.41	2020	Paper ↗Code ↗	Looks wrong?
04	pre-train w/ code only From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	15.12	2020	Paper ↗Code ↗	Looks wrong?
05	RoBERTa From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	14.92	2020	Paper ↗Code ↗	Looks wrong?
06	Transformer From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	13.44	2020	Paper ↗Code ↗	Looks wrong?
07	seq2seq From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	13.04	2020	Paper ↗Code ↗	Looks wrong?

§ 04 · Submit a result

Add to the leaderboard.

← Back to Document OCR