Who leads the codesearchnet---ruby benchmark?

CodeTrans-MT-Base currently leads codesearchnet---ruby with a score of 15.26 on Smoothed Bleu 4.

What is the state-of-the-art score on codesearchnet---ruby?

The state-of-the-art result on codesearchnet---ruby is 15.26 (Smoothed Bleu 4), achieved by CodeTrans-MT-Base as of 2021.

How many models are tracked on codesearchnet---ruby?

Codesota tracks 7 models on codesearchnet---ruby.

When was the codesearchnet---ruby leaderboard last updated?

The codesearchnet---ruby leaderboard on Codesota includes results through 2021, with the earliest tracked result from 2020.

Codesota · Benchmark · codesearchnet---rubyHome/Leaderboards/Vision & Documents/Document OCR/codesearchnet---ruby

Unknown

codesearchnet---ruby.

Name: codesearchnet---ruby Benchmark Results
Creator: Unknown
Published: 2020-01-01
License: https://creativecommons.org/licenses/by/4.0/

codesearchnet---ruby is a state-of-the-art machine learning benchmark indexed on Codesota. This page tracks published model results, top scores per metric, and the SOTA timeline for codesearchnet---ruby.

Paper ↗Leaderboard ↓

§ 01 · SOTA history

Year over year.

§ 02 · Leaderboard

Results by metric.

Found a wrong score or missing run?

Use row edits to send a sourced correction into moderation.

Add / edit result ↗Report issue ↗

Smoothed Bleu 4

Smoothed Bleu 4 is the reported evaluation metric for codesearchnet---ruby. Codesota tracks published model scores on this metric so readers can compare state-of-the-art results across sources and model families.

Higher is better

Trust tiers for Smoothed Bleu 4verifiedpapervendorcommunityunverified

Rank	Model	Trust	Score	Year	Links	Fix
01	CodeTrans-MT-Base From paper: CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing	verified	15.26	2021	Paper ↗Code ↗	Looks wrong?
02	CodeBERT (MLM+RTD) From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	8.46	2020	Paper ↗Code ↗	Looks wrong?
03	CodeBERT (MLM) From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	7.95	2020	Paper ↗Code ↗	Looks wrong?
04	Transformer From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	7.87	2020	Paper ↗Code ↗	Looks wrong?
05	pre-train w/ code only From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	7.36	2020	Paper ↗Code ↗	Looks wrong?
06	RoBERTa From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	7.26	2020	Paper ↗Code ↗	Looks wrong?
07	seq2seq From paper: CodeBERT: A Pre-Trained Model for Programming and Natural Languages	verified	6.96	2020	Paper ↗Code ↗	Looks wrong?

§ 04 · Submit a result

Add to the leaderboard.

← Back to Document OCR