Model card
Gemini Ultra.
Google DeepMindproprietaryUnknown paramsTransformer (decoder-only)
Largest Gemini 1.0 model. Released December 2023.
§ 01 · Benchmarks
Every benchmark Gemini Ultra has a recorded score for.
| # | Benchmark | Area · Task | Metric | Value | Rank | Date | Source |
|---|---|---|---|---|---|---|---|
| 01 | SNLI | Natural Language Processing · Natural Language Inference | accuracy | 91.9% | #3 | 2023-12-19 | source ↗ |
| 02 | SuperGLUE | Natural Language Processing · Text Classification | average-score | 90.0% | #4 | 2023-12-19 | source ↗ |
| 03 | GSM8K | Reasoning · Mathematical Reasoning | accuracy | 94.4% | #22 | 2024-02-01 | source ↗ |
Rank column shows this model’s position vs all other models scored on the same benchmark + metric (competitors after the slash). #1 in red means current SOTA. Sorted by rank, then newest result.
§ 02 · Strengths by area
Where Gemini Ultra actually performs.
§ 03 · Papers
1 paper with results for Gemini Ultra.
- 2023-12-19· Natural Language Processing· 2 results
Gemini: A Family of Highly Capable Multimodal Models
§ 04 · Related models
Other Google DeepMind models scored on Codesota.
§ 05 · Sources & freshness
Where these numbers come from.
arxiv
2
results
gsm8k-shadow-page
1
result
2 of 3 rows marked verified. · first result 2023-12-19, latest 2024-02-01.