What Happened to Papers With Code?
For seven years, Papers with Code was the definitive resource for ML research. Then Meta shut it down without warning. Here's what was lost, what remains, and why CodeSOTA exists.
Timeline
What Papers With Code Was
Papers with Code was "the Wikipedia of machine learning research." Founded by Robert Stojnic (one of Wikipedia's original developers) and Ross Taylor, it solved a fundamental problem: connecting research papers with working code implementations.
Before Papers with Code, finding an implementation of a paper meant:
- Searching GitHub with random keyword combinations
- Hoping the authors released code (most didn't)
- Finding outdated implementations in deprecated frameworks
- Spending days reproducing papers from scratch
Papers with Code changed this. Every paper had linked implementations (official and community), benchmark results, dataset information, and method explanations. The wiki model let anyone contribute, building a comprehensive knowledge base that became essential infrastructure.
The Benchmark System
The most valuable contribution was the benchmark tracking system. Every benchmark was organized as a <Task, Dataset, Metric> tuple:
Content was organized into 16 research areas covering everything from Computer Vision to Robotics. Each area contained tasks, sub-tasks, and specific benchmarks with leaderboards showing:
- Model rankings with metric scores
- Links to papers and code implementations
- Visual timelines showing SOTA progression over years
- Official vs community implementation badges
Why It Mattered
For Researchers
Establish baselines instantly. Know what SOTA looks like before starting a project. Find existing implementations to build on rather than reimplementing from scratch.
For Engineers
Find working code for papers. Compare frameworks (PyTorch vs TensorFlow implementations). Skip the "reproducing papers" phase and go straight to production.
For the Field
Research shows papers with linked code get ~20% higher citation rates. Papers with Code made reproducibility the norm, not the exception.
For Decision Makers
Clear benchmark comparisons across models. Understand what's actually state-of-the-art vs marketing claims. Make informed technology choices.
"Papers with Code was bread and butter for Research Engineers and Scientists." - ML community sentiment
The Shutdown
On July 24-25, 2025, Meta "sunsetted" Papers with Code without prior notice. Users reported "Bad Gateway 502" errors and garbled text. GitHub issues went unanswered. The site now redirects to Hugging Face's "Trending Papers" feature.
What Was Lost
- - Comprehensive SOTA leaderboards across 9,327 benchmarks
- - Paper-to-code linkages for 79,817 papers
- - Method explanations and connections
- - The unified research workflow millions relied on
The irony: when Meta acquired Papers with Code in 2019, they promised it would "remain a neutral, open and free resource." That promise lasted five and a half years.
The Vacuum Left Behind
Hugging Face's successor provides paper discovery and code links, but lacks the comprehensive SOTA leaderboards that defined Papers with Code. Hugging Face focuses on model-centric leaderboards (like the Open LLM Leaderboard), not paper-centric benchmark tracking.
| Feature | Papers With Code | Hugging Face |
|---|---|---|
| Paper discovery | Yes | Yes |
| Code links | Yes | Yes |
| SOTA leaderboards | 9,327 benchmarks | Limited |
| Dataset registry | 5,628 datasets | Different focus |
| Method explanations | Yes | No |
| Task hierarchy | 16 areas, nested | No |
The current alternatives (Semantic Scholar, Connected Papers, Kaggle) each cover parts of what Papers with Code did, but none replicate the unified paper-code-benchmark-dataset-method linkages. The integrated experience is gone.
What Was Saved
The community moved quickly to preserve data:
- GitHub archives - Historical JSON dumps at
paperswithcode/paperswithcode-data - Hugging Face datasets -
pwc-archivewith papers, abstracts, evaluation tables - ORKG - Imported 2021 benchmark data into Open Research Knowledge Graph
The data exists. The integrated experience doesn't.
Why CodeSOTA Exists
CodeSOTA is building what Papers with Code provided: verified benchmarks, practical recommendations, and runnable code. We're starting focused (OCR and document AI) rather than trying to index everything at once.
Our approach is different in key ways:
- Verified results - We don't just aggregate claims. We run the benchmarks ourselves where possible.
- Practical focus - Not just leaderboards. Which model for your use case? What are the real tradeoffs?
- Open data - All benchmark data available as JSON. Build on it, cite it, contribute to it.
- Independent - Not owned by a big tech company that might shut it down.
The Lesson
Papers with Code's shutdown underscores a risk the academic community faces: critical research infrastructure controlled by commercial entities can disappear without warning. Meta had every right to shut it down. They owned it. That's the problem.
The ML community needs benchmark tracking infrastructure that isn't dependent on corporate goodwill. That's what we're building.