Knowledge Base
Building knowledge systems? Evaluate graph completion, relation extraction, and entity linking performance.
Knowledge base construction and reasoning spans knowledge graph completion, entity linking, relation extraction, and question answering over structured knowledge. LLMs have transformed the field, enabling zero-shot KG construction while raising questions about whether explicit knowledge graphs remain necessary.
State of the Field (2025)
- Knowledge graph completion models (RotatE, CompoundE, SimKGC) achieve MRR >0.50 on FB15k-237 and WN18RR, with text-aware methods leveraging LLM embeddings pushing boundaries further
- LLM-based knowledge graph construction enables zero-shot entity and relation extraction from unstructured text, reducing manual ontology engineering effort by 60-80%
- Question answering over knowledge bases (KBQA) increasingly uses LLMs as semantic parsers, with GPT-4 and similar models achieving 70%+ accuracy on WebQuestionsSP and ComplexWebQuestions
- Hybrid RAG-KG systems combine retrieval augmented generation with structured knowledge graphs, improving factual accuracy by 15-25% over pure RAG on domain-specific benchmarks
Quick Recommendations
Knowledge graph completion (link prediction)
SimKGC or RotatE
SimKGC leverages pre-trained language model embeddings for text-aware completion. RotatE provides strong geometric embedding baseline with interpretable relation patterns.
Entity linking and disambiguation
GENRE or ReFinED
GENRE uses autoregressive entity name generation for end-to-end linking. ReFinED offers fast, scalable entity disambiguation for production pipelines processing millions of documents.
Relation extraction from text
LLM-based extraction (GPT-4/Claude) with schema constraints
Zero-shot extraction with structured output formatting eliminates training data requirements. Schema constraints ensure ontology compliance. Fine-tuned smaller models for high-volume processing.
Question answering over enterprise knowledge graphs
LLM semantic parsing + SPARQL/Cypher execution
LLMs translate natural language to graph queries with 70%+ accuracy. Execution against the actual KG ensures factual grounding. Combine with RAG fallback for questions outside KG coverage.
Tasks & Benchmarks
Show all datasets and SOTA results
Entity Linking
No datasets indexed yet. Contribute on GitHub
Knowledge Graph Completion
No datasets indexed yet. Contribute on GitHub
Relation Extraction
No datasets indexed yet. Contribute on GitHub
Honest Takes
Knowledge graphs are not dead, but their role changed
LLMs store enormous implicit knowledge, making simple fact lookup KGs redundant. But for enterprise applications requiring audit trails, provenance tracking, and structured reasoning over domain-specific relationships, explicit KGs remain essential. The future is hybrid: LLMs for flexibility, KGs for reliability.
Zero-shot KG construction is good enough to start
You no longer need months of ontology engineering to build a useful knowledge graph. LLM-based extraction gets you 70-80% of the way there in days. Clean up iteratively rather than designing the perfect schema upfront. The old waterfall approach to KG construction is dead.
RAG without structure is hitting a ceiling
Pure vector retrieval struggles with multi-hop reasoning and relational queries. Adding a knowledge graph layer to your RAG pipeline improves factual accuracy by 15-25% on complex questions. If your users ask questions like 'which suppliers serve both regions,' you need structured knowledge.