IntelligentBench is an evaluation suite introduced in the paper "Emerging Properties in Unified Multimodal Pretraining" (BAGEL). It is designed to evaluate free-form image manipulation and complex multimodal reasoning capabilities of unified multimodal models. The paper reports an initial release of 350 examples and that evaluations were run with GPT-4o. The benchmark is intended to probe advanced multimodal reasoning behaviours demonstrated by BAGEL (e.g., free-form image manipulation, future-frame prediction, 3D manipulation and world navigation). No public Hugging Face dataset entry for IntelligentBench was found during the search (dataset appears to be introduced in the BAGEL paper and may be hosted later on the project/GitHub page).
No results indexed yet — be the first to submit a score.
Submit a checkpoint and a reproduction script. We will run it, publish the score, and — if it takes the top — annotate the step on the progress chart with your name.