MMNeedle (MultiModal Needle-in-a-haystack) is a benchmark for evaluating long-context capabilities of multimodal large language models (MLLMs). The benchmark stresses sub-image level retrieval and understanding by asking models to locate a target "needle" (a sub-image or region) inside a large "haystack" composed of many images or stitched images to create very long visual contexts. The benchmark includes a protocol to generate labels for sub-image retrieval and supports multi-image and stitched-image inputs to scale context length; evaluation focuses on the model's ability to find the correct sub-image given textual instructions and visual context. The dataset, code and leaderboard are linked from the project page and GitHub repository for the MMNeedle benchmark.
No results indexed yet — be the first to submit a score.
Submit a checkpoint and a reproduction script. We will run it, publish the score, and — if it takes the top — annotate the step on the progress chart with your name.