Multi-IF is a benchmark for evaluating large language models on multi-turn, multilingual instruction-following. It extends the IFEval framework by incorporating multi-turn sequences and translating English prompts into seven additional languages, producing 4,501 multilingual conversations where each conversation has three turns. The benchmark uses a hybrid annotation/evaluation framework combining LLMs and human annotators and was used to evaluate state-of-the-art LLMs. Languages covered include English, French, Spanish, Portuguese, Hindi, Chinese, Russian, and Italian. The dataset and evaluation code are hosted by Facebook/Meta on Hugging Face and GitHub (license: CC-BY-NC-2.0).
No results indexed yet — be the first to submit a score.
Submit a checkpoint and a reproduction script. We will run it, publish the score, and — if it takes the top — annotate the step on the progress chart with your name.