Do you really trust the responses of Large Language Models (LLMs) to your religious questions?
Are you looking for a high-value research problem or competition for your undergraduate or MSc thesis, or to enhance your research skills?
Registration is now open for the IslamicEval 2026 shared task, “Accurate Detection of Hallucinations in Arabic Islamic Content,” which will be held as part of the ArabicNLP 2026 conference, concurrently with the EMNLP 2026 conference in Budapest, Hungary.
This task focuses on detecting, verifying, correcting, and assessing the relevance of Quranic and Hadith quotations in the responses generated by large linguistic models (LLMs) to answer religious questions posed to them. Building on last year’s first edition of the shared task (IslamicEval 2025), this edition adopts a more rigorous methodology with four independent sub-tasks.
✨Subtasks
💡Subtask 1 Span Detection: Find the spans of claimed fragments and label each as Ayah, Hadith matn, isnad, or claimed source.
💡Subtask 2 Hallucination Identification: Label each fragment correct or incorrect. Isnad and claimed source are judged only when their corresponding Ayahs and matns are correct, N/A otherwise.
💡Subtask 3 Hallucination Correction: Provide the canonical text for incorrect Ayahs and Hadith matns. Covers Ayahs and Hadiths two types only.
💡Subtask 4 Answer Relevance: Decide whether a citation is actually relevant to answering the question it was given for.
⚠️Each subtask is independent and has its own dataset, so teams can participate in any of them.
⚠️Participating teams are allowed to use models with 13 billion parameters or fewer.
⚠️Important Dates
- July 1, 2026: Training and Development Data Blog Available
- July 20, 2026: Registration Deadline
- July 23, 2026: Test Data Blog Published
- August 1, 2026: Results Delivered
Leave a Reply