Using TF-IDF to Enhance Information Retrieval in Hadith Corpus

Dr. Samah Mohamed Osman Hassan, Dr. Eric Atwell

Abstract: This paper aims to address the challenge of Information Retrieval from the Hadith, focusing on multiple languages and utilizing a Hadith parallel corpus. The proposed approach involves employing a matching algorithm for the retrieval process. It calculates the weight of words in the query based on their importance and compares them with existing documents that have undergone processing to determine the significance of words in each document. Subsequently, a similarity coefficient is computed between the specific query and the existing documents. To enhance performance, the system utilizes a dictionary of words, implementing an inverted index to identify all files containing those words. The proposed solution is designed and evaluated by selecting important concepts, for which manual results have been predetermined independently from the system. The evaluation process measures both average precision and average recall for each language.

Keywords: Information Retrieval, Hadith, parallel corpus, matching algorithm, similarity coefficient