Downloads: 112

Research Paper | Computer Science & Engineering | India | Volume 4 Issue 11, November 2015

Removing Dedepulication Using Pattern Serach Suffix Arrays

Pratiksha Dhande | Supriya Kumari | Sushmita Tupe | Laukik Shah

Abstract: With the increase of de-duplication in data sets of voter card or pan card, removing the de-duplication is the major challenge. Record linkage is the process of matching records from several databases that refer to the same entities. When appliedon a single database, this process is known as de-duplication. In this paper the investigation is done to how to remove the de-duplication with the help of suffix arrays. Suffix array is well organized data structure for pattern searching. This paper covers similarity metrics that are commonly used to spot similar field entries, and present a widespread set of duplicate detection algorithms that can identify almost duplicate records in a database. It also covers multiple techniques for improving the effectiveness and scalability of estimated duplicate detection algorithms. Finally, based on the algorithms, the paper presents how to remove the de-duplication from dataset.

Keywords: String search, pattern matching, suffix array, suffix tree

Edition: Volume 4 Issue 11, November 2015,

Pages: 1217 - 1219

Similar Articles with Keyword 'String search'