Generic Solutions

Deduplication is the task of identifying duplicate entities in a given set of data with the same matching information. Traditional data matching performs matching of records in the database sequentially. First record is searched against all other records in the set. Then the second record is searched against all other records except the first record and so on. The sequential nature of the linear search operations are very expensive resulting in slower response and almost impossible when the data dealt is beyond certain volume. For instance even when a query is very fast and fetches results in 1 sec against a data of 10 millions, the estimated time for deduplication of this 10 million data is 4 months. As the matching rules increases, it becomes almost impossible to dedupe. Even though indexing helps searching to certain extent, matching partial identities across heterogeneous databases to find duplicate records can be made achievable only by specialized software like SetMatch.

SetMatch is the next level innovative search engine technology that aggregates voluminous data into multiple sets of clusters for efficient and super fast matching. It is based on PrimeMatch^® (URL here) and leverages all the benefits of alternative identity searching and matching.

Posidex provided numerous deduplication solutions based on SetMatch technology to deduplicate millions of records with unmatched speed and accuracy. Some of the largest deduplication exercise in India has been carried out by Posidex. For knowing further about the expertise of Posidex in deduplication of entities in very large databases,, please contact and we shall be happy to demonstrate the fact.