Posted on

Why Is Fuzzy Matching Software a Key for Deduplication?

Identifying golden and unique records across or within datasets is crucial to prevent identity theft, meet compliance regulations, and improve customer acquisition. Banks, government organizations, healthcare providers, and marketing companies all require matching algorithms to identify and deduplicate redundant entries to enrich their master database.

Fuzzy matching is a known set of algorithms for measuring the distance between two similar entities. But certain limitations hinder its effectiveness to quickly find matches for larger, disparate datasets. 

Posted on

State Change and NoSQL Databases

Have no fear, NoSQL is here!

Let's take another look at F. L. Stevens's spreadsheet with agencies and agents. It's — of course — an unholy mess. Why? It's difficult to handle state change and deduplication.

Let's look at state changes.

Posted on

UseStringDeduplication: Pros and Cons

Let me start this article with an interesting statistic (based on the research conducted by the JDK development team):

  • 25 percent of Java applications memory is filled up with strings.
  • 13.5 percent are duplicate strings in Java applications.
  • Average string length is 45 characters.

Yes, you are right — 13.5 percent of memory is wasted due to duplicate strings. 13.5 percent is the average amount of duplicate strings present in Java application. To figure out how much memory your application is wasting because of duplicate strings, you may use tools like HeapHero, which can report how much memory is wasted because of duplicate strings and other inefficient programming practices.