CiviCRM Dedupe - How does it really work?

Question

There seems to be some confusion at my current organization re: de-duping. I'm under the impression that if I have two records (organizations)

Case 1: name: St. John's University Zip: 11439

Name: St. John's University Zip: 10301

In order to de-dupe, I would place a "weight" on particular fields. HOWEVER

What if I only have the name ? How does one dedupe "St. John's" "St Johns" "St. John's U." Is that technically called deduping ? (pardon if this is a noob question. My first on Stack Exchange)

Answer 1

For your specific example, "St. John's" "St Johns" "St. John's U.", you can set up a rule that would find a couple of these but sadly not all of them.

As Erikh implies you could setup a rule that only has one field, in this case organization name. Then if you limit to length of 6 or 7, it would find both the first and last examples plus a couple other possibilities.

The one with the missing period is tough since limited it to 2 would find anything that starts with "St". Of course, if you have one other piece of info like address, city or zip code than the list of possible duplicates could be reduced to something manageable.

A nice enhancement to CiviCRM might be to have an option for dedupe to ignore punctuation.

Finally for completeness here is a link to the online documentation on deduping.

https://docs.civicrm.org/user/en/latest/common-workflows/deduping-and-merging

Answer 2

Whether it is technically called deduping or not, these kind of things do occur in real life :-). You should be able to set up dedupe rules that catches them, and one organization I worked with used different 'levels' of deduping rules in their process. For example:

1. dedupe on something like social security number where it is pretty obvious that they are quite possibly real dedupes, and fix those
2. dedupe on email_address where there is a reasonable chance that they are dedupes, and fix those
   3. dedupe on something like part of first_name and part of last_name, where chances are that in quite a few cases there are no real dedupes, and fix those if fixes are required.

Obviously the system can not tell you if I have a girl called Louise Tomlinson and another girl called Louise Tomlynson that they are the same or actually someone else. It can only flag that they look the same.