Contact Dedupe Basics

Question

I'm new to CiviCRM and I'm trying to help a non-profit clean up what looks to be a very large number of duplicate contact entries. I have searched this site and a few other resources and I have a grasp on the process, as I understand it. While I'm new to CiviCRM, I have worked with many other database and CRM solutions. Here are the two questions I have:

1) What are dedupe exceptions? While most things related to CiviCRM and dupe are well documented, I've not located an explanation for this. I assume it is some sort of white list of possible dupe contacts? What happens when I remove the exception? Do they become "eligible" for the dedupe rules engine?

2) None of the General rules that have been setup will work. The Supervised/Unsupervised rules work just fine when I use them manually. What I'm experiencing is basically a very long period where I'm waiting on the database followed by an unable to perform at this time message. Any thoughts on using General Rules. I have tried various approaches but basically trying to match on 3-4 fields. I understand the weight/threshold approach but it doesn't seem to be linked to that as much as to General rules. None of the rules I have setup nor the ones that existed before me seem to work as General rules.

Thanks in advance and please take it easy on a noob. :)

Dave

Answer 1

Dedupe exceptions are those contacts that have been flagged as potential duplicates, but someone clicked the "Not a Duplicate" link next to them. If you remove the exception, they can be considered duplicates again.

CiviCRM has two kinds of dedupe rules - those that you build and some pre-built rules that give much better performance. Many of the default supervised/unsupervised rules are pre-built rules. The rules you're building are timing out. To deal with this, try increasing the server's PHP timeout or reducing the complexity of your rules. Also - dedupe has received a lot of love in recent months, so consider upgrading CiviCRM to the latest 4.7 for a performance boost.

If you're handy with SQL and want to write a "pre-built" dedupe query, the hook is hook_civicrm_dupeQuery. Veda's dedupe extension is outdated and may not work on current versions of CiviCRM but has a large number of high-performance dedupe rules.