3

We have a very large Civi database and are looking to dedupe. I have created smaller groups to work from (ex. all records with last name starting with "A") but even that group has 12,000 records. It takes FOREVER to run the dedupe from that group against the whole database.

Is there any way to limit the duplicate search so it only searches within the group?

2
  • we did some work for a client in similar position where we got the dedupe to run against contacts where both were in the same group. we also did it so the deduping ran on some cached data which also really sped things up. don't think we published but if Pradeep's suggestion doesn't help then come back to me and i will dig further
    – petednz - fuzion
    Commented May 28 at 19:39
  • 1
    You could also use the extension X-Dedupe: docs.civicrm.org/xdedupe/en/latest It has better performance when working with larger databases and quite a few nice features.
    – Fabian
    Commented May 29 at 6:58

3 Answers 3

2

Unfortunately, the dedupe screen provided by the CiviCRM searches contacts outside the group.

But I have found Deduper extension much useful for larger databases.

1

You could also have a look at our extension here

Adds the potential duplicate contacts to a group based on config settings.

If a group is selected for dedupe, optionally ext can list the duplicate row only when contact A and B are both part of the group.

0

I would create a smart group based on contact-ids instead of names.

The first time you run it, it will take long time anyway. (So probably good idea to split into smaller groups)

But when you have done that, you could limit every new duplicate search to contact-ids over a certain number. In that way you only search the new ones. The old ones you have already deduped.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.