4

We are attempting to keep two systems in live sync with each other, one is CiviCRM.

There are hooks in civicrm for:

Pre database saving of data

hook_civicrm_pre($op, $objectName, $objectId, &$objectRef) {}

and

Post database saving of data

hook_civicrm_post($op, $objectName, $objectId, &$objectRef) {}

but both functions are passed all of the data of the relevant form/group/profile that has been submitted rather than just data that has changed. i.e. civicrm considers all of the fields to have been updated even if it isn't the case.

Is the only option to go through each submitted field and compare the posted value to it's current value during callhooks_civicrm_pre? and then only update the 'other' system on callhooks_civicrm_post so that data that isn't actually saved isn't sent to the 'other' system.

Perhaps there is a better way, like a hook from the changelog?

Improve this question
1
  • I'm pretty sure you can use the changelog to know if anything has changed in the record since the last sync. You might not know what changed but you would at least knock out the records with no changes. I would think about whether you want to do it in this way or whether you want to trigger an external save every time there is a local save.
    – Elin
    Commented Mar 26, 2015 at 19:44

2 Answers 2

Reset to default
6

In having to do this with Salesforce and CiviCRM a number operations included the creation of a new contact being initiated on one end, which then had to be replicated on the other. In this case we wanted to prevent both sending unnecessary data and creating duplicate contacts.

Whenever the data being synced didn't involve a chain of transactions that required the result of each previous one (like a primary key id), we used the log to create the data package as each row only contains the exact fields that were changed. In the other cases we did use the pre and post hook along with storing data in the session to create a package that was ultimately sent in the post hook. In both cases, only the data that was added or modified was sent, and not full entity records. We put the responsibility on the receiving systems to understand whether they were adding a new record or updating an existing one, based on certain characteristics of the data package.

Pushing this strategy let us take advantage of the cron and the bulk Apis whenever the data allowed it.

Improve this answer
2
  • Hi Jeremy, thanks for your response. We have concluded that we will most likely be going down the same/similar route you describe. We are however working with Raiser' Edge which is decades behind sales force technological speaking. Do you happen to have released any of the code you have created to do the task to the CiviCRM community?
    – RocXa
    Commented Apr 1, 2015 at 13:48
  • The code was part of a project for a client, so we haven't released it publicly. I'd be happy to write up a blog post on civicrm.org as a guide to creating that setup, though I can't specifically speak to the capabilities of your particular version of Raiser's Edge. If you had budget to support some development, this may also be something that my company or one of the other CiviCRM service providers could handle for you.
    – Jeremy Proffitt
    Commented Apr 1, 2015 at 14:28
5

Unfortunately, there's no good one-step approach that I know of. Two options:

  1. Use hook_civicrm_pre to trigger fetching the data from the database and stashing it somewhere prior to the change. Then, use hook_civicrm_post to compare the saved record to the stashed one and send the difference.
  2. (The preferable way, in my mind) Just use hook_civicrm_post, send over everything, and then have something smarter on your remote system decide whether something has changed or not.

You might also consider the tradeoff in performance of deciding whether something has changed versus just saving everything, knowing that some things will be redundant. I suspect that you'll just take a small performance hit but save yourself a lot of headaches in making sure you accurately filter out which things are unchanged.

Improve this answer
3
  • How would you stop hook_civicrm_post from sending over updates to the other system that have just been sent to CiviCRM by that other system?
    – RocXa
    Commented Apr 14, 2015 at 10:45
  • Good question. You could have a log on the CiviCRM side that records syncs from the remote server, and you could check that before syncing to the remote server. Alternatively, you could have a similar log on the remote server. Finally, assuming you check for changes before you save on the remote server, that server's database itself could be your log: the change is made on remote, CiviCRM is updated, triggering an update to remote, except there's nothing to change, so the update there is never saved, preventing the cycle from continuing.
    – Andie Hunt
    Commented Apr 14, 2015 at 11:44
  • 2
    Using the remote system to check against before updating it seems like the best approach. That way a separate log isn't required as civi is already logging everything and so is the other system. On a side note, we have discovered there is no civicrm_custom_pre hook to go with the civicrm_custom hook (should be renamed civicrm_custom_post). This hinders creating a consistent approach so we are probably going to try and get a new civicrm_custom_pre hook added to core.
    – RocXa
    Commented Apr 15, 2015 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.