Tracking links that have been added via tokens

Question

I have created a token using hook_civicrm_tokenValues that inserts content into a CiviMail. The content includes URLs that we want to track.

As I understand it (and I don't understand it very well) in CiviMail, token substitution happens after links have been converted to trackable urls, which means that any links that are inserted into tokens will not / cannot be tracked.

I'm wondering what the smartest way to get around this is?

Options I am considering are:

1) run the relevant CRM_Mailing_BAO_TrackableURL functions again once the tokens have been subsituted and observing if there are any side effects (maybe some stuff coming from other tokens should not be tracked). FWIW, it looks like CRM_Mailing_BAO_TrackableURL::getTrackerURL() already takes some steps to not track links in the format civicrm/extern/ which would cover links like civicrm/extern/url.php?u=129&qid=77.

// hack for basic CRM-1014 and CRM-1151 and CRM-3492 compliance:
// let's not replace possible image URLs and CiviMail ones
if (preg_match('/\.(png|jpg|jpeg|gif|css)[\'"]?$/i', $url)
  or substr_count($url, 'civicrm/extern/')
  or substr_count($url, 'civicrm/mailing/')
) {
  // let's not cache these, so they don't get &qid= appended to them
  return $url;
}

2) switch the order of execution so that token substitution happens before links have been converted to trackable urls (but there may important reasons why it happens in that particular order).

3) create trackable URLs as part of the token generation though I wonder if I would be able to access the data I need for the appropriate CRM_Mailing_BAO_TrackableURL function at this point and if the function is expecting to be called in a certain context.

Answer 1

@Andrei - thanks a lot for pointing me to the solution!

Based on your answer, I wrote this function that did the trick for me for finding and converting links in arbitrary $html:

function trackUrls($html, $contactId, $jobId){

    // Find all URLs
    preg_match_all('#\bhttps?://[^\s()<>]+(?:\([\w\d]+\)|([^[:punct:]\s]|/))#', $html, $matches);

    // We need to find the mailingId and the eventQueueId
    $mailingJob = civicrm_api3('MailingJob', 'getsingle', ['id' => $jobId]);
    $mailingEventQueue = civicrm_api3('MailingEventQueue', 'getsingle', [ 'job_id' => $jobId, 'contact_id' => $contactId]);

    foreach($matches[0] as $url){
        $replacements[$url] = CRM_Mailing_BAO_TrackableURL::getTrackerURL($url, $mailingJob['mailing_id'], $mailingEventQueue['id']);
    }

    $html = str_replace(array_keys($replacements), $replacements, $html);

    return $html;
}