Upgraded to PHP 7.2; no errors but Sparkpost doesn't receive and send CiviMail

Question

I upgraded to PHP 7.2 and received the system warning​ stating "your SMTP password will not be stored encrypted, and if you have recently upgraded from a PHP that stored it with encryption, it will not be decrypted correctly." I updated our Sparkpost API key and everything on CiviMail's side seems to be working--all test messages go through and no errors pop up when sending email. However, when we tried to send a mass mailing, we got 100% successful deliveries but zero openings; upon further investigation, Sparkpost did not receive send any of the emails. Has anyone else encountered this or know a fix?

Answer 1

PHP 7.2 does not have the mcrypt module, and the Sparkpost extension needs that module. If you can install php7.2-mcrypt as a pecl module Sparkpost will work. If not, I would move to php 7.1 for the time being.

Answer 2

I encountered the same problem and investigated a bit further...

The SparkPost extension does not directly use mcrypt but it does use CRM_Utils_Crypt which makes use of mcrypt if it is available. If you upgrade to php 7.2 without mcrypt and you already have an API key saved, the result is that the extension cannot decrypt the key. That's why the instruction is to resave your API key which will store it unencrypted. (If you don't remember it, just generate a new one on the SparkPost site.) However, even having done that correctly you may still get the symptoms described.

The basic problem is that if you have multiple versions of php installed you may end up calling the wrong one - particularly in a shared hosting environment.

First, check that you can send a test mail successfully (create a mailing and use the test option). If that does not work, you have a different problem and the rest of this won't help. Providing that works, you have verified that the extension is working with php 7.2

The thing to look at is how you are running the scheduled jobs. There are various different methods but if you are doing that through cv/drush/wp then the version of php that those commands use is key. If they run a version of php where mcrypt is available (eg 5.6) then when the SparkPost extension runs CRM_Utils_Crypt::decrypt() it will 'decrypt' via mcrypt the API string - but since that was saved with 7.2 it was not encrypted with mcrypt in the first place and the result is an invalid authorization string. Unfortunately the SparkPost extension does not handle this condition properly and results in it incorrectly saying that the mails were successfully delivered.

Drush has a somewhat convoluted invocation process and even if you call the initial drush using php 7.2, the eventual php that gets run may be a different version and lead to this problem. The best solution I have currently is to use cv and explicitly run it under 7.2.

A cron script like this works:

#!/bin/bash

rootdir=~/www/www              # website root dir
phppath=/usr/local/bin/php72   # path to PHP 7.2 binary
cvpath=~/private/bin/cv        # path to cv script
cronuser=admin                 # login to run commands as

cd $rootdir
$phppath $cvpath --user=$cronuser api Job.execute

Answer 3

OK, the answers here finally sunk in. The key is Aidan's answer in my mind, to simply use the same PHP version in your cron job as in your CiviCRM setup. Thanks, @Aidan, this is the 2nd (unrelated) problem you solved for me today! In our case, we run CiviCRM 5.10.3 under PHP 7.2.11 with CMS WordPress 5.2.1. After updating our Sparkpost API Key at Sparkpost and pasting into "Outgoing Email (Sparkpost)" I just changed our CRON job to use the following script:

CIVI_ROOT="{YOUR_HOME}/sdbikecoalition/wp-content/plugins/civicrm/civicrm" 
PARAMS="-j -ssdbikecoalition.org -u {YOUR_ADMIN_USER} -p {YOUR_PASSWORD} -e Job -a process_mailing" 
nice -n19 /usr/local/php72/bin/php $CIVI_ROOT/bin/cli.php $PARAMS

Worked perfectly. We had been getting 100% delivery and 0% opens.