The CiviMail documentation and settings pages indicate that it is possible to have multiple mailer delivery jobs executing simultaneously.
See Settings - CiviMail (/civicrm/admin/mail?reset=1) :
Mailer Job Size: If you want to utilize multi-threading enter the size you want your sub jobs to be split into. Recommended values are between 1,000 and 10,000. Use a lower value if your server has multiple cron jobs running simultaneously, but do not use values smaller than 1,000. Enter "0" to disable multi-threading and process mail as one single job - batch limits still apply.
Mailer CRON job limit: The maximum number of mailer delivery jobs executing simultaneously (0 = allow as many processes to execute as started by cron)
I have not been able to get this to work in practice in CiviCRM 4.4.14 . I have tried various settings, e.g.
batch limit = 2500
throttle time = 5
job size = 2500
cron job limit = 2
civimail_server_wide_lock = off
Civi cron is running every minute. Mailings scheduler is the only scheduled job that runs on every cron run (others are hourly or daily). The mailing is correctly split into sub-jobs of 2500 mails, however these get processed consecutively. Each batch takes about 6 minutes to run. Every time a new instance of the scheduled job tries to start while an existing instance is running, we get an error like this:
Finished execution of Mailings scheduler with result:
Failure, Error message: lock aquisition for mydb.1.civimail.job.12415
attempted when mydb.1.civimail.job.12414is not released
Hence only one instance runs at a time.
The error message above is generated by CRM_Core_Lock::hackyHandleBrokenCode(), which was added for CRM-12856 and is called from CRM_Core_Lock::__construct(). It prevents multiple locks being created with names containing "civimail.job.". If this is attempted, hackyHandleBrokenCode() throws an exception and so the scheduled job exits. This appears to be what is preventing multiple mailer delivery jobs executing simultaneously.
Looking at CRM_Mailing_BAO_Mailing::processQueue(), the code for $config->mailerJobsMax > 1 seems to work on the assumption that a CRM_Core_Lock can be created independently for each of the mailerJobsMax cron jobs. But since CRM-12856, this is disallowed. I believe this may not have worked correctly in any case, given that each MySQL connection can only acquire one lock: acquiring a second, even if it has a different name, releases the first: https://dev.mysql.com/doc/refman/5.5/en/miscellaneous-functions.html#function_get-lock.
However some users have reported this feature to be working correctly - can anyone confirm that mail jobs are indeed sending in parallel in their post-CRM-12856 Civi?
