1

We're currently investigating why our mailings are sending slowly from CiviCRM. We use an external SMTP server.

Our mailing settings:

  • Mailer batch limit: 30000
  • Mailer throttle time: 2100
  • Mailer job size: 0
  • Cron job limit: 1

However, when we look at the delivery table, the times individual emails were delivered are a bit strange:

  • 10 emails delivered all on the same second
  • A delay of 2-3 seconds
  • Another 10 emails delivered on the same second
  • A delay of 2-3 seconds
  • Rinse, repeat

Has anyone run into a similar issue?

Update: After some testing, this only happens for SMTP (ie, not mail()), and it is happening with two separate SMTP providers. Investigations continue...

Update: Emails also send slowly from a standalone script which uses the PHPMailer library on the same server. That actually sends slower, at 2/3 emails per second but without a delay inbetween. The next step would be for us to try concurrent sending jobs (once this is fixed in 4.6.10).

  • Any luck with your investigation? Was it a CiviCRM issue, problem at your server, or with your SMTP provider? – Sanjay Jain Oct 26 '15 at 18:09
  • Still no luck I'm afraid Sanjay. Our server team says there are no limits on SMTP connections from the server, so that was a dead end. We'll be focusing back on this issue soon. Any thoughts appreciated in the mean time! – John Oct 27 '15 at 10:44
1

when civimail uses smtp to deliver, it doesn't create a new connection to the smtp server for every contact, but pipeline them. this might explain the bursts pattern?

  • Ah I need to update this! The 'bursts' are actually a side effect of CiviCRM's coding... We've found out quite a lot since asking this question. – John Jan 19 '16 at 13:39
1

There are a couple of points to this.

1 - SMTP is chatty, it is a fairly slow protocol. To combat this, run several mailing jobs in parallel. This can be achieved by calling the process_mailing cron job multiple more frequently. Ensure that the 'cron job limit' is equal to the maximum number mailing jobs you would want to be active at any one time. This is useful if you have a sending limit on your SMTP server of x per second.

2 - The issue with 10 emails sent 'on the same second' and then a delay: MailingJob.php processes 10 emails at once, and then writes the transactions to the database all at once. So it looks like the emails were all sent on the same second. The '10' is defined in BULK_MAIL_INSERT_COUNT in CRM_Mailing_Config. Ideally the actual time of sending would be recorded in the database, rather than the time of writing.

To speed up mailings we now call the process_mailing job independently every 5 minutes, and limit the number of emails that are sent on a cron run. This means we have several cron jobs running sending emails at once for large mailings, which greatly improves our send rate.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.