3

Tips for troubleshooting email performance.

I have a CiviCRM instance that currently sends 800 emails an hour and I have the feeling that it can do better. Some details.

  • CiviCRM 5.51.3
  • The CMS is Drupal 9.
  • The mail is sent by SMTP to AmazonSES (The Paris data-center is used, and port 587)

The CiviMail Settings are as follows:enter image description here

The cron starts 4 times per hour. So, it is ready for 2000 emails.

The mailing process keeps the server busy. There is a long-running PHP process that asks 25% processor time, and MySQL needs 15%.

The SMTP server sometimes emits the following error.

(code: 451, response: 4.4.2 Timeout waiting for data from client.)]

This suggests that Amazon is fast enough and waits for CiviCRM. It also bothers me a bit, and that is why the database update rate is 1 (to prevent double emails).

The sending limit for Amazon is 50 mails a second (with a maximum of, 100000 a day).

Has someone tips, how to troubleshoot this? I already disabled the custom tokens, unfortunately without impact.

3
  • 1
    We started to have this problem several versions ago (trying to find which one) and I limited the batches to 250 (with no throttle time). Without batches, it would crash and not send at all and had to dupe email and start over. We also use AWS SES, and it used to work like a dream. (We have database Update Frequency at 10). email-smtp.us-east-1.amazonaws.com and port 587. Drupal 7.
    – TimETown
    Sep 13, 2022 at 22:43
  • What do you have as values for the other settings? Do you have specific settings for the scheduled mail cronjob? Sep 14, 2022 at 13:36
  • 1
    We had throttling problems with a client using SES as well. after roughly 1000 Emails we had SMTP errors that weren't parsed as errors in CiviCRM. We ended up using a relay Postfix that was able to queue mails and could adapt to SES sending rate. I don't exactly know what the problem was. We have other clients using SES with default civi settings that send without problems.
    – Phil_B
    Sep 20, 2022 at 9:08

3 Answers 3

2

I found the culprit. It was CiviRules. In the system, a rule configured on activity with a complicated test. For every e-mail sent, a mass mail activity is created and the activity rule fires. Disabling the rule did speed up the email job.

1
  • Well, we don't have that extension installed and have problems. But I do have civiextensions/nz.co.fuzion.cssinline installed, and I know that uses a lot of resources per email. I still don't know why the problem cropped up only after upgrade to 5.48.0.
    – TimETown
    Sep 21, 2022 at 14:37
0

We usually get around 6000 per hour through AWS, with the following settings and a cron job running every ~5 minutes:

enter image description here

Perhaps try a lower throttle time? I don't think the batch size should be much different between 400 and 500, but I do recall some performance improvement from reducing it in the past.

1
  • Our problems started in April of 2022. when we upgraded to 5.48.0 from 5.46.3. That's when we had to start batching or the entire process would hang.
    – TimETown
    Sep 16, 2022 at 21:23
0

Our problems started in April of 2022. when we upgraded to 5.48.0 from 5.46.3. That's when we had to start batching or the entire process would hang.

(Embarrassed to say I have been using the Drupal Cron and so just runs 1x/hour, which I need to fix.)

our settings

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.