4

When doing an AB test on two groups of 7k contacts, what seem to happen is that first the A group is sent in full, then the B group is processed.

It obviously means that the second group is going to be sent later (in the latest experiment, 30 min later) than the first one, so obviously, it's a huge bias and it's hardly possible to compare the open or click rates, because A has such a head start.

Is this possible to change the way the mails are sent to process the two groups as the same time? ie. change something on the cronjob?

2
  • We've also run in to this issue. Even when using multiple 'send' processes - CiviCRM still sends all the A batches and then starts on the B batches. This is perhaps something to raise in the issue queue? Since it sort of defeats the object of an A/B test.
    – John
    Oct 7, 2016 at 15:03
  • I did open an issue: issues.civicrm.org/jira/browse/CRM-20309
    – Djizeus
    Mar 22, 2017 at 11:55

1 Answer 1

3

What I did was to keep big batches (bigger than the test groups) and allow to run several jobs, and then:

drupal_root=/var/www/drupal

*/5 * * * * www-data nice -n19 /usr/bin/drush -u 1 -r $drupal_root civicrm-api job.process_mailing auth=0 -y -l wemove
*/5 * * * * www-data nice -n19 /usr/bin/drush -u 1 -r $drupal_root civicrm-api job.process_mailing auth=0 -y -l wemove
5
  • This is still not solving the problem 100%: if one of the processor, is busy processing another mailing, A and B will still be sent at different time
    – Xavier
    Mar 21, 2017 at 16:00
  • Has this been resolved yet?
    – Graham
    Jan 23, 2018 at 20:37
  • Still not perfect. The ultimate plan is to move so the AB are part of the same mailing. Part of mosaico next version... and still looking for financial help to make it happen. Could you help?
    – Xavier
    Jan 24, 2018 at 10:22
  • I'm not clear as to why AB testing functionality has to be bound up with mosaico. If there are details somewhere about what financial help is needed and what work needs to be done that would help if I'm approaching clients who may be interested in getting AB testing working properly.
    – Graham
    Jan 24, 2018 at 16:08
  • 2
    Basically, to be sure A and B are sent at the same time, it needs to be a single mailing, hence change of UI... and it's better to build on the top of mosaico for that ;) More information: wiki.civicrm.org/confluence/display/CRM/ABn+testing+v2
    – Xavier
    Jan 26, 2018 at 14:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.