1

I'm attempting to override the fpm time out from nginx just for the import. We are importing very large files, and it might take up to a couple hours to complete, so I need to set:

fastcgi_send_timeout 0;
fastcgi_read_timeout 0;

I'm having difficulties understanding what exactly happens during the import in which location I should add this override to.

There seems to be a lot happening during the import, ajax, POST requests etc, and I have tried adding this override to just certain paths, all of which didn't fix the problem.

Our installation is at /vendor/civicrm/civicrm-core, and adding the following location block didn't work.

location ^~ /vendor/civicrm/civicrm-core/CRM/Contact/Import {
        try_files $uri /index.php?$query_string;
        location ~ \.php$ {
            include snippets/fastcgi-common.conf;
            fastcgi_send_timeout 0;
            fastcgi_read_timeout 0;
            fastcgi_pass unix:/run/php/php7.2-fpm.sock;
        }
}

I have tried /civicrm/import/contact, but it still times out with:

upstream timed out (110: Connection timed out) while reading response header from upstream, client: ::ffff:REDACTED, server: REDACTED.com, request: "POST /civicrm/import/contact HTTP/2.0", upstream: "fastcgi://unix:/run/php/php7.2-fpm.sock", host: "REDACTED.com", referrer: "https://REDACTED.com/civicrm/import/contact"

Trying to use an invalid fastcgi socket seems to imply it's not being matched since it doesn't error out:

location ^~ /civicrm/import/contact {
    ## Standard site protection
    try_files $uri /index.php?$query_string;
    location ~ \.php$ {
        include snippets/fastcgi-common.conf;
        fastcgi_send_timeout 0;
        fastcgi_read_timeout 0;
        fastcgi_pass unix:/tmp/for_sure_will_500_and_doesnt_exist.sock;
    }
}

location ~ \.php$ {
    include snippets/fastcgi-common.conf;
    fastcgi_pass unix:/run/php/php7.2-fpm.sock;
}

The debug log also shows its matching ~ \.php$, why would it not match the import path first?

1

The import do generate a lot of requests indeed, the one you should care about is the POST one (that does the import). The others are mostly cosmetic to update the counter of how many contacts have been processed so far.

What matters for your rewrite isn't where the code is, but what's the url of that post. It depends of your install (check your log or the console on your browser), but it's likely not including the full path (vendor/civicrm/civicrm-core), eg something like:

location ^~ /civicrm/import/contact { ...

one trick I use to see if the location nginx rule is the correct one is to put an invalid fastcgi_pass, 50x errors are easier to see than waiting and trying to guess if the timeout worked or not ;)

This being said, if you have a lot of contacts to import, it's probably safer/easier to write a php script that you run from the CLI and that uses civicrm api to create the contacts (you don't have to deal with timeout, you have more memory, easier to do exactly the tests you want and create the data you need the way you want it.... and that's way easier to re-run if something breaks because some input is invalid)

X+

  • Thanks, I have also tried this path, but it continues to timeout. This is the path I see in the Chrome web console, but it doesn't seem to fix the timeout. – johnramsden Oct 12 at 1:09
  • I added a link to the error message I'm getting in my post. Why would it be matching that path but still using the wrong fastcgi settings? It matches the global value exactly when it times out. – johnramsden Oct 12 at 1:14
  • first be sure you are using the rewrite rule you want, replace fastcgi_pass unix:/run/php/php7.2-fpm.sock by fastcgi_pass unix:/tmp/for_sure_will_500_and_doesnt_exist.suck and confirm you get a 50x error – Xavier Oct 12 at 9:19
  • OK, it doesn't seem to be matching no matter what I do. I added some more to the post, any ideas? – johnramsden Oct 17 at 19:45
  • you'll probably find more experts on an nginx more focussed community (eg stackoverflow) – Xavier Oct 18 at 7:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.