3

I have a great quantity of Contact to import in CiviCRM and I absolutely need to increase the actual 3MB limit to, at least, 150 MB, with a view to fit my needs.

Many thanks for the help!

Regards

5

The size limit that is an issue is on your webserver php configuration, usually under /etc/php5 on linux

However and by experience, importing a lot of contacts via the web interface is painful. it will time out, you will need to restart and do the mapping of columns->fields, go through the wizard, wonder where it got stuck, see that civi expects "GB" and not "UK" or whatever...

It's much better to write a simple cli script that uses the api to import them, it's going to be easier to run again if needed and way faster.

importing 1.5 million organisations is definitely something you want to do from the cli if you value your time, and care about your mental health.

check out the api explorer, it will generate you the code needed to import a contact

$result = civicrm_api3('Contact', 'create', array(
  'sequential' => 1,
  'contact_type' => "Organization",
  'organization_name' => "bla",
  'api.Address.create' => array('country' => "FR", 'city' => "Paris"),
));

wrap that with your loop reading your csv and you're good to go ;)

| improve this answer | |
  • Hi Xavier, Bonjour Xavier ! – Michel GREGOIRE Dec 30 '16 at 12:16
  • Hi Xavier, Bonjour Xavier ! I think we are both french ! But lets continue in English, it will maybe help someone ! I confirm to you: import though the CiviCRM procedure is really painful. My import timeout only authorize me a 8 000 import at each time. It means, this way, near 200 files to cut and import ! A titan work ! You have to know that I'm a real newbie with this kind of import, hosting management, and command line mode. Could you explain to me very precisely how to import my files through the CLI procedure you're talking about ! Many thanks bye advance ! Michel – Michel GREGOIRE Dec 30 '16 at 12:25
  • hi, I was suggesting to write a program to run the import. This isn't a complex one, but this needs a bit of coding. Do you have the skills or inhouse skills? you might want to look at bin/csv/import.php civicrm.org/blog/xavier/… – Xavier Dec 30 '16 at 13:00
  • I'm sorry but your explanations are not enough clear for my little skills ! At this time, the CSV look like : see here : All the contacts are "Organization", with Tag "Entreprise"link – Michel GREGOIRE Dec 30 '16 at 14:00
  • Hi ! I've tried to export /tmp/tagall.csvas you explain in your post. I use (like in Cron : username : Username, Password : Userpassword, site : CiviCRM-SiteKey. Is the followin command the good one ? Do I left the -u, -p -s before the credentials ? Command done : php bin/csv/export.php -uUsername -pUserpassword -sCiviCRM-SiteKey Tag > /tmp/tagall.csv – Michel GREGOIRE Dec 31 '16 at 8:47
2

I'm guessing 3Mo in some non-English language means 3MB...

The upload limit is under Administer > System Settings > Misc. That cannot be larger than the maximum file size in your php.ini so you will need to increase that as well.

But whatever file sizes you configure there, things are unlikely to work well trying to import a single 150MB file.

Does your 150MB figure relate just to name/address/email/phone information, or is that other data as well?

If it has other data, you will need to prepare separate files for importing basic contact data, contributions, memberships, activities etc so your actual import files will be smaller.

You can import multiple files of each type - it does not need to be a single import.

Importing via the web pages works well enough for small data sets but there are other approaches that you might want to look at it if the data is large. How many contacts do you have?

| improve this answer | |
  • Many thanks for the help ! The file only contains data regarding only orgaizations (Name, email address, and certain custom dats like Activity sector...) – Michel GREGOIRE Dec 25 '16 at 13:06
  • The number of contact is approximately 1.5 millions organizations (split in 3 csv files) – Michel GREGOIRE Dec 25 '16 at 13:10
1

First, change file upload size to 150 MB in the php.ini file and then, try to increase upload size through the CiviCRM(The upload limit is under Administer > System Settings > Misc > Maximum File Size (in MB)).

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.