1

Upgrade instructions include:

Use your backup to create a copy of your site (directories and database), and ensure that this copied site works correctly.

How can I create a copy of my CiviCRM site (on Drupal)? Is there a big difference between making a copy on the same server or another one?

2

There are some instructions at https://docs.civicrm.org/sysadmin/en/latest/misc/switch-servers.

The wiki page referenced in a couple similar questions is still there but getting out of date, and there's something wrong with the styling at the moment, but it's still legible: https://wiki.civicrm.org/confluence/display/CRMDOC/Moving+an+Existing+Installation+to+a+New+Server+or+Location

2

I wrote instructions on how to do this on the old CiviCRM Wiki in around 2010 for Drupal 6 and then updated it again for Drupal 7. I just had to do it again on a Drupal 7 site and my old instructions were out of date.

The workflow for creating a copy of a Drupal/CiviCRM website with a database for the Drupal site and a database for the CiviCRM database is as follows:

1) Copy the CiviCRM data from the Live site with specific options.

2) Create a fresh empty Drupal site.

3) Copy all of the site assets (files, libraries modules) from the live site to the fresh empty Drupal site.

4) Install a clean copy of CiviCRM in the fresh empty Drupal site.

5) Copy/Import the Drupal database into the fresh empty Drupal site.

6) Import the CiviCRM data from step 1 into the fresh empty Drupal site.


Here is a step by step process of the above:


From the LIVE SITE get a full MySQL dump of the database with the following key options. You can do this with MySQLAdmin or PHPMyAdmin.

1) Do not copy the following tables:

  • civicrm_acl_cache
  • civicrm_acl_contact_cache
  • civicrm_cache
  • civicrm_domain
  • civicrm_group_contact_cache

2) Disable foreign key checks

3) Add "IF NOT EXISTS" clause under the "Add CREATE TABLE statement"

4) Add "Truncate table before insert" option


Install a fresh, blank Drupal website with the same version of the Drupal 7 that the live website uses.

After it has been installed do the following:

1) Copy /sites/default/files from the live site to the local site.

2) Copy /sites/all/libraries from the live site to the local site.

3) Copy /sites/all/themes from the live site to the local site.

4) Copy /sites/all/modules from the live site to the local site except for the civicrm module. We will use a brand new installation of CiviCRM to ensure it is clean.


Install CiviCRM in the local site

Get a fresh download of the CiviCRM version you are using on the live website. Place it in the /sites/all/modules directory and run the installer at: /sites/all/modules/civicrm/install/install.php.

After the installer runs you should have a blank Drupal 7 website and a blank copy of CiviCRM running within it. Check the Admin controls and Drupal's log messages to ensure that the installation is clean and that CiviCRM is running correctly before you proceed to import the website and CiviCRM data.


Migrate/Import the Drupal database into the local site

Migrate the Drupal database from the existing live site to the new blank local Drupal website. I use the excellent Drupal Backup and Migrate module to accomplish this but you can also do this with a MySQL Dump and import it into the site's drupal database as well.

If you are using Backup and Migrate on the Live drupal site, navigate to the Configuration -> System -> Backup and Migrate UI (/admin/config/system/backup_migrate), clear caches and download a backup of the Drupal site. On the new Local site navigate to the Configuration -> System -> Backup and Migrate -> Restore UI (/admin/config/system/backup_migrate/restore) and load the backup file that you downloaded and click restore. It may take a couple of minutes but the new local site should take on all of the Drupal features and settings of the Live site.

At this point I clear the site caches on the local site and prepare to import the CiviCRM data.


Import the CiviCRM data

Importing the CiviCRM data may be problematic. When you exported the CiviCRM database dump, you disabled foreign key checks, added a if not exists clause to the table create clause and you added the option to truncate the table data before importing data so you should be able to simply import the CiviCRM dump from the live site into your local site. I've done it with PHPMyAdmin but CiviCRM is a monster database so I like to use the terminal window with the following command:

mysql -u db_user_account -p civicrm_database_name < path-to-the-civicrm-dump.sql


Post import housekeeping

Once the Data has imported there are a few small things that need to be attended to:

1) delete the templates_c directory from the /sites/default/files/civicrm directory.

2) Clear the caches from CiviCRM, Clear the Drupal caches.

3) Make sure that your Resource URL's are correct. They will more than likely be set to the live website's resource URL settings.

4) Synchronize users to contacts

5) Go to the configuration checklist and make sure that it is correct.

6) Go to CiviCRM Admin -> System -> CMS Integration and copy any custom fields that you need to add to the /sites/default/settings.php file that are necessary for Drupal Views integration.


That should do it.

5
  • I've done countless site migrations, but never omitted the cache tables - and omitting civicrm_domain seems dangerous. Is that step still necessary? Likewise, I don't do full reinstalls - I simply copy the entire filesystem (minus templates_c) and modify settings.php/wp-config.php/civicrm.settings.php, which allows me to skip steps like syncing contacts. Oct 9 '20 at 13:00
  • It may be that the domain table can be copied in todays version of CiviCRM. I've been active with Drupal/CiviCRM since 2010 and in the earlier versions if you tried to move a site with the domain table, CiviCRM would break. The rule was that if you were moving your site from example.com on Server A to example.com on Server B then you can leave the domain table but if you move CiviCRM from example.com on the production server to mylocalcopy.local on your workstation or local server you omit the domain table from your dump and use the one that was produced when you installed it on your local. Oct 9 '20 at 16:54
  • @Jon G - Megaphone Tech, it might also be a difference between how Drupal/CiviCRM work and WP/CiviCRM work. Oct 9 '20 at 16:56
  • Agreed that it was necessary in 2010, which is when I started too! I work with D7/D8/Backdrop/WP, and can confirm that you no longer need to omit tables when moving a site. Oct 9 '20 at 18:22
  • @Jon G - Megaphone Tech, thanks for confirming that. That is awesome. I looked in the table the other day and the data seemed trivial. That will make cloning a site so much simpler. 2010 was the year I built my first CiviCRM Drupal site. They're still running it on D6 using MyDropWizard modules. I'm still a fan of Civi. It's a mighty powerful CRM. Oct 14 '20 at 1:11
2

Andrew's answer is very comprehensive but in some cases you might opt for a simpler approach.

This answer assumes the goal is to set up a "Clone" of the site for testing/development. It also assumes you have ssh/command line access.

The goal is an exact copy of your live site. So you need to sync a few things.

  • Codebase
  • Database
  • Uploaded files

Uploaded Files

  1. If you are using Drupal's stagefileproxy module, and not uploading much as files into civicrm you can often get away without copying files over.

  2. If you are using nginx and your dev site and production are on the same server you can use nginx's try files directive to check first in sites/default/files (or similar) then look under the live site. Note: this isn't going to work for private files.

  3. Copy files across. Using rsync can avoid you copying files that exist already. Be kind to the planet and your bandwidth limits at the same time!

    rsync -avz --progress files destserver:destdirectory/sites/default/files/

    Be careful with your trailing slashes and perhaps do a --dry-run first.

Database

Depending on your setup you might have your civicrm in a different database. If so don't forget to update the settings.php aliases mapping after you have finished if you have new custom field groups and are using civicrm drupal views/webform integration.

Things to be aware of. Civicrm uses sql triggers and some sites will have functions and views. When exported these include the username that "defined" them. This will result in issues unless you have a duplicate username/host on the destination machine. And you definitely don't want to load these back into the same machine.

You can strip these out with the mysqldump however I recommend making a practice of using the following perl one liner - that way if you forget this. Or get a dump from someone else who hasn't run it you wont hit issues.

How do I safely export my database and import it on a new server?

perl -pi -e 's#\/\*\!5001[7|3].*?`[^\*]*\*\/##g' /path/to/mysqldump.sql

pv - if you have a large civicrm database pv is your friend. It will give you a very limited idea on how long the import will run for - and for the modern generation that are not used to waiting for things to run in the background with no feedback (that's me) give you some reassurance that everything is going to plan.

On debian aptitude install pv

pv reads from stdin (or the filename provided as an argument) outputs to stdout and prints a progress bar in the terminal.

generate your dumpfile from live via:

In MySQL 5.7.8+ you can use --skip-definer

mysqldump --single-transaction databasename  | pv |  gzip > databasename.sql.gz

Top tip - Have massive log files but don't need them on your copy - then

mysqldump --single-transaction databasename $(mysql -BN -e "select table_name from information_schema.tables where table_schema = 'dbname' and table_name not like 'log_civicrm%'")  | pv | gzip > databasename.sql.gz

Load the database into mysql. If you are refreshing your copy from live easiest to just drop the existing database - top tip make sure it's the right one, and maybe back it up first...

You might want to run the load step in steps.

mysql -e "drop database databasename"
mysql -e "create database databasename"
pv databasename.sql.gz | gunzip | | perl -p -e 's#\/\*\!5001[7|3].*?`[^\*]*\*\/##g' | mysql databasename

Codebase

The initial copy can be everything but you'll need to update civicrm.settings.php and settings.php subsequent copies a git based workflow works well in drupal 7. Drupal 8/9 offer composer based workflows.

2
  • I'll give this a spin on a test site @Luke Stewart. The official docs are lacking so it's good to see some alternatives. Oct 9 '20 at 16:59
  • hopefully some of what is here can make it in to official docs
    – petednz - fuzion
    Oct 11 '20 at 22:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.