8

I'm wanting to (re)develop an extension and I'd like to use some 3rd party libraries, eg. guzzle, that I would normally manage via composer.

So my github repo will contain a composer.json file, but not the vendor directory. So would I need to package installable versions with the libraries included?

Also, what if I were to install libraries that are already included (eg dependencies of the libraries I want might be already in use leading to a namespace clash).

Is there a workflow for this sort of thing?

  • Are you using an architecture like github.com/drupal-composer/drupal-project or composer.rarst.net/recipe/site-stack to manage the entire site/build with composer? Or looking for a way to distribute one extension to the general-public via composer? – Tim Otten Apr 23 '16 at 1:36
  • The latter - this project is already in use by quite a number of organisations and I want to do more on it. (to add to your useful links Drupal composer_manager is also related). Basically in development I want to use composer, but in production/packaged install via CiviCRM's normal extensions thing I'd like the user not to need to know about composer, and it "just work" – artfulrobot Apr 23 '16 at 5:47
8

Approaches

There are a few different patterns one could follow in connecting Civi extensions with composer:

  1. (Broad scope) Site build via composer: The entire web site is defined with a custom composer.json file. It doesn't matter what kind of dependency you need to pull in (CMS, CMS module, CMS theme, CRM, CRM extension, PHP library, CSS library, etc), it's all brought in through the same tools/workflows.
  2. (Medium scope) Use composer for the extensionDir: Civi has a special folder (e.g. sites/default/files/civicrm/ext) for storing downloaded extensions. We could keep this folder but manage it differently -- e.g. rewrite CRM_Extension_Downloader as a wrapper for composer.
  3. (Narrow scope) Intra-extension dependency: If one particular extension requires a particular library, then the extension could have its own copy of composer.json and vendor/. The final zip file would embed vendor/.

Site build via composer

IMHO, this is the holy-grail. It's consistent with the architecture of PHP and composer, and it resolves problems that afflict dependency-management in Drupal (libraries/ guffaw!) and CiviCRM (dependencies guffaw!) and probably Joomla/WordPress.

It's worth drilling down on the architecture concern for a moment. Civi is deployed as a PHP app which shares a runtime with another PHP app. In PHP, the runtime can only load one version of a library -- e.g. you can load Guzzle v5.0 XOR v5.1 but not both, so it's important to pick a version of Guzzle that is acceptable to all libraries/modules/extensions. When using composer in narrow/piecemeal fashion, it doesn't get the full list of libraries/modules/extensions, so it cannot help you pick a version of Guzzle that satisfies the overall system. You're more likely to suffer from hidden version-conflicts (if, e.g., a Drupal module loads Guzzle v5.0 while your Civi extension loads Guzzle v5.1).

The main challenge is that many of the relevant systems (Drupal, Joomla, WordPress, Civi, etc) predate composer and have some impedance mismatch with composer (e.g. composer lacks a GUI; Drupal has multisite PHP loading; post-install hooks work differently). Getting these communities to switch is a major undertaking.

I think there are some fairly clear steps we can take toward supporting this model, but your question was really aimed at a different approach...

Intra-extension dependency

No one controls enough of the ecosystem to quickly switch a critical-mass of site-building to composer. And this version-conflict stuff may sound theoretical. In fact, you can get some mileage out of piecemeal dependency management. (Case in point: civicrm-*.tar.gz ships with its own bundled copy of vendor/!) And your comments indicate an interest in this approach.

Unfortunately, you're basically on-your-own to make this work. Loosely, I'd try:

  • In the myextension.git repo, add a composer.json file which lists any dependencies. Use .gitignore to avoid committing vendor/. In myextension.php, use require_once __DIR__.'/vendor/autoload.php';.
  • When registering your extension on civicrm.org, do not specify a Github URL. Neither github.com nor civicrm.org is smart enough to build your releases automatically.
  • Setup a CI tool which monitors your git repo and publishes the final zip archive. As a very loose example, I publish the civix.phar file with Jenkins (job, script). You could write a similar script to run composer install, zip, and then publish the final file.
  • Whenever you want to publish a new release, login to civicrm.org and manually create a new release node (with suitable info.xml and download URL).

Unfortunately, there is a risk that your dependencies will conflict with others'. There's not a whole lot you can do about this... except pick obscure dependencies that aren't likely to conflict.

(In theory, if you know that your extension and the conflicted one never use the conflicted dependencies within the same page-request, then you could do some fiddly bits in the class-loading to only load the correct one. But in a powerful system like Drupal/Civi, it's really hard to provide this guarantee, and the other party may not play-along.)

  • Second thought: it's worth noting that -- as an extension author -- these two aren't mutually exclusive. You can do both: (a) add composer.json to your repo, (b) register on packagist.org, and (c) setup CI and publish zip files via civicrm.org. Old-school site-builders can get the package through civicrm.org/zip-files; new-school site-builders can get them through packagist. – Tim Otten Apr 29 '16 at 4:05
  • "Unfortunately, there is a risk that your dependencies will conflict with others'. There's not a whole lot you can do about this... except pick obscure dependencies that aren't likely to conflict." -> so two extensions that use different versions of the same package are quite possiblly incompatible? I was imagining that is a problem that namespacing could solve... – michaelmcandrew Jan 25 '17 at 20:48
  • That's fine for a dependency written specifically to work that way, but it's pretty atypical for a PHP library to handle versioning through their namespaces -- and changing third-party libraries to use that technique would be a hard sell because it would be a compatibility-break. – Tim Otten Jan 25 '17 at 20:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.