How to develop extension that relies on libs brought in by composer

Question

I'm wanting to (re)develop an extension and I'd like to use some 3rd party libraries, eg. guzzle, that I would normally manage via composer.

So my github repo will contain a composer.json file, but not the vendor directory. So would I need to package installable versions with the libraries included?

Also, what if I were to install libraries that are already included (eg dependencies of the libraries I want might be already in use leading to a namespace clash).

Is there a workflow for this sort of thing?

Answer 1

Approaches

There are a few different patterns one could follow in connecting Civi extensions with composer:

1. (Broad scope) Site build via composer: The entire web site is defined with a custom composer.json file. It doesn't matter what kind of dependency you need to pull in (CMS, CMS module, CMS theme, CRM, CRM extension, PHP library, CSS library, etc), it's all brought in through the same tools/workflows.
2. (Medium scope) Use composer for the extensionDir: Civi has a special folder (e.g. sites/default/files/civicrm/ext) for storing downloaded extensions. We could keep this folder but manage it differently -- e.g. rewrite CRM_Extension_Downloader as a wrapper for composer.
3. (Narrow scope) Intra-extension dependency: If one particular extension requires a particular library, then the extension could have its own copy of composer.json and vendor/. The final zip file would embed vendor/.

Site build via composer

IMHO, this is the holy-grail. It's consistent with the architecture of PHP and composer, and it resolves problems that afflict dependency-management in Drupal (libraries/ guffaw!) and CiviCRM (dependencies guffaw!) and probably Joomla/WordPress.

It's worth drilling down on the architecture concern for a moment. Civi is deployed as a PHP app which shares a runtime with another PHP app. In PHP, the runtime can only load one version of a library -- e.g. you can load Guzzle v5.0 XOR v5.1 but not both, so it's important to pick a version of Guzzle that is acceptable to all libraries/modules/extensions. When using composer in narrow/piecemeal fashion, it doesn't get the full list of libraries/modules/extensions, so it cannot help you pick a version of Guzzle that satisfies the overall system. You're more likely to suffer from hidden version-conflicts (if, e.g., a Drupal module loads Guzzle v5.0 while your Civi extension loads Guzzle v5.1).

The main challenge is that many of the relevant systems (Drupal, Joomla, WordPress, Civi, etc) predate composer and have some impedance mismatch with composer (e.g. composer lacks a GUI; Drupal has multisite PHP loading; post-install hooks work differently). Getting these communities to switch is a major undertaking.

I think there are some fairly clear steps we can take toward supporting this model, but your question was really aimed at a different approach...

Intra-extension dependency

No one controls enough of the ecosystem to quickly switch a critical-mass of site-building to composer. And this version-conflict stuff may sound theoretical. In fact, you can get some mileage out of piecemeal dependency management. (Case in point: civicrm-*.tar.gz ships with its own bundled copy of vendor/!) And your comments indicate an interest in this approach.

Unfortunately, you're basically on-your-own to make this work. Loosely, I'd try:

• In the myextension.git repo, add a composer.json file which lists any dependencies. Use .gitignore to avoid committing vendor/. In myextension.php, use require_once __DIR__.'/vendor/autoload.php';.
• When registering your extension on civicrm.org, do not specify a Github URL. Neither github.com nor civicrm.org is smart enough to build your releases automatically.
• Setup a CI tool which monitors your git repo and publishes the final zip archive. As a very loose example, I publish the civix.phar file with Jenkins (job, script). You could write a similar script to run composer install, zip, and then publish the final file.
• Whenever you want to publish a new release, login to civicrm.org and manually create a new release node (with suitable info.xml and download URL).

Unfortunately, there is a risk that your dependencies will conflict with others'. There's not a whole lot you can do about this... except pick obscure dependencies that aren't likely to conflict.

(In theory, if you know that your extension and the conflicted one never use the conflicted dependencies within the same page-request, then you could do some fiddly bits in the class-loading to only load the correct one. But in a powerful system like Drupal/Civi, it's really hard to provide this guarantee, and the other party may not play-along.)