5

Drupal has a project called "S3 filesystem" which allows a site to put some of it's "files" on a remote S3 bucket (or other object-based filesystem). It works because Drupal accesses files through a pluggable api.

It appears that most civicrm access of its "local data files" is now programmatic, i.e. those files are rarely accessed based on an assumption of where they are relative to the document root, so it seems like it might not be too far off that all of civicrm core's file access could be pluggable and a civicrm extension similar to the s3 filesystem possible.

Any thoughts about this? I'm particularly interested in the files in customFileUploadDir.

2

1 Answer 1

2
  1. Tim has answered this (as of Apr 2023) here:

    https://lab.civicrm.org/dev/core/-/issues/4251#note_90037

    Short version is: yes, a more consistent api for file serving is under consideration, it's currently a bit of mess, but with an entertainingly written back story.

  2. For the narrower question of "how would I do something like this now", the answer would be to look at this file:

    https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/BAO/File.php

    which contains the methods for managing (uploading, moving, etc.) "entity files", i.e. those that are attached to an entity, which probably is the main scalability concern.

  3. @bgm has answered a different question of "how can I use S3 to host my CiviCRM uploaded files" with a straightforward solution of using the S3 Fuse filesystem. That might be the best way to solve my specific issue of scalability, by mounting a bucket to customFileUploadDir. The potential downside is that it relies on translating low level filesystem requests, which means some types of operations might be quite inefficient (e.g. big directory listings).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.