9

We would like to help test the monthly CiviCRM Release Candidate (RC) versions to contribute to more reliable releases. I found out the RC are published here along with the stable releases, so e.g. when CiviCRM 5.2.0 is released, it's accompanied by a RC for 5.3.0.

Now, in order to do focused and meaningful testing, we need to find out which changes went into the Release Candidate so we know which functionalities we should scrutinize most. Where can we get that information?

9

This is an excellent question. Before trying to answer, let me note a bit of the context:

  • When defining a test plan, several organizational and technical factors can weigh-in (e.g. perhaps certain extensions or use-cases are more important or more within your expertise; perhaps there's a rotation of responsibilities). The question focused on how to examine the change-list -- that's not the only factor, but it is an important one, so we should have a good answer.
  • The RC releases and general releases are lined up on rolling monthly cycles. You can get more specifics by reading the schedule of releases. I usually think about the schedule in reverse (from the release date). For example, v5.3.0 will be released at the start of July. The testing period for the RC (v5.3.beta1) is the previous month, June. The development period for v5.3.alpha1 is the month before that, May.
  • CiviCRM development is managed in Git/Github -- this is where changes are submitted and reviewed. Most reports about changes are derived from this data.
  • In a typical monthly cycle, there are 80-150 accepted proposals (aka "PR"s or "pull-requests").

There is no one perfect way to list changes during the RC period, but there are a few tools/techniques available.

Release Notes: This is a curated document (usually written by Andrew Hunt and Alice Frumin).

Github PR Queue: All changes are proposed in Github by filing a "pull request" (PR).

  • Good:
    • The full list is available when you want it, and it can be loaded in a web-browser.
  • Bad:
    • The filter options aren't good for your use-case. You need to try a few different filters.
    • You may need to check multiple repos (civicrm-core is the main one; but sometimes interesting things happen in civicrm-packages, civicrm-drupal, civicrm-wordpress, civicrm-backdrop, civicrm-joomla).
  • Examples:
  • Comments:
    • We have an open issue to improve the infrastructure so that it's easier to filter Github data based the target version ("Milestone").

Git: Every individual change and every PR approval is available in the git database.

  • Good:
    • This is the most accurate and flexible way to filter a change list. It is always up-to-date.
  • Bad:
    • This requires more technical proficiency -- e.g. understanding git and the Unix CLI.
    • The text in git is written by many different developers. The text varies a lot in terms of length/language/quality.

This is more powerful and more technical, so I want to give a longer discussion of examples.

Git allows you to display developer notes over a range of versions. For example, when studying v5.3, you might to view notes after 5.2 and before 5.3. This range is called 5.2..5.3 -- or, more specifically, origin/5.2..origin/5.3.

You can review developer notes for the range:

# Get a list of all developer notes.
git log origin/5.2..origin/5.3

Additionally, you can filter based on the affected code/subsystem.

# Get a list of all developer notes relevant to the table definitions
git log origin/5.2..origin/5.3 -- xml/schema/

# Get a list of all developer notes relevant to the DB upgrade steps
git log origin/5.2..origin/5.3 -- CRM/Upgrade/

# Get a list of all developer notes relevant to the "Financial" subsystem
git log origin/5.2..origin/5.3 -- CRM/Financial templates/CRM/Financial xml/schema/Financial

# Get a list of all developer notes relevant to the APIv3 framework
git log origin/5.2..origin/5.3 api/ CRM/Utils/API/ Civi/API/

All those examples use git log to show developer notes, but the same filters (by range and by folder) can produce other types of output. Compare:

# Get a list of developer notes for a folder (as above)
git log origin/5.2..origin/5.3 -- CRM/Upgrade/

# Get the same list and display it in a GUI tool (on local Linux/OSX workstation)
gitk origin/5.2..origin/5.3 -- CRM/Upgrade/

# Get the same list and display in short, one-line, summary format
git log --oneline origin/5.2..origin/5.3 -- CRM/Upgrade/

# Get the same list and include the relevant code changes
git log -p origin/5.2..origin/5.3 -- CRM/Upgrade/

# Get cumulative diff reflecting all those changes
git diff origin/5.2..origin/5.3 -- CRM/Upgrade/

If you're a CLI guru, then you can do further filtering. This is usually easiest with the --oneline format.

# Get a summary list which identifies the approved pull requests
git log origin/5.2..origin/5.3 --oneline | grep 'Merge pull'

# As above, but only show the PR#
git log origin/5.2..origin/5.3 --oneline | grep 'Merge pull' | awk '{print $5}'

# As above, but show the URL for each PR
git log origin/5.2..origin/5.3 --oneline | grep 'Merge pull' | awk '{print $5}' | sed 's;#;https://github.com/civicrm/civicrm-core/pull/;'

Of course, there may be other tools, but that's the one I use on a day-to-day basis to examine change history.

(Update) If this kind of reporting is useful on a regular basis, then you might want to put these commands in a bash script -- then you can quickly re-run the same report. Here is an example of a custom report.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.