Joomla:Release procedure and checklist

Content is Incomplete

This is a general checklist for managing releases of Joomla! and the Joomla! Framework. This list is best suited for a Minor Release or Maintenance Release, however the steps will generally apply to a Major Release as well but will require some judgement calls from the release manager and/or leadership team.

This document is a work in progress. The information on this page is currently undergoing a major revision, the previous version of this document is available below as a point of reference for these revisions.

CMS Release Cycle

Issue Triage, Patch Review, Merging Patches

The release cycle for a new software version begins immediately after the previous one is completed. In the immediate hours and days after a release, the issue tracker and support forums, in close coordination with the Bug Squad and CMS Release Team, should be monitored for reports of potential regressions or critical bugs that would warrant a quick follow-on release to address the issue. It is generally suggested to wait 48-72 hours before resuming merging of reviewed and ready for commit pull requests to keep the release branch "clean" and ready for a release (note this is only a suggestion, if need be a release manager could create a branch from the release tag however this involves a bit more effort therefore this suggestion is for simplicity's sake).

Throughout the release cycle, the release manager should be involved with reviewing patches and merging reviewed patches to the appropriate release branch. As of this writing (where 3.9.1 is the present release), the following branches are generally active:

• staging: This is the main release branch and is where most activity occurs, generally only bug fixes and "general" maintenance work should merge to this branch although minor features may be merged at the discretion of the release manager and/or leadership team
• 3.10-dev: This is the release branch for the next minor release (the branch's name will always be the version it represents), new features should be merged to this branch
• 4.0-dev: This is the release branch for the next major release (the branch's name will always be the version it represents)

Branch Merging

All branches should remain in-sync and all work should be regularly merged forward. Given the above example, the following is the recommended merging workflow (in general, the suggested workflow is that staging is merged to the next minor branch and the next minor branch is merged to the next major branch):

• staging should be merged into 3.10-dev
• 3.10-dev should be merged to 4.0-dev

It is suggested that at least once per week branch merging occurs.

Preparing To Release

When scheduling and preparing a release, the timeline should be communicated with the Production and Marketing & Communication Department leadership so that all teams are prepared to support the release. Note that when scheduling releases, consideration should be given to holidays around the world and the impact of issuing a release near those periods considered. Helpful resources include http://interfaith-calendar.org/ and https://www.timeanddate.com/holidays/

Release Schedule

Minor Release

Given the nature of a minor release, the following is the suggested release timeline:

• Six weeks prior to planned stable release: Tag the first Beta release
  • At this point the release should be considered feature complete and new features should be deferred to the next minor release
• Weekly Beta releases should follow this until the Release Candidate
• Two weeks prior to planned stable release: Tag the Release Candidate
  • At this point the release branch (generally staging) should remain closed for committing except for high priority bug fixes and language updates
  • With the Release Candidate, language freeze is also in effect and no language changes are permitted unless there is an urgent need (which should be communicated with the translations coordinator); note that if necessary the language freeze may be declared earlier but the Release Candidate should be considered the "no later than" time
• One week prior to planned stable release: If necessary, create a second Release Candidate
• Planned release date: Tag the stable release

Maintenance Release

Maintenance releases operate on a 4-6 week cycle and the following is the suggested release timeline:

• 4-5 days before planned release date: Tag a Release Candidate
  • With the Release Candidate, language freeze is also in effect and no language changes are permitted unless there is an urgent need (which should be communicated with the translations coordinator); note that if necessary the language freeze may be declared earlier but the Release Candidate should be considered the "no later than" time
• Planned release date: Tag the stable release

Reviewing Release Changes

Before tagging a release, the list of changes in the release should be reviewed by the release manager and/or leadership team. The list of changes can be found using GitHub's compare function with the URL https://github.com/joomla/joomla-cms/compare/<version>...staging which will compare the changes from <version> to the current staging branch (for example, https://github.com/joomla/joomla-cms/compare/3.9.0...3.9.1-rc is the comparison from the 3.9.0 release tag to the 3.9.1-rc tag). Key purposes of this review include:

• Ensuring all patches are compliant with Semantic Versioning (SemVer) (i.e. no backward compatibility breaks in minor or maintenance releases and no features in maintenance releases); SemVer violations should be cleared with the Production Department leadership
• Ensuring all code revisions are compatible with Joomla's minimum supported versions (i.e. no PHP 5.4 syntax in 3.x releases where PHP 5.3 is the minimum supported version, exceptions may be made if introducing a code path and files that will only be executed on newer PHP versions)
  • Joomla's testing integrations should not be relied on in full for this as there are large portions of the Joomla code base which do not have automated test coverage, and as of this writing, there is not a code linting process in place checking for potential issues
• Ensuring features and "significant" changes are documented

Security Fixes

From time to time, security fixes must be included in a release. Prior to every release, the release manager should check with the Security Strike Team to determine if security fixes are required, and if so ensure this is communicated to all members involved with release work (i.e. the marketing team liaison who assists with the publication of release related material). When a security release is necessary, an internal Release Candidate should be built for testing by the Security Strike Team, CMS Release Team, and CMS Maintenance Team with all security patches included. These teams should also receive information about the vulnerabilities being addressed to ensure there are no regressions in the impacted areas.

Note, when a security fix involves a Joomla Framework package, the Framework package should be patched first and a release created of the affected package then the CMS updated with composer update.

Building a Release

Building a release starts with checking that any files and folders that have been deleted are included in the arrays in the installer script at administrator/components/com_admin/script.php. There is a helper script in the repository to assist with this located at build/deleted_file_check.php. Running php build/deleted_file_check.php with its required --from option will write a file to build/deleted_files.txt with the list of files deleted since the from reference. Please see the PHP file for all documentation about this script.

The following steps require the individual to create commits and tags. It is suggested that these commits and tags be GPG signed as an extra security step but this is not a mandatory requirement as of this writing.

If creating a security release, at this point you should apply all patches provided by the security team.

Once it is confirmed that the deleted files listing is correct, the build/bump.php script should be executed to set the new version number. This script will also perform maintenance steps such as correcting the copyright year ranges for all files and replacing the __DEPLOY_VERSION__ placeholder. Please see the file for all documentation about this script.

After running this script, review all changes with git diff as a final check before creating your release commit. Once you are satisfied that all changes are correct, commit your changes locally. Once committed, create the release tag locally; all release tags should be the version number you are creating and should match the output of Joomla\CMS\Version::getShortVersion() (so if you have created 3.9.0-beta2 then the tag should be named 3.9.0-beta2).

Once the tag has been created, you will run the build/build.php script to generate the release packages. To ensure a clean build is produced, the script makes an archive of the tag you are building using the git archive command and extracts this to the build/tmp/<timestamp> directory. Depending on the patch version number, the following packages will be created in the build/tmp/packages:

• 0: Full installation and upgrade packages
• 1: Full installation, upgrade packages, and patch packages for x.y.0 to x.y.1
• 2+: Full installation, upgrade packages, patch packages for x.y.0 to x.y.<new> and x.y.<new-1> to x.y.<new> (i.e. when creating 3.8.8 this will create a patch package suitable for all 3.8.x releases and an optimized patch package for 3.8.7 to 3.8.8)

Additionally, the script will generate a build/tmp/checksums.txt file with the checksums for all generated packages.

After the packages are created, you should validate that you are able to install and update a site using the ZIP packages (as these are the most frequently used type and served by our update system).

Publishing a Release

Release Packages

Depending on the type of release being created, you will publish the release packages in different locations.

Alpha, Beta, Release Candidate

The packages for these releases are uploaded to GitHub. When ready to publish, you should push the release tag you created to GitHub and use their interface to create a release from the tag and upload the packages. There is a text template used for the releases, generally you can copy a previous release's text and adjust it to the version you are publishing.

Stable

The packages for stable releases are uploaded to multiple locations, this is in part driven by the fact that the Amazon Web Services (AWS) S3 region used by Joomla is blocked in some locations therefore download mirrors are available when necessary.

Joomla! Downloads Portal

The canonical source for all stable releases is the Joomla! Downloads Portal. The packages served from the site use Joomla's AWS account. You will need to log into the AWS console to upload the packages to S3 under the joomla-official-downloads bucket. The exact path inside the bucket is joomladownloads/joomla<version> where <version> is the major version number (i.e. joomla3), note for compatibility with Akeeba Release System folders for each individual release are not used.

Joomla! Update Server

The Joomla! Update Server is used as a mirror for the download packages. After connecting to the server, you should create a new folder at www/releases/<version> (i.e. www/releases/3.9.1 for the 3.9.1 release) and upload the packages here.

GitHub

The packages are also uploaded to GitHub. This is similar to the steps for alpha, beta, and release candidate releases above.

Old Version

This is a general checklist when a version of Joomla! is released. Keep in mind that a major release differs a lot from a Minor Release or even a Maintenance Release. The checklist here describes the steps that need to be done for (at least) Maintenance Release and Minor Release.

Release checklist

It depends on the Development Cycle when the checklist is triggered. A release can be done during every stage of the Development Cycle, it does not matter if you release a beta or a stable version, this is a general checklist that can be used when releasing a new version of Joomla! The checklist starts when it's decided to release a version:

Preparation phase

1. Communication pre-release: check with PLT on timing
2. Communication pre-release: inform Bug Squad and LT's about timing
3. Decision: when the above has positive result, set a date and time for release
4. [Communication pre-release: inform Global Forum Moderators about upcoming release]
5. Communication pre-release: check availability and assign release tasks to JBS and PLT members
6. Minimum 14 days prior to a minor release: freeze trunk for language changes
7. Minimum 7 days prior to a patch release: freeze trunk for language changes
8. Minimum 3 days prior to release: freeze trunk for all changes, draft release notes to translation teams, package release candidate for testing
9. Day of release: add the new release on Joomlacode and upload the files, update web pages
10. Announce on forum, announce to LT's
11. Update specific web pages with current version data
12. Contact caped crusader for updating Joomla related sites
13. Update the sites

Pre-release phase

1. Apply latest installation translations (need to be delivered by translation work group Thomas Hunziker).
2. Create pre-release package for testing.
3. Offer pre-package to testers: Bug Squad members, Development Team members, translation members, and JTesters
4. If problems are found during this stage, go back to fix the problem. Use the Joomla! Maintenance Procedures and repeat until tests are performed successfully.
5. Draft the release notes on Joomla.org and post draft to the private translation forum.
6. Obtain any CVE numbers needed.
7. Draft security notices as needed and leave unpublished on Joomla! Developer Network.

Release phase

1. Check with JSST for security fixes. *
2. Create release notes document on joomla.org.
3. Run the bump.php script
4. Check language/en-GB/en-GB.files_joomla.sys.ini file for version number (not needed for minor update).
5. Commit and tag release locally
6. Package build by running build/build.php
7. Packages are now in the build/tmp folder
8. Push to GitHub
9. If problems are found during this stage, go back to fix the problem. Use the Joomla! Maintenance Procedures and repeat until tests are performed successfully.
10. Add package to GitHub
    1. This must be done after the version has been tagged and the tag pushed to GitHub due to how the Releases system works
    2. After pushing the tag, edit the release and attach all of the release packages to the release.
    3. Copy the description from a previous release and update for the new release.
11. Update XML file on update.joomla.org site for the new version number information.
    1. FTP connect to update1.joomla.org and Navigate to www/core
    2. All releases
       1. Change version attribute in list.xml (don't add new line except when incrementing major/minor version numbers)
    3. LTS Release
       1. Change the update elements for the series with the new version number and new downloadurl and link to new archive file to extension.xml
    4. STS Release
       1. Navigate to www/core/sts
       2. Change version attribute in list_sts.xml (don't add new line except when incrementing major/minor version numbers)
       3. Change the update elements for the series with the new version number, new downloadurl and link to new archive file to extension_sts.xml, and the infourl linking to the release announcement
    5. Test auto update from prior version(s) (make sure XML files have been copied to the CDN on update.joomla.org first)
       1. All releases
          1. Navigate to www/core/test
          2. Change version attribute in list_test.xml (don't add new line except when incrementing major/minor version numbers)
       2. LTS
          1. Upload patch package to this folder
       3. Change the update elements for the series with the new version number, new downloadurl and link to new archive file to extension_sts.xml, and the infourl linking to the release announcement
       4. STS
          1. Navigate to www/core/teststs
          2. Upload patch package to this folder
       5. Change the update elements for the series with the new version number, new downloadurl and link to new archive file to extension_sts.xml, and the infourl linking to the release announcement
12. Update Joomla Download Portal with new release data *
13. Publish security articles on developer site. *
14. Publish release announcement on Joomla.org
15. Publish announcement on the forum [Peter Martin]
16. Update Wiki: FAQ, Version history [Mat]
17. Post to leadership in Glip
18. Create the Web Platform Installer packs and submit them *
19. Add package to joomlacode.org
    1. Before the release is public, set the JoomlaCode package flags as follows: Visible=No, Public=No, Require Login=Yes.
    2. Set the Release flags as follows: Visible=Yes, Released=Yes.
    3. Once package is released, change the Package flags to Visible=Yes, Public=Yes, Require Login=No.
    4. Change the prior version package to Visible=No but leave the prior version release flags as they are.

Note: Items with an asterisk can be skipped for non-stable releases

Post-release & Finalization phase

1. Update GitHub repo for new version development
   1. Run build/bump.php with next version
2. Delete old release branch after confirming all commits are merged to series development head and there are no open pull requests
3. Close security issues in security tracker

Pages to update

1. Global:
   • Development Status
   • Downloads Portal

Sites to update

1. PLT Owned sites
   • Developer Site
   • Downloads Portal
   • API Site
     • Regenerate API docs for the new release via Jenkins

Calculating Release Statistics

Using JoomlaCode and CHANGELOG

1. Extract *all* issues from Tracker into spreadsheet. Make note of the date of the last release and the number of active issues at the time of the last release.
2. Set a filter on the "Closed Date" column to be "after" "the day of the last release".
   • Calculate "Closed" count by aggregating status values for Closed, Duplicate Report, Known Issue, Not a bug, Not Joomla! Core, Unable to Confirm.
   • Calculate "Fixed in SVN" count by aggregating status values for Open and Information Required. (Note: this value is presented two times in the report.)
3. Set a filter on the "Closed Date" column to be blanks:
   • Calculate "Open" count by aggregating status values for "Open" and "Information Required."
   • Calculate "Confirmed" count by aggregating status values for "Confirmed" and "In Progress."
   • Calculate "Pending" count by aggregating status values for "Pending" and "Ready to Commit."
4. Calculate the "New Active Issues" by aggregating the "Open", "Confirmed" and "Pending" counts.
5. Calculate the "net increase/decrease" by subtracting the "Last Release Active Issues" from the "New Active Issues."
6. Count the "number of commits" by reviewing the CHANGELOG.php file and counting the documented commits.

Using GitHub

1. Go to https://github.com/joomla/joomla-cms/compare/<version>...master where <version> is the previous release, this is GitHub's comparison view and will give stats from the previous release tag to the currently published master
2. From this page, you can compile the number of commits, contributors, and files changed during the release period

Download Statistics

1. Go to http://joomlacode.org/gf/project/joomla/frs/?action=FrsReport
2. In the filter bar, set the start date to the date of the previous release and the end date to the day before the new release
3. The "Downloads By Package" chart is the total number of downloads for each release (i.e. 2.5.16 and 3.2.0) and includes an "Other" piece for all older version downloads
4. The "Downloads By Release" chart separates the packages by new install and updates

Alternative Method for Stats

• Fixed in SVN: Create query with close date > date of prior release and status = Fixed in SVN.
• Commits: Count from CHANGELOG.php as indicated above.
• New Reports: Run query with all status codes and open date > date of prior release.
• Increase / decrease in active issues: See note below
• Closed: Run query with closed date > date of prior release and status code one of Closed, Duplicate Report, Known Issue, Not a bug, Not Joomla! Core, Unable to Confirm.
• Open at time of release: query of all issues with status = Open or Information Required
• Confirmed at time of release: query of all issues with status = Confirmed or In Progress
• Pending at time of release: query of all issues with status = Pending or Ready to Commit

Increase/Decrease in Active Issues

• Total active issues now = Open at time of release + Confirmed at time of release + Pending at time of release
• This number minus the total active as of prior release is the net increase or decrease

As a check, make sure the following formula works:

• Prior release total active issues + New Reports - (Closed + Fixed in SVN) should equal the Total Active Issues now.