Releasing¶
Krini follows semantic versioning. Features and
fixes are merged into dev. Branch main only ever moves
forward to commits on dev that are released, and release tags are on
main. Releases are cut by the maintainers with the release workflow
(.github/workflows/release.yml), which publishes the image
docker.io/elixircloud/krini:X.Y.Z and the GitHub release, with the Helm
chart attached (krini-X.Y.Z.tgz, and its Sigstore bundle
krini-X.Y.Z.tgz.sigstore.json). Until 1.0.0,
all releases are GitHub pre-releases; stable versions (without -alphaN,
-betaN or -rcN) still update the image tag latest.
The tag is the only source of the chart's version: the release workflow
packages deployment/ with version and appVersion set to the release
version (scripts/package-chart.sh), so deployment/Chart.yaml keeps its
placeholders (0.0.0 and dev, the image of the newest commit on dev)
and is never bumped in a release-prep pull request. The packaged chart
deploys the release's image by default (appVersion is its tag).
Steps¶
-
Shortly before the release, open a release-prep pull request into
dev: add the curated release notes asdocs/releases/<version>.mdand tonavinmkdocs.yml(underReleases, newest first; the strict docs build fails on pages missing fromnav), set the version inpackage.jsonand inCITATION.cff(version; add or updatedate-released), and update the authors inCITATION.cff(see "Citation metadata" in the contributing guide). Each version, also each pre-release (e.g.,X.Y.Z-rc1), needs its own such change. CI'sRelease readinessjob checks it, on every pull request and push: the versions inpackage.jsonandCITATION.cffmust always agree, and unless the version is tagged on an earlier commit, adate-released, release notes or a version bump after an earlier release start the release preparation: thendate-releasedmust be set and the notes must exist, without TODOs, in the organization's format. Merge it.The first release-prep pull request adds the section
Releasesat the end ofnav; later ones add their notes at its top:The release workflow publishes the notes as the start of the GitHub release, so they follow the organization's format: an intro paragraph saying what the release brings, with a link to the upgrade notes if operators must act, then the sections
## Breaking changes,## Features,## Security,## Fixes,## Documentationand## Maintenance(in this order, empty ones left out), holding bullets only, each ending with its issue and pull request numbers, e.g.,(#12, #34). Links must be absolute, as they must also work on GitHub; no## Commitssection and no**Full Changelog**line, which the workflow adds. The shared actionpublish-releasedescribes the format and its checks. -
Wait until CI has passed for the push to
devof the commit that the merge created ondev(<sha>, not the head of the pull request): the release workflow requires a successful CI run for a push of the tagged commit (runs for pull requests test merge commits, not the commit itself). CI also scans the image with the flags of the release scan, so that a fixable critical or high vulnerability fails CI before the tag rather than the release; fix it (e.g., by bumping the Bun image of theDockerfile) or, with a reason and an expiry date, accept it in.trivyignore.yaml. Vulnerabilities published between the CI run and the release can still fail the release scan: tag soon after CI has passed. -
Run the release workflow's checks that need no tag, as it runs them (keep these commands in sync with the
preparejob of.github/workflows/release.yml), in Bash, withgit,jq,yq(mikefarah/yq v4),curl, Python 3,bun,tar,sha256sum, Helm 3.22.1 (the version ofrelease.yml) and the GitHub CLI. Run them before tagging: tags can't be moved or deleted, and the release workflow checks the tagged commit, so a check that fails after the tag is pushed (other than a CI run that hasn't finished yet) fails again on re-runs, and the release then needs a new version, with its own release-prep pull request. The commands expectoriginto beelixir-cloud-aai/krini(replace it with the name of your remote for it otherwise).version=X.Y.Z # e.g. 0.1.0 sha="" git fetch origin && sha="$(git rev-parse --verify "<sha>^{commit}")" # the full SHA ( [[ -n "${sha}" ]] || { echo "Fetch failed, or the commit is unknown"; exit 1; } git merge-base --is-ancestor "${sha}" origin/dev || { echo "${sha} is not on dev"; exit 1; } pattern='^v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-(alpha|beta|rc)[1-9][0-9]*)?$' [[ "v${version}" =~ ${pattern} ]] || { echo "Not a release version: v${version}"; exit 1; } # The files of the commit, not of the working tree [[ "$(git show "${sha}:package.json" | jq -r .version)" == "${version}" ]] || { echo "Wrong version in package.json"; exit 1; } [[ "$(git show "${sha}:CITATION.cff" | yq -r .version)" == "${version}" ]] || { echo "Wrong version in CITATION.cff"; exit 1; } [[ -n "$(git show "${sha}:CITATION.cff" | yq -r '.date-released // ""')" ]] || { echo "No date-released in CITATION.cff"; exit 1; } # The notes of the commit, with the checker of the action that # `release.yml` pins (as the `readiness` job of `ci.yml` does) notes="$(mktemp -d)" && git show "${sha}:docs/releases/${version}.md" > "${notes}/notes.md" && curl -fsSL "https://raw.githubusercontent.com/elixir-cloud-aai/.github/bf3840c9c072aaf37d6cafe199aaa1994087d1ef/actions/publish-release/publish.py" -o "${notes}/publish.py" && python3 -I "${notes}/publish.py" check "${notes}/notes.md" || { echo "Release notes docs/releases/${version}.md missing or not in the format"; exit 1; } # The chart of the commit, packaged and checked as the release # workflow packages it (version from the tag, lint, render); in a # fork, use the image of its DOCKERHUB_ORG chart="$(mktemp -d)" && git archive "${sha}" deployment scripts/package-chart.sh | tar -x -C "${chart}" && "${chart}/scripts/package-chart.sh" "${version}" docker.io/elixircloud/krini "${chart}/dist" > /dev/null || { echo "The chart can't be packaged"; exit 1; } # Also drafts, which the release workflow can't see before it fails on them release="$(gh release view "v${version}" --repo elixir-cloud-aai/krini 2>&1)" && { echo "A release or draft of v${version} exists: delete a draft"; exit 1; } [[ "${release}" == *"release not found"* ]] || { echo "Cannot check the releases: ${release}"; exit 1; } runs="$(gh api "repos/elixir-cloud-aai/krini/actions/workflows/ci.yml/runs?head_sha=${sha}&branch=dev&event=push&status=success&per_page=1" --jq .total_count)" (( runs >= 1 )) || { echo "No successful CI run for the push of ${sha} to dev"; exit 1; } git merge-base --is-ancestor origin/main "${sha}" || { echo "main can't be fast-forwarded to ${sha}"; exit 1; } # Not in the release workflow: advisories published since the CI run # (the `audit` job of `ci.yml`), in the lockfile of the commit audit="$(mktemp -d)" && git show "${sha}:package.json" > "${audit}/package.json" && git show "${sha}:bun.lock" > "${audit}/bun.lock" || { echo "Cannot read the lockfile of ${sha}"; exit 1; } (cd "${audit}" && bun audit --prod --audit-level=high) || { echo "High or critical advisories in the production dependencies"; exit 1; } ) && echo "All checks passed"Only if all checks passed, tag the commit with a signed, annotated tag
vX.Y.Z(orvX.Y.Z-alphaN,vX.Y.Z-betaNorvX.Y.Z-rcNfor a pre-release), and push it together withmain, moved to the commit, in one atomic push: either both refs are updated or neither (e.g. ifmaincan't be fast-forwarded or a ruleset rejects the tag), so that neither a tag on a commit that isn't onmain(which would use up the version) nor an untaggedmaincan result. If the push fails, delete the local tag (git tag -d "v${version}") before you try again.git tag -s "v${version}" "${sha}" -m "v${version}" && git push --atomic origin "${sha}:refs/heads/main" "refs/tags/v${version}"No workflow of this repository deploys
main, but a hosting project may follow it (a Vercel project follows this repository): check its settings before you movemain. -
Approve the jobs of the release workflow in the
releaseenvironment, after checking that the run is for the tag you pushed. It checks that GitHub verified the tag's signature, that its commit is onmain, the versions, the curated notes and the CI run, and packages the chart (Prepare release); builds and pushes the image by digest (Push release image, needs approval); scans exactly that digest (Scan release image) and runs the smoke test of CI on it (Smoke test,scripts/check-image.sh, and the end-to-end tests of CI,End-to-end tests (release image)); attests and signs it, attests and signs the chart, and only then tags the imageX.Y.Z(andlatestfor the highest stable version), so that a failure before leaves the image untagged (Release image, needs approval); and then, without another approval, creates the GitHub release (Publish release): the curated notes of the tagged commit, then GitHub's generated notes under## Commits(without their## What's Changedheader, with "New Contributors"), and GitHub's**Full Changelog**line last, with the chart and its Sigstore bundle attached (it creates a draft, uploads them, and publishes it only then, as assets of immutable releases can't be added later; if that fails, the draft is deleted); versions below 1.0.0 and-alpha,-betaand-rcversions are pre-releases. Releases aren't edited after publishing (by convention): fix mistakes in the notes of the next release. If a job fails, use "Re-run failed jobs", not "Re-run all jobs": once the image is tagged, a new attempt of all jobs stops at the existing tag and can't finish the release, and once the GitHub release exists,preparefails; a re-run ofPublish releasepasses without changes if an earlier attempt created the release after all.preparecan't see draft releases andPublish releasefails on them, so delete any draft of the tag before you push the tag. IfPublish releasewas cancelled (or lost its runner) after it created the draft, the draft stays, and a re-run fails on it: delete the draft (gh release delete v<version> --repo elixir-cloud-aai/krini, without--cleanup-tag, which would delete the tag) and re-run the failed jobs. A draft isn't a release, so deleting it is safe. Ifpreparefails because CI hasn't finished for the commit, re-run the workflow once it passed.
Repository Settings¶
The repository needs, once:
- an environment
releasewith a required reviewer, deployments limited tov*tags, and the Docker Hub secretsDOCKERHUB_LOGINandDOCKERHUB_TOKEN(allowed to pushelixircloud/krini), only in that environment, not as repository secrets; - optionally the variable
DOCKERHUB_ORGfor another Docker Hub organization (forks: set it before pushing a test tag, as the workflows fall back toelixircloud); the packaged chart then deploys that organization's image by default; - a ruleset for
devthat requires the checkCI passed(the gate job ofci.yml); a ruleset formainthat lets only maintainers update it and protects it against non-fast-forward pushes; a ruleset that lets only maintainers createv*tags and keeps them from being moved or deleted; and immutable releases.
Tags and published releases and images can't be changed: if a release fails after its tag was pushed and re-running its failed jobs doesn't help, release the next patch version (with its own release-prep pull request).
Scheduled Scan¶
After a release, the workflow Scan (.github/workflows/scan.yml) scans
the image of the latest release (or, before the first stable release, of
the newest pre-release) every Monday, and can be run by hand. It uploads
the fixable findings to code scanning and a full report as a workflow
artifact, and fails on fixable critical and high vulnerabilities: release a
patch version with the fix.