Helm chart¶
The Helm chart in
deployment/
deploys Krini on Kubernetes and OpenShift: a Deployment of the container
image, a Service, and an Ingress, a Gateway API HTTPRoute or an OpenShift
Route. It has no defaults for any deployment: you set the host name, the
Cloud Registry and the login providers, and the login providers' secrets
come only from a Kubernetes Secret of your own.
Each release, from
0.1.0 on, has the chart attached, packaged with the release's version
(krini-X.Y.Z.tgz), which deploys that release's image by default. The
chart isn't published to a chart repository: install it from that file,
after verifying it. The chart in the repository
(version 0.0.0) deploys the image of the newest commit on dev by
default; use it for development and testing only.
Prerequisites¶
- Kubernetes 1.27 or later (or OpenShift 4.14 or later) and Helm 3.
- One way to reach Krini from outside the cluster: an ingress controller, a Gateway of the Gateway API, or OpenShift's router.
- An OpenID Connect client at each login provider, with the redirect URI
https://<host name>/auth/api/callback, the scopes of Scopes, and the scope with which the provider releases the claim with the user's groups (groupsClaim), if access tiers are used: add it to the provider'sscopes(see Login providers). - A Cloud Registry whose list of services (
GET /services, without login) has the TES services and gateways (typestesandtes-gateway) and WES services that Krini should offer; Krini lists what the registry returns, so register only services that you have checked. Krini only reads the registry. - TES services, gateways and WES services that accept access tokens
issued to Krini's
clientIdof each login provider, and allow Krini's origin in their CORS settings: see Service configuration and CORS configuration.
Installation¶
-
Write the login providers (
LOGIN_PROVIDERS) to a file outside the repository, e.g.login-providers.json: a JSON array with one object per provider (see Login providers), for example: -
Create a Secret with it in the namespace of the release (or with your secrets manager, e.g. External Secrets or Sealed Secrets), then delete the file:
-
Write the values of the deployment to a file outside the repository, e.g.
values-prod.yaml:hostName: krini.example.org config: cloudRegistryUrl: https://registry.example.org privacy: controller: Example Institute, Example Street 1, Example City contact: privacy@example.org loginProviders: existingSecret: krini-login-providersThe chart of a release deploys the image of that release (see Image), so the values need no image tag. On OpenShift, add
krini: {ingress: {enabled: false}}, so that the chart creates a Route with TLS instead of an Ingress (see Exposing Krini). -
Download the chart of the release that you deploy (here 0.1.0) from its GitHub release, with the GitHub CLI (or from the release page), verify it, and install it:
version=0.1.0 gh release download "v${version}" --repo elixir-cloud-aai/krini \ --pattern "krini-${version}.tgz*" # verify it (see "Verifying the chart"), then: helm install krini "krini-${version}.tgz" -n krini -f values-prod.yamlkriniis the name of the release, which upgrades andhelm uninstallrefer to. Helm prints Krini's URL and how to check the rollout;helm get notes krini -n krinishows them again. -
Check the release:
The test requests the privacy notice and the start page from the Service, so it shows that Krini serves pages; it doesn't log in.
Rendering fails, with a message naming the value, if hostName (or
config.appOrigin), config.cloudRegistryUrl or
loginProviders.existingSecret is missing, if a value has the wrong type
or isn't known to the chart (values.schema.json), or if values contradict
each other.
Values¶
All values with their defaults and comments are in
deployment/values.yaml.
Configuration¶
The chart passes Krini's settings (see Configuration)
to its container as environment variables. Empty values aren't passed, so
that Krini's defaults apply. Changing a value restarts the pods on helm
upgrade.
| Value | Variable | Default |
|---|---|---|
hostName |
— | none; the public host name of the Ingress, HTTPRoute or Route, e.g. krini.example.org |
config.cloudRegistryUrl |
CLOUD_REGISTRY_URL |
none; required |
config.appOrigin |
APP_ORIGIN |
https://<hostName>; an https origin without path or trailing slash |
config.oidcRedirectUri |
OIDC_REDIRECT_URI |
not set: Krini uses <APP_ORIGIN>/auth/api/callback |
config.computeAccessRequestUrl |
COMPUTE_ACCESS_REQUEST_URL |
not set |
config.privacy.controller |
PRIVACY_CONTROLLER |
not set |
config.privacy.contact |
PRIVACY_CONTACT |
not set |
config.privacy.noticeUrl |
PRIVACY_NOTICE_URL |
not set |
config.privacy.loginNoticeUrl |
PRIVACY_LOGIN_NOTICE_URL |
not set |
loginProviders.existingSecret |
LOGIN_PROVIDERS |
none; required: the Secret with the login providers |
loginProviders.secretKey |
— | login-providers: the key of the Secret |
extraEnv |
any | []: further variables, as Kubernetes EnvVars |
The config values reach the browser or are public by design (see
Configuration): never put a secret into them.
Secrets go into Secrets, which loginProviders and extraEnv
(valueFrom.secretKeyRef) refer to.
extraEnv sets variables that the chart has no value for, e.g.
LISTEN_HOSTNAME for IPv6-only clusters:
Rendering fails if extraEnv sets a variable of the table above or PORT,
which the chart sets: use the chart's value instead.
Krini reads the Secret only when it starts: after changing it, restart the
pods with kubectl rollout restart deployment/krini -n krini.
Login providers¶
The key loginProviders.secretKey of the Secret loginProviders.existingSecret
holds the JSON array of LOGIN_PROVIDERS. Each provider is an object with
these fields; providers without one of the required fields, or with the
id of an earlier one, are ignored:
| Field | Required | Description |
|---|---|---|
id |
yes | Identifier of the provider, unique in the array |
issuer |
yes | OpenID Connect issuer URL (Krini reads its discovery document) |
clientId |
yes | Client ID of Krini at the provider |
clientSecret |
yes | Client secret of Krini at the provider |
name |
no | Name on the login page; the id if not set |
logo |
no | URL of a logo for the login page |
scopes |
no | Scopes to request; ["openid", "profile", "email", "offline_access"] if not set |
groupsClaim |
no | Claim with the user's groups, for the access tiers |
groupsFromUserinfo |
no | Whether to read the groups from the user info (default true) |
computeGroups |
no | Groups whose members may run tasks and workflows |
adminGroups |
no | Groups whose members are admins |
groupSubgroupSeparator |
no | Separator of subgroups in group names (e.g. :), so that members of subgroups count too |
Image¶
The image is krini.image (default docker.io/elixircloud/krini, fully
qualified, as some container runtimes refuse short names) with the tag
krini.tag. Without a tag, the chart deploys its appVersion: in the chart
of a release (krini-X.Y.Z.tgz), the image of that release (X.Y.Z); in
the chart in the repository, dev, the image of the newest commit on the
branch dev, which changes with every commit. Set krini.tag only to
deploy another image than the chart's, or include a digest in
krini.image (e.g. docker.io/elixircloud/krini@sha256:...; krini.tag
must then be empty) to pin the image; see Container
image for the tags and how to verify an
image.
The image is pulled whenever a pod starts (krini.imagePullPolicy: Always),
so that moving tags deploy their latest image; set IfNotPresent for
releases or digests, or Never for images loaded into the nodes (e.g. with
kind load).
Exposing Krini¶
The chart exposes Krini for hostName in one of three ways, routing all
paths to it:
- Ingress (
krini.ingress, enabled by default) for any ingress controller:classNamesets the Ingress class (default: the cluster's default class), andannotationssets annotations for the controller; they take precedence over the chart's, andnullremoves one of the chart's. The chart sets no annotations for particular controllers. By default, the Ingress has no TLS, so TLS must be terminated in front of the cluster (e.g. by a load balancer); withhttps.enabled: true, cert-manager issues a certificate via the ClusterIssuerhttps.issuer(required then, e.g.letsencrypt-prod), and the Ingress serves HTTPS with it (Secret<release>-krini-tls, orkrini-tlsfor a release namedkrini). -
Gateway API HTTPRoute (
krini.httpRoute), attached to an existing Gateway whose listener terminates TLS, with the annotations ofkrini.httpRoute.annotations; the Ingress must be disabled: -
OpenShift Route (
krini.route): if neither the Ingress nor the HTTPRoute is enabled and the cluster serves Routes, the chart creates a Route with TLS terminated at the router, which redirects HTTP to HTTPS.krini.route.annotationssets annotations of the Route, e.g. the router's timeouts (haproxy.router.openshift.io/timeout). Setkrini.route.enabled: falsefor a Route of your own. On OpenShift, disable the Ingress (krini.ingress.enabled: false), which is enabled by default: OpenShift turns a plain Ingress into a Route without TLS.
Krini must be served over HTTPS: it sets secure cookies, and sends users
back to APP_ORIGIN (https://<hostName>) after the login. If none of
them is enabled, Krini is reachable only within the cluster, and Helm's
notes say so.
Security¶
The pod runs with the restricted Pod Security
Standard:
as the image's non-root user bun (UID 1000, set as runAsUser so that
Kubernetes can check runAsNonRoot), without privilege escalation or
capabilities, with the seccomp profile RuntimeDefault and a read-only
root file system; Next.js's cache goes to an emptyDir at /tmp (up to
krini.tmpSizeLimit, default 256Mi; null for no limit). These are the
defaults of krini.podSecurityContext and krini.securityContext (see
values.yaml). The pod gets no service account token, as Krini
doesn't call the Kubernetes API.
On OpenShift (if the cluster serves security.openshift.io/v1), the chart
leaves out runAsUser, runAsGroup, fsGroup and seccompProfile: the
security context constraints assign a user ID from the namespace's range
and a seccomp profile, and some clusters reject others. The image runs
with any user ID. To render the chart for OpenShift without access to the
cluster (helm template), pass --api-versions security.openshift.io/v1
--api-versions route.openshift.io/v1. Override krini.podSecurityContext
and krini.securityContext for other platforms.
Network policy¶
With krini.networkPolicy.enabled, a NetworkPolicy allows connections to
Krini only from the peers in krini.networkPolicy.ingressFrom (e.g. the
namespace of the ingress controller), and from the node (the kubelet's
probes). There are no default peers, as they depend on the cluster:
rendering fails if the policy is enabled without any, or with a peer
without podSelector, namespaceSelector or ipBlock, which would
allow all sources. The policy also blocks the test pod of helm test
(pod labels can be forged, so it gets no exception): run the test before
enabling the policy, or check Krini through the ingress instead. For example, for an
ingress controller in the namespace ingress-nginx:
krini:
networkPolicy:
enabled: true
ingressFrom:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: ingress-nginx
Egress isn't restricted: Krini's server connects to the Cloud Registry, the login providers and the WES services.
Pods¶
krini.replicas(default 1): sessions are in cookies, but each pod remembers the refresh tokens it has just used only in memory, to tell a concurrent refresh of the same session (e.g. in two tabs) from a replay. With more than one pod, two such refreshes that reach different pods can log the user out, so keep one replica, or let the ingress keep each browser on one pod (sticky sessions, e.g. cookie-based affinity of the ingress controller). With more than one, a PodDisruptionBudget (krini.podDisruptionBudget) lets node drains evict one pod at a time (maxUnavailable: 1), or as set byminAvailableormaxUnavailable.krini.resources: requests and limits of the container (default: 100m CPU and 256Mi memory requested, limits of 500m and 512Mi).krini.probePath(default/privacy): the startup, readiness and liveness probes request the privacy notice, which needs no login and no other service;krini.startupProbe,readinessProbeandlivenessProbereplace the default probes.krini.preStopSleepSeconds(default 5): Krini keeps serving for that long after its pod is deleted, so that the ingress controller stops sending requests to it first.krini.nodeSelector,krini.tolerationsandkrini.affinityare set on the pod.
Names and labels¶
Resources are named <release>-krini, or after the release if its name
contains krini (e.g. krini for the release krini, prod-krini for
the release prod); fullnameOverride replaces that name, nameOverride
the chart name krini. Set them only when installing: changing them later
changes the pod selector, which can't be upgraded. All resources have
Helm's standard
labels, and the pods
app.kubernetes.io/component: app, e.g.:
Upgrades¶
Read the release notes (see Upgrading), then download and
verify the chart of the new release, and upgrade
with it (if your values file sets krini.tag, set the new version there,
or remove it):
version=0.2.0
gh release download "v${version}" --repo elixir-cloud-aai/krini \
--pattern "krini-${version}.tgz*"
helm upgrade krini "krini-${version}.tgz" -n krini -f values-prod.yaml
Upgrade with the complete values file rather than --reuse-values, so that
new values of the chart get their defaults. The Deployment replaces its pods
one after the other (rolling update).
Verifying the chart¶
The chart of a release is packaged, attested and signed by the release
workflow (.github/workflows/release.yml), like the image: it has a
GitHub artifact
attestation
of its provenance and a keyless Sigstore
signature, in the bundle krini-X.Y.Z.tgz.sigstore.json attached to the
release next to it. Check both before installing it, with the GitHub
CLI and
cosign 3 or
later (gh release download above fetches both files), e.g. for 0.1.0:
version=0.1.0
gh attestation verify "krini-${version}.tgz" --repo elixir-cloud-aai/krini \
--signer-workflow elixir-cloud-aai/krini/.github/workflows/release.yml \
--source-ref "refs/tags/v${version}"
cosign verify-blob "krini-${version}.tgz" \
--bundle "krini-${version}.tgz.sigstore.json" \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--certificate-identity "https://github.com/elixir-cloud-aai/krini/.github/workflows/release.yml@refs/tags/v${version}"
--source-ref and the ref in --certificate-identity check that the
workflow ran for the release's tag. The chart deploys the release's image,
which you can verify too.
Uninstallation¶
This deletes all resources of the release, but not the Secret with the
login providers, which isn't part of it, nor the pod of the last helm
test (krini-test; Helm keeps it for its logs and replaces it on the
next test): delete it with kubectl delete pod krini-test -n krini. Krini keeps no data, so nothing
else is lost.
Testing the chart¶
CI lints the chart (helm lint --strict) and runs its tests
(deployment/tests/test-chart.sh, which renders it with helm template and
checks the objects and the errors of invalid values), and installs it on a
kind cluster with the image built from the commit
(deployment/tests/install-chart.sh). It also packages the chart as the
release workflow does (scripts/package-chart.sh, tested by
deployment/tests/test-package.sh). To run them locally, with Helm 3,
yq, jq and, for the installation, Docker, kind and kubectl:
helm lint deployment --strict -f deployment/ci/test-values.yaml
deployment/tests/test-chart.sh
deployment/tests/test-package.sh
export KUBECONFIG="$PWD/kind-kubeconfig" # leaves your kubeconfig alone
kind create cluster --name krini
docker build -t krini:chart-ci .
kind load docker-image krini:chart-ci --name krini
deployment/tests/install-chart.sh krini:chart-ci
kind delete cluster --name krini