From 6537bdbd335503427a42b4c161c92b1d1d64bf34 Mon Sep 17 00:00:00 2001 From: Arnault_LPC <194310+arnault_lpc@noreply.gitea.com> Date: Tue, 21 Jul 2026 06:41:06 +0000 Subject: [PATCH] feat: add per-container resource configuration for runner and DinD #160 (#168) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This a correction that is forf from original work of [anders.eficode](https://gitea.com/anders.eficode) on [pull request #160](https://gitea.com/gitea/helm-actions/pulls/160) Description of the change Adds statefulset.dind.resources and statefulset.runner.resources as optional per-container resource overrides. When set, each takes precedence over the shared statefulset.resources for that container. When unset (default {}), statefulset.resources is used as before. Benefits The DinD sidecar and the runner container have very different resource profiles — DinD is memory-hungry (image layer cache, concurrent builds, image pulls) while the runner is a lightweight coordinator that is mostly idle between jobs. Separate resource limits allow right-sizing each container independently, avoiding the choice between over-provisioning the runner or under-provisioning DinD. Possible drawbacks None. Fully backward-compatible — both new values default to {}, causing the shared statefulset.resources fallback to apply exactly as before. Checklist - [x] Parameters are documented in the `values.yaml` and added to the `README.md` using [readme-generator-for-helm](https://github.com/bitnami-labs/readme-generator-for-helm) - [x] Breaking changes are documented in the `README.md` - [x] Helm templating unittests are added (required when changing anything in `templates` folder) - [x] Bash unittests are added (required when changing anything in `scripts` folder) - [x] All added template resources MUST render a namespace in metadata --------- Co-authored-by: Le Prévost-Corvellec Arnault Co-authored-by: Anders Reviewed-on: https://gitea.com/gitea/helm-actions/pulls/168 Reviewed-by: DaanSelen <135789+daanselen@noreply.gitea.com> Co-authored-by: Arnault_LPC <194310+arnault_lpc@noreply.gitea.com> --- README.md | 82 ++++++++------- docs/README.md | 1 + docs/resources.md | 173 ++++++++++++++++++++++++++++++++ templates/_helpers.tpl | 14 +++ templates/statefulset.yaml | 4 +- unittests/helm/statefulset.yaml | 82 +++++++++++++++ values.yaml | 9 +- 7 files changed, 323 insertions(+), 42 deletions(-) create mode 100644 docs/resources.md diff --git a/README.md b/README.md index 4e505da..fb6e256 100644 --- a/README.md +++ b/README.md @@ -65,45 +65,49 @@ If `.Values.statefulset.dind.rootless: true` is set, then the following will be ### Gitea Actions -| Name | Description | Value | -| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ | -| `enabled` | Create a Gitea Runner StatefulSet. | `false` | -| `statefulset.replicas` | the amount of (replica) runner pods deployed | `1` | -| `statefulset.timezone` | is the timezone that will be set in the runner image | `Etc/UTC` | -| `statefulset.annotations` | Gitea Runner annotations | `{}` | -| `statefulset.labels` | Gitea Runner labels | `{}` | -| `statefulset.resources` | Gitea Runner resources | `{}` | -| `statefulset.nodeSelector` | NodeSelector for the statefulset | `{}` | -| `statefulset.tolerations` | Tolerations for the statefulset | `[]` | -| `statefulset.affinity` | Affinity for the statefulset | `{}` | -| `statefulset.extraVolumes` | Extra volumes for the statefulset | `[]` | -| `statefulset.persistence.size` | Size for persistence to store Gitea Runner data | `1Gi` | -| `statefulset.securityContext` | Customize the SecurityContext | `{}` | -| `statefulset.serviceAccountName` | Customize the service account name | `""` | -| `statefulset.runtimeClassName` | Select a different RuntimeClass for pods | `""` | -| `statefulset.hostAliases` | Inject entries into the /etc/hosts file | `[]` | -| `statefulset.persistence.size` | Size for persistence to store Gitea Runner data | `1Gi` | -| `statefulset.runner.registry` | image registry, e.g. gcr.io,docker.io | `docker.gitea.com` | -| `statefulset.runner.repository` | The Gitea Runner image | `runner` | -| `statefulset.runner.tag` | The Gitea Runner tag | `2.0.1` | -| `statefulset.runner.digest` | Image digest. Allows to pin the given image tag. Useful for having control over mutable tags like `latest` | `""` | -| `statefulset.runner.pullPolicy` | The Gitea Runner pullPolicy | `IfNotPresent` | -| `statefulset.runner.fullOverride` | Completely overrides the image registry, path/image, tag and digest. | `""` | -| `statefulset.runner.extraVolumeMounts` | Allows mounting extra volumes in the Gitea Runner container | `[]` | -| `statefulset.runner.extraEnvs` | Allows adding custom environment variables | `[]` | -| `statefulset.runner.flushCache` | whether to clear the .runner (cache) file by creating an extra init container, can slightly increase boot-up time | `false` | -| `statefulset.runner.config` | Gitea Runner custom configuration. See [Gitea Runner documentation](https://docs.gitea.com/usage/actions/act-runner#configuration) for details. | `Too complex. See values.yaml` | -| `statefulset.dind.rootless` | a simple flag to let helm know we are dealing with a rootless dind container | `false` | -| `statefulset.dind.uid` | a field to set the running user id for the rootless dind container, so it knows where to look for the socket | `""` | -| `statefulset.dind.registry` | image registry, e.g. gcr.io,docker.io | `docker.io` | -| `statefulset.dind.repository` | The Docker-in-Docker image | `docker` | -| `statefulset.dind.tag` | The Docker-in-Docker image tag | `29.5.2-dind` | -| `statefulset.dind.digest` | Image digest. Allows to pin the given image tag. Useful for having control over mutable tags like `latest` | `""` | -| `statefulset.dind.fullOverride` | Completely overrides the image registry, path/image, tag and digest. | `""` | -| `statefulset.dind.pullPolicy` | The Docker-in-Docker pullPolicy | `IfNotPresent` | -| `statefulset.dind.extraVolumeMounts` | Allows mounting extra volumes in the Docker-in-Docker container | `[]` | -| `statefulset.dind.extraEnvs` | Allows adding custom environment variables, such as `DOCKER_IPTABLES_LEGACY` | `[]` | -| `statefulset.dind.extraArgs` | Allows adding custom arguments to the Docker Daemon | `[]` | +For resource limit examples (runner vs DinD), see [docs/resources.md](./docs/resources.md). + +| Name | Description | Value | +| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------ | +| `enabled` | Create a Gitea Runner StatefulSet. | `false` | +| `statefulset.replicas` | the amount of (replica) runner pods deployed | `1` | +| `statefulset.timezone` | is the timezone that will be set in the runner image | `Etc/UTC` | +| `statefulset.annotations` | Gitea Runner annotations | `{}` | +| `statefulset.labels` | Gitea Runner labels | `{}` | +| `statefulset.resources` | Shared resource requests/limits for both containers. Overridden by statefulset.runner.resources and statefulset.dind.resources. See docs/resources.md. | `{}` | +| `statefulset.nodeSelector` | NodeSelector for the statefulset | `{}` | +| `statefulset.tolerations` | Tolerations for the statefulset | `[]` | +| `statefulset.affinity` | Affinity for the statefulset | `{}` | +| `statefulset.extraVolumes` | Extra volumes for the statefulset | `[]` | +| `statefulset.persistence.size` | Size for persistence to store Gitea Runner data | `1Gi` | +| `statefulset.securityContext` | Customize the SecurityContext | `{}` | +| `statefulset.serviceAccountName` | Customize the service account name | `""` | +| `statefulset.runtimeClassName` | Select a different RuntimeClass for pods | `""` | +| `statefulset.hostAliases` | Inject entries into the /etc/hosts file | `[]` | +| `statefulset.persistence.size` | Size for persistence to store Gitea Runner data | `1Gi` | +| `statefulset.runner.registry` | image registry, e.g. gcr.io,docker.io | `docker.gitea.com` | +| `statefulset.runner.repository` | The Gitea Runner image | `runner` | +| `statefulset.runner.tag` | The Gitea Runner tag | `2.0.1` | +| `statefulset.runner.digest` | Image digest. Allows to pin the given image tag. Useful for having control over mutable tags like `latest` | `""` | +| `statefulset.runner.pullPolicy` | The Gitea Runner pullPolicy | `IfNotPresent` | +| `statefulset.runner.fullOverride` | Completely overrides the image registry, path/image, tag and digest. | `""` | +| `statefulset.runner.resources` | Resource requests/limits for the runner container. Takes precedence over statefulset.resources when set. | `{}` | +| `statefulset.runner.extraVolumeMounts` | Allows mounting extra volumes in the Gitea Runner container | `[]` | +| `statefulset.runner.extraEnvs` | Allows adding custom environment variables | `[]` | +| `statefulset.runner.flushCache` | whether to clear the .runner (cache) file by creating an extra init container, can slightly increase boot-up time | `false` | +| `statefulset.runner.config` | Gitea Runner custom configuration. See [Gitea Runner documentation](https://docs.gitea.com/usage/actions/act-runner#configuration) for details. | `Too complex. See values.yaml` | +| `statefulset.dind.rootless` | a simple flag to let helm know we are dealing with a rootless dind container | `false` | +| `statefulset.dind.uid` | a field to set the running user id for the rootless dind container, so it knows where to look for the socket | `""` | +| `statefulset.dind.registry` | image registry, e.g. gcr.io,docker.io | `docker.io` | +| `statefulset.dind.repository` | The Docker-in-Docker image | `docker` | +| `statefulset.dind.tag` | The Docker-in-Docker image tag | `29.5.2-dind` | +| `statefulset.dind.digest` | Image digest. Allows to pin the given image tag. Useful for having control over mutable tags like `latest` | `""` | +| `statefulset.dind.fullOverride` | Completely overrides the image registry, path/image, tag and digest. | `""` | +| `statefulset.dind.pullPolicy` | The Docker-in-Docker pullPolicy | `IfNotPresent` | +| `statefulset.dind.resources` | Resource requests/limits for the DinD sidecar container. Takes precedence over statefulset.resources when set. | `{}` | +| `statefulset.dind.extraVolumeMounts` | Allows mounting extra volumes in the Docker-in-Docker container | `[]` | +| `statefulset.dind.extraEnvs` | Allows adding custom environment variables, such as `DOCKER_IPTABLES_LEGACY` | `[]` | +| `statefulset.dind.extraArgs` | Allows adding custom arguments to the Docker Daemon | `[]` | ### Gitea Actions Init diff --git a/docs/README.md b/docs/README.md index 76452ce..9520517 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,3 +1,4 @@ # Gitea Actions Helm Chart Docs +- [Resource limits and capacity](./resources.md) - [connectionCommandOverride explanation](./connectionCommandOverride.md) diff --git a/docs/resources.md b/docs/resources.md new file mode 100644 index 0000000..8f434ca --- /dev/null +++ b/docs/resources.md @@ -0,0 +1,173 @@ +# Resource limits and capacity + +By default, all resource values are empty (`{}`). Without explicit limits, runner pods can consume unbounded CPU and memory on a node. This guide explains how to configure resource usage properly. + +## Pod architecture + +Each runner pod contains two resource-consuming containers: + +| Container | Role | Helm value | +| --- | --- | --- | +| `runner` | Polls Gitea and orchestrates CI jobs | `statefulset.runner.resources` | +| `dind` | Docker-in-Docker daemon; executes job containers | `statefulset.dind.resources` | + +The DinD container runs as a native sidecar (`initContainer` with `restartPolicy: Always`). + +## Helm resource keys + +Three values control Kubernetes resource requests and limits: + +| Key | Purpose | +| --- | --- | +| `statefulset.resources` | Shared fallback applied to **both** containers when no override is set | +| `statefulset.runner.resources` | Override for the `runner` container only | +| `statefulset.dind.resources` | Override for the `dind` container only | + +Precedence: + +```text +statefulset.runner.resources → else statefulset.resources +statefulset.dind.resources → else statefulset.resources +``` + +**Recommendation:** set `statefulset.runner.resources` and `statefulset.dind.resources` +explicitly instead of relying on the shared fallback. The runner process is lightweight; +DinD and CI workloads need most of the budget. + +## Example: separate runner and DinD limits + +```yaml +enabled: true +giteaRootURL: https://gitea.example.com +existingSecret: runner-secret +existingSecretKey: runner-token + +statefulset: + replicas: 1 + + runner: + resources: + requests: + cpu: 100m + memory: 256Mi + limits: + cpu: 500m + memory: 512Mi + config: | + log: + level: info + cache: + enabled: false + runner: + capacity: 1 + container: + require_docker: true + docker_timeout: 300s + + dind: + resources: + requests: + cpu: 500m + memory: 2Gi + limits: + cpu: 2 + memory: 4Gi +``` + +## Two layers of limiting + +Kubernetes limits and act-runner job limits serve different purposes. Use both for a robust setup. + +### 1. Kubernetes limits (Helm values) + +These apply to the `runner` and `dind` containers in the pod. + +- **`dind` limits** cap the Docker daemon and everything it runs inside the pod (images, build caches, job containers). +- **`runner` limits** cap the act-runner process itself. + +If neither is set, a runaway build can exhaust the entire node. + +### 2. Per-job Docker limits (`container.options`) + +CI jobs run as Docker containers spawned by act-runner through the Docker socket. They are +**not** separate Kubernetes containers. Configure per-job limits in `statefulset.runner.config` +— see [act-runner configuration](https://docs.gitea.com/usage/actions/act-runner#configuration) +and [config.example.yaml](https://gitea.com/gitea/runner/src/branch/main/internal/pkg/config/config.example.yaml). + +## act-runner settings that affect resource usage + +Settings such as `runner.capacity` and `container.options` live in `statefulset.runner.config`, not in the Helm resource values. Refer to the runner documentation for details: + +- [act-runner configuration](https://docs.gitea.com/usage/actions/act-runner#configuration) +- [config.example.yaml](https://gitea.com/gitea/runner/src/branch/main/internal/pkg/config/config.example.yaml) in the [Gitea/runner](https://gitea.com/gitea/runner) repository + +When concurrent jobs or per-job Docker limits increase expected load, size `statefulset.dind.resources` accordingly on the Helm side. + +## Capacity planning + +These formulas apply to the Helm resource values (`statefulset.runner.resources`, `statefulset.dind.resources`, `statefulset.replicas`): + +### Per pod + +```text +pod budget ≈ runner.limits + dind.limits +``` + +### Per node + +```text +node budget ≈ (runner.limits + dind.limits) × statefulset.replicas + system overhead +``` + +Example with the configuration above and `replicas: 3`: + +- runner: 512Mi × 3 = 1.5Gi +- dind: 4Gi × 3 = 12Gi +- total: ~13.5Gi minimum, excluding other workloads on the node + +Use `statefulset.nodeSelector` and `statefulset.tolerations` to place runners on dedicated nodes when needed. + +## Why not only set `statefulset.resources`? + +Before chart 0.1.2, one value was copied to **both** containers. **Requests** are where it +hurts most: the scheduler reserves capacity per container, and both inherit the same numbers. + +```yaml +# Only statefulset.resources — requests copied to runner AND dind: +statefulset: + resources: + requests: + cpu: 500m + memory: 2Gi # what DinD needs… + # runner also requests 500m + 2Gi → pod ~1 CPU + ~4Gi reserved (mostly wasted) + # dind requests 500m + 2Gi ✓ +``` + +```yaml +statefulset: + resources: + requests: + cpu: 100m + memory: 256Mi # what the runner actually needs… + # runner requests 100m + 256Mi ✓ + # dind requests 100m + 256Mi too → tiny slot, builds starve ✗ +``` + +```yaml +# Separate overrides — scheduler sees the real footprint: +statefulset: + resources: {} + runner: + resources: + requests: + cpu: 100m + memory: 256Mi + dind: + resources: + requests: + cpu: 500m + memory: 2Gi + # pod requests ~600m CPU + ~2.25Gi RAM — not 1 CPU + 4Gi, nor 200m + 512Mi +``` + +Limits follow the same split. Unset overrides (`{}`) still fall back to `statefulset.resources`. diff --git a/templates/_helpers.tpl b/templates/_helpers.tpl index 5147da6..7850d79 100644 --- a/templates/_helpers.tpl +++ b/templates/_helpers.tpl @@ -116,6 +116,13 @@ Create image for the Gitea Actions Act Runner {{ include "gitea.actions.common.image" (dict "root" . "image" .Values.statefulset.runner) }} {{- end -}} +{{/* +Resolve resource requests/limits for the runner container. +*/}} +{{- define "gitea.actions.runner.resources" -}} +{{- toYaml (default .Values.statefulset.resources .Values.statefulset.runner.resources) -}} +{{- end -}} + {{/* Create image for DinD */}} @@ -123,6 +130,13 @@ Create image for DinD {{ include "gitea.actions.common.image" (dict "root" . "image" .Values.statefulset.dind) }} {{- end -}} +{{/* +Resolve resource requests/limits for the DinD container. +*/}} +{{- define "gitea.actions.dind.resources" -}} +{{- toYaml (default .Values.statefulset.resources .Values.statefulset.dind.resources) -}} +{{- end -}} + {{/* Create image for Init */}} diff --git a/templates/statefulset.yaml b/templates/statefulset.yaml index 6d1a98b..b319f91 100644 --- a/templates/statefulset.yaml +++ b/templates/statefulset.yaml @@ -120,7 +120,7 @@ spec: - /var/run/docker.sock {{- end }} resources: - {{- toYaml .Values.statefulset.resources | nindent 12 }} + {{- include "gitea.actions.dind.resources" . | nindent 12 }} volumeMounts: {{- if .Values.statefulset.dind.rootless }} - mountPath: /run/user/{{ .Values.statefulset.dind.uid | default 1000 }}/ @@ -155,7 +155,7 @@ spec: {{- toYaml .Values.statefulset.runner.extraEnvs | nindent 12 }} {{- end }} resources: - {{- toYaml .Values.statefulset.resources | nindent 12 }} + {{- include "gitea.actions.runner.resources" . | nindent 12 }} volumeMounts: - mountPath: /runner/config.yaml name: runner-config diff --git a/unittests/helm/statefulset.yaml b/unittests/helm/statefulset.yaml index 1f29fb2..b8b2962 100644 --- a/unittests/helm/statefulset.yaml +++ b/unittests/helm/statefulset.yaml @@ -451,3 +451,85 @@ tests: - equal: path: spec.template.spec.initContainers[0].image value: test.io/busybox:1.37.0 + +# +## RESOURCES +# + + - it: shared statefulset.resources applies to both runner and dind containers + template: templates/statefulset.yaml + set: + enabled: true + statefulset.resources: + requests: + memory: "512Mi" + cpu: "250m" + limits: + memory: "1Gi" + asserts: + - hasDocuments: + count: 1 + - equal: + path: spec.template.spec.containers[0].resources + value: + requests: + memory: "512Mi" + cpu: "250m" + limits: + memory: "1Gi" + - equal: + path: spec.template.spec.initContainers[1].resources + value: + requests: + memory: "512Mi" + cpu: "250m" + limits: + memory: "1Gi" + + - it: statefulset.runner.resources overrides shared resources for runner container only + template: templates/statefulset.yaml + set: + enabled: true + statefulset.resources: + requests: + memory: "512Mi" + statefulset.runner.resources: + requests: + memory: "256Mi" + asserts: + - hasDocuments: + count: 1 + - equal: + path: spec.template.spec.containers[0].resources + value: + requests: + memory: "256Mi" + - equal: + path: spec.template.spec.initContainers[1].resources + value: + requests: + memory: "512Mi" + + - it: statefulset.dind.resources overrides shared resources for dind container only + template: templates/statefulset.yaml + set: + enabled: true + statefulset.resources: + requests: + memory: "512Mi" + statefulset.dind.resources: + requests: + memory: "4Gi" + asserts: + - hasDocuments: + count: 1 + - equal: + path: spec.template.spec.containers[0].resources + value: + requests: + memory: "512Mi" + - equal: + path: spec.template.spec.initContainers[1].resources + value: + requests: + memory: "4Gi" diff --git a/values.yaml b/values.yaml index 4305dbd..ead8ae3 100644 --- a/values.yaml +++ b/values.yaml @@ -1,12 +1,15 @@ # Configure Gitea Actions ## @section Gitea Actions +## @descriptionStart +## For resource limit examples (runner vs DinD), see [docs/resources.md](./docs/resources.md). +## @descriptionEnd # ## @param enabled Create a Gitea Runner StatefulSet. ## @param statefulset.replicas the amount of (replica) runner pods deployed ## @param statefulset.timezone is the timezone that will be set in the runner image ## @param statefulset.annotations Gitea Runner annotations ## @param statefulset.labels Gitea Runner labels -## @param statefulset.resources Gitea Runner resources +## @param statefulset.resources Shared resource requests/limits for both containers. Overridden by statefulset.runner.resources and statefulset.dind.resources. See docs/resources.md. ## @param statefulset.nodeSelector NodeSelector for the statefulset ## @param statefulset.tolerations Tolerations for the statefulset ## @param statefulset.affinity Affinity for the statefulset @@ -25,6 +28,7 @@ ## @param statefulset.runner.digest Image digest. Allows to pin the given image tag. Useful for having control over mutable tags like `latest` ## @param statefulset.runner.pullPolicy The Gitea Runner pullPolicy ## @param statefulset.runner.fullOverride Completely overrides the image registry, path/image, tag and digest. +## @param statefulset.runner.resources Resource requests/limits for the runner container. Takes precedence over statefulset.resources when set. ## @param statefulset.runner.extraVolumeMounts Allows mounting extra volumes in the Gitea Runner container ## @param statefulset.runner.extraEnvs Allows adding custom environment variables ## @param statefulset.runner.flushCache whether to clear the .runner (cache) file by creating an extra init container, can slightly increase boot-up time @@ -38,6 +42,7 @@ ## @param statefulset.dind.digest Image digest. Allows to pin the given image tag. Useful for having control over mutable tags like `latest` ## @param statefulset.dind.fullOverride Completely overrides the image registry, path/image, tag and digest. ## @param statefulset.dind.pullPolicy The Docker-in-Docker pullPolicy +## @param statefulset.dind.resources Resource requests/limits for the DinD sidecar container. Takes precedence over statefulset.resources when set. ## @param statefulset.dind.extraVolumeMounts Allows mounting extra volumes in the Docker-in-Docker container ## @param statefulset.dind.extraEnvs Allows adding custom environment variables, such as `DOCKER_IPTABLES_LEGACY` ## @param statefulset.dind.extraArgs Allows adding custom arguments to the Docker Daemon @@ -75,6 +80,7 @@ statefulset: digest: "" pullPolicy: IfNotPresent fullOverride: "" + resources: {} extraVolumeMounts: [] extraEnvs: [] @@ -104,6 +110,7 @@ statefulset: digest: "" pullPolicy: IfNotPresent fullOverride: "" + resources: {} extraVolumeMounts: [] # If the container keeps crashing in your environment, you might have to add the `DOCKER_IPTABLES_LEGACY` environment variable.