Strategic Merge Patch

This document describes how to overwrite the configuration generated by the operator using strategic merge patches.

When users need to apply a specific configuration to the containers that is either not exposed in the custom resource definitions or already defined by the operator, strategic merge patch can be used.

How does it work?

The Prometheus, Alertmanager, and ThanosRuler CRDs expose a spec.containers field which allows to:

  • Override fields for the containers generated by the operator.
  • Inject fields for existing containers.

How to patch a container probe

Merging patch for Prometheus

The following manifest overwrites the failureThreshold value of startup probe of the Prometheus container:

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: overwrite-failureThreshold
spec:
  containers:
  - name: prometheus
    startupProbe:
      failureThreshold: 500

Merging patch for Alertmanager

The following manifest overwrites the failureThreshold values of the readiness and liveness probes for the Alertmanager container.

apiVersion: monitoring.coreos.com/v1
kind: Alertmanager
metadata:
  name: overwrite-probes
spec:
  containers:
  - name: alertmanager
    livenessProbe:
      failureThreshold: 5
    readinessProbe:
      failureThreshold: 5

How to inject an environment variable in an existing container

The following manifest injects the environment variable GOMEMLIMIT to the Prometheus container:

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: inject-env-var
spec:
  containers:
  - name: "prometheus"
    env:
    - name: GOMEMLIMIT
      value: 6Gi

How to inject a sidecar container

The following manifest injects an additional container to the generated StatefulSet:

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: inject-sidecar
spec:
  containers:
  - name: "sleep"
    image: "busybox"
    args:
    - sleep
    - "3600"

How to inject additional CLI arguments into the prometheus container

The following manifest injects an additional CLI argument in the default Prometheus argument list. Note the use of .spec.additionalArgs in this example. Using .spec.containers[*].args directly would instead overwrite the container’s args list completely, including the default arguments.

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: additional-arguments
spec:
  additionalArgs:
  - name: "scrape.timestamp-tolerance"
    value: "15ms"

How to support Restricted and Baseline policies

Pod Security Standards define policies such as Baseline and Restricted which require to patch the container definitions generated by the Prometheus operator.

How to patch Prometheus for the Restricted policy

The following manifest changes the securityContext of containers in Prometheus Pod. If the Thanos sidecar is enabled, similar changes should be applied for the thanos-sidecar container.

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: prometheus-restricted-baseline-ns
  namespace: restricted-baseline-ns
spec: 
  serviceAccountName: prometheus
  containers:
    - name: prometheus
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
        seccompProfile:
          type: RuntimeDefault
    - name: config-reloader
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
        seccompProfile:
          type: RuntimeDefault
  initContainers:
    - name: init-config-reloader
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
        seccompProfile:
          type: RuntimeDefault

How to patch Alertmanager for the Restricted policy

The following manifest changes the securityContext of containers in Alertmanager Pod.

apiVersion: monitoring.coreos.com/v1
kind: Alertmanager
metadata:
  name: alertmanager-restricted-baseline-ns
  namespace: restricted-baseline-ns
spec:
  containers:
    - name: alertmanager
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
        seccompProfile:
          type: RuntimeDefault
    - name: config-reloader
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
        seccompProfile:
          type: RuntimeDefault
  initContainers:
    - name: init-config-reloader
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
        seccompProfile:
          type: RuntimeDefault

How to patch ThanosRuler for the Restricted policy

The following manifest changes the securityContext of containers in ThanosRuler Pod.

apiVersion: monitoring.coreos.com/v1
kind: ThanosRuler
metadata:
  name: thanos-ruler-restricted-baseline-ns
  namespace: restricted-baseline-ns
  labels:
    example: thanos-ruler
spec:
  image: quay.io/thanos/thanos:v0.28.1
  ruleSelector:
    matchLabels:
      role: my-thanos-rules
  queryEndpoints:
    - dnssrv+_http._tcp.my-thanos-querier.monitoring.svc.cluster.local
  alertmanagersConfig:
    key: alertmanager-configs.yaml
    name: thanosruler-alertmanager-config
  containers:
    - name: thanos-ruler
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
        seccompProfile:
          type: RuntimeDefault
    - name: config-reloader
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
        seccompProfile:
          type: RuntimeDefault