Manage pre-installed resources

Revision	Date	Description
`1.0`	24.07.2024	Init Changelog

Introduction

Some of the resources are pre-installed with cluster (e.g. CoreDNS installed by Kubespray) and managing them force us to manually applying changes by editing them on cluster (with kubectl edit command).

FluxCD allows us, to add these resources as IaC and manage them GitOps way.

Using just YAML manifest

Follow and adjust steps below to add pre-installed resources into FluxCD:

Create YAML manifest for resource import with basic config (only required information that should identify resource by Flux):

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: coredns
  namespace: kube-system
  labels:
    k8s-app: kube-dns
    kubernetes.io/name: coredns
spec:
  selector:
    matchLabels:
      k8s-app: kube-dns
  template:
    metadata:
      labels:
        k8s-app: kube-dns
    spec:
      containers: []

Add annotations for FluxCD:

  ---
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: coredns
    namespace: kube-system
    labels:
      k8s-app: kube-dns
      kubernetes.io/name: coredns
+   annotations:
+     kustomize.toolkit.fluxcd.io/prune: disabled
+     kustomize.toolkit.fluxcd.io/ssa: merge
  spec:
    selector:
      matchLabels:
        k8s-app: kube-dns
    template:
      metadata:
        labels:
          k8s-app: kube-dns
      spec:
        containers: []

Add changes you want to apply:

  ---
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: coredns
    namespace: kube-system
    labels:
      k8s-app: kube-dns
      kubernetes.io/name: coredns
    annotations:
      kustomize.toolkit.fluxcd.io/prune: disabled
      kustomize.toolkit.fluxcd.io/ssa: merge
  spec:
+   replicas: 4
    selector:
      matchLabels:
        k8s-app: kube-dns
    template:
      metadata:
        labels:
          k8s-app: kube-dns
      spec:
-       containers: []
+       containers:
+       - name: coredns
+         resources:
+           requests:
+             cpu: 100m
+             memory: 300Mi
+           limits:
+             cpu: 500m
+             memory: 600Mi

Create kustomization.yaml file with valid syntax and add your YAML manifest (e.g. deployment.yaml) as resource:

---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kube-system
resources:
- deployment.yaml

Commit and push changes.
Wait for flux to reconcile and verify your patch by describing resource.

Using Kustomize tools like generators

If you want to manage pre-installed ConfigMap or Secret with Kustomize and generator by FluxCD, you need to:

Import resource metadata into YAML (without any data):

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns
  namespace: kube-system

Add FluxCD annotations:

  ---
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: coredns
    namespace: kube-system
+   annotations:
+     kustomize.toolkit.fluxcd.io/prune: disabled
+     kustomize.toolkit.fluxcd.io/ssa: merge

Create kustomization.yaml and add created YAML manifest as resource (e.g. configmap.yaml):

---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kube-system
resources:
- configmap.yaml

Define generator with behavior: merge and disableNameSuffixHash option (remember to create your entires for resource like envs, config files, etc.):

  ---
  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization
  namespace: kube-system
+ configMapGenerator:
+ - name: coredns
+   namespace: kube-system
+   behavior: merge
+   options:
+     disableNameSuffixHash: true
+   files:
+   - config/Corefile
  resources:
  - configmap.yaml

Commit and push changes.
Wait for flux to reconcile and verify your patch by describing resource.

Last modified: 17 February 2025