GitOps with ArgoCD on Kubernetes — Setup Guide and Best Practices

GitOps transformed how teams manage Kubernetes deployments — replacing imperative kubectl apply workflows with a declarative, Git-driven model where your repository is the single source of truth for cluster state. ArgoCD is the dominant GitOps controller, used by thousands of organizations to manage deployments from a handful of services to thousands of applications across dozens of clusters.

This guide covers ArgoCD architecture, the patterns that actually work at scale (App of Apps, ApplicationSets), and the operational practices that prevent GitOps implementations from becoming a new source of technical debt.

GitOps Principles

Before diving into ArgoCD specifics, the four core GitOps principles from the OpenGitOps specification:

1. Declarative — the entire desired state of the system is expressed declaratively
2. Versioned and immutable — desired state is stored in a version control system that enforces immutability
3. Pulled automatically — software agents automatically pull the desired state from the source
4. Continuously reconciled — software agents continuously observe actual system state and attempt to apply the desired state

The key shift from traditional CI/CD: in GitOps, the cluster pulls desired state from Git rather than a CI system pushing changes to the cluster. This means your cluster credentials never need to leave the cluster — a major security improvement.

ArgoCD Architecture

Understanding ArgoCD’s architecture is essential for operating it correctly.

Core components:

Application Controller — the main reconciliation loop. Watches ArgoCD Application objects and continuously compares the live cluster state against the desired state in Git. When drift is detected, it either alerts (manual sync) or corrects it (automated sync).

Repo Server — clones and caches Git repositories. Runs Helm template rendering, Kustomize builds, and other manifest generation. Stateless — can be scaled horizontally.

Server — the API server and UI backend. Handles authentication, serves the ArgoCD UI, and exposes the gRPC/REST API used by the CLI and CI/CD integrations.

Redis — cache layer for application state, used by the Application Controller to avoid unnecessary Git fetches.

Dex (optional) — built-in OIDC provider for SSO integration. Used when you want to connect ArgoCD to your existing identity provider (Okta, Azure AD, GitHub).

Git Repo ──pull── Repo Server ──template── Application Controller ──apply── Kubernetes API
                                                      │
                                               Redis (cache)
                                                      │
                                              ArgoCD Server (UI/API)

Installing ArgoCD

# Create namespace and install ArgoCD
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Wait for pods to be ready
kubectl wait --for=condition=Ready pods --all -n argocd --timeout=120s

# Get initial admin password
kubectl get secret argocd-initial-admin-secret -n argocd -o jsonpath="{.data.password}" | base64 -d

# Port forward to access UI
kubectl port-forward svc/argocd-server -n argocd 8080:443

For production, use the Helm chart which gives more control over values:

helm repo add argo https://argoproj.github.io/argo-helm
helm repo update
helm install argocd argo/argo-cd \
  --namespace argocd \
  --create-namespace \
  --values argocd-values.yaml

Your First Application

An ArgoCD Application is the core resource — it defines what to deploy (source), where to deploy it (destination), and how (sync policy).

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/my-org/my-app
    targetRevision: HEAD
    path: manifests/
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true           # Delete resources removed from Git
      selfHeal: true        # Revert manual changes made directly to cluster
    syncOptions:
      - CreateNamespace=true

The App of Apps Pattern

The App of Apps pattern is how you bootstrap ArgoCD itself and manage multiple applications declaratively. Instead of manually creating Application objects in the ArgoCD UI, you create a root Application that watches a directory of Application manifests.

argocd/
├── root-app.yaml           # Root Application watched by ArgoCD
└── apps/
    ├── frontend.yaml       # Application manifest
    ├── backend.yaml        # Application manifest
    ├── database.yaml       # Application manifest
    └── monitoring.yaml     # Application manifest

The root Application:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: root-app
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/my-org/cluster-config
    targetRevision: HEAD
    path: argocd/apps/          # Watch this directory
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd           # Deploy Application objects into argocd namespace
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Now adding a new application to the cluster means adding a YAML file to argocd/apps/ and merging a PR. No manual CLI or UI actions needed.

Bootstrapping: When setting up a new cluster, you apply only the root Application once:

kubectl apply -f argocd/root-app.yaml

ArgoCD picks up the root app, which then deploys all other applications. The entire cluster configuration is in Git.

ApplicationSets for Multi-Cluster

ApplicationSets extend the App of Apps pattern to multiple clusters using generators — they programmatically create Application objects based on a pattern.

Cluster generator — creates one Application per registered cluster:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: guestbook
  namespace: argocd
spec:
  generators:
  - clusters: {}             # All registered clusters
  template:
    metadata:
      name: '{{name}}-guestbook'
    spec:
      project: default
      source:
        repoURL: https://github.com/my-org/guestbook
        targetRevision: HEAD
        path: manifests/
      destination:
        server: '{{server}}'
        namespace: guestbook

Git directory generator — creates one Application per directory in a Git repo:

generators:
- git:
    repoURL: https://github.com/my-org/cluster-config
    revision: HEAD
    directories:
    - path: services/*      # One app per subdirectory

This pattern is particularly powerful for environment promotion — you can have directories for dev/, staging/, production/ and use different Helm values or Kustomize overlays per environment, all managed from one ApplicationSet.

Sync Policies: Automated vs Manual

Manual sync (default) — ArgoCD detects drift and shows “OutOfSync” status, but waits for a human to trigger sync. Use this for production environments where you want a human gate on deployments.

Automated sync — ArgoCD automatically applies changes when Git changes. Best for non-production environments, or production with sufficient testing in the pipeline.

syncPolicy:
  automated:
    prune: true       # Delete resources not in Git
    selfHeal: true    # Revert direct kubectl changes

Important caveats with automated sync:

• prune: true will delete resources that are removed from Git — make sure your Git state is authoritative before enabling
• selfHeal: true will revert manual kubectl changes — inform your team before enabling (they’ll be surprised when their hotfix gets reverted)

Sync windows — restrict when automated syncs occur:

spec:
  syncWindows:
  - kind: deny
    schedule: "0 22 * * *"     # No syncs starting at 10pm
    duration: 8h                # For 8 hours (until 6am)
    applications: ["*"]
    clusters: ["production"]

Health Checks and Custom Health Checks

ArgoCD has built-in health checks for standard Kubernetes resources (Deployment, StatefulSet, DaemonSet, Service, Ingress). It reports Healthy, Progressing, Degraded, or Unknown.

For CRDs (Custom Resource Definitions) — like cert-manager Certificate objects or Crossplane managed resources — you need custom health checks written in Lua:

# In argocd-cm ConfigMap
resource.customizations.health.cert-manager.io_Certificate: |
  hs = {}
  if obj.status ~= nil then
    if obj.status.conditions ~= nil then
      for i, condition in ipairs(obj.status.conditions) do
        if condition.type == "Ready" and condition.status == "False" then
          hs.status = "Degraded"
          hs.message = condition.message
          return hs
        end
        if condition.type == "Ready" and condition.status == "True" then
          hs.status = "Healthy"
          hs.message = condition.message
          return hs
        end
      end
    end
  end
  hs.status = "Progressing"
  hs.message = "Waiting for certificate"
  return hs

Without custom health checks, ArgoCD will report CRD-based resources as Healthy immediately after creation regardless of their actual state — a common source of false green deployments.

RBAC for ArgoCD

ArgoCD has its own RBAC system layered on top of Kubernetes RBAC. Define it in the argocd-rbac-cm ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-rbac-cm
  namespace: argocd
data:
  policy.default: role:readonly
  policy.csv: |
    # Developers can sync non-production apps
    p, role:developer, applications, sync, dev/*, allow
    p, role:developer, applications, get, */*, allow
    p, role:developer, logs, get, */*, allow

    # Platform team has full access
    p, role:platform, applications, *, */*, allow
    p, role:platform, clusters, get, *, allow

    # Bind groups from your OIDC provider
    g, my-org:platform-team, role:platform
    g, my-org:developers, role:developer

Key principle: default to role:readonly for all users and explicitly grant elevated permissions to specific groups. This prevents accidental syncs or deletions by users unfamiliar with GitOps workflows.

Argo CD Image Updater

Argo CD Image Updater automates updating container image tags in Git when new images are pushed to a registry — closing the loop on fully automated GitOps deployments.

# Annotations on your ArgoCD Application
metadata:
  annotations:
    argocd-image-updater.argoproj.io/image-list: my-app=registry.example.com/my-app
    argocd-image-updater.argoproj.io/my-app.update-strategy: semver
    argocd-image-updater.argoproj.io/my-app.allow-tags: regexp:^v[0-9]+\.[0-9]+\.[0-9]+$
    argocd-image-updater.argoproj.io/write-back-method: git

With write-back-method: git, Image Updater creates a commit in your Git repo when a new image is available — keeping Git as the source of truth even for image tag updates.

Production GitOps Checklist

Before going live:

• App of Apps bootstrapping pattern implemented
• Automated sync enabled for dev/staging, manual for production
• Sync windows configured to prevent off-hours deployments in production
• ArgoCD RBAC configured (not everyone should be able to sync production)
• Custom health checks written for all CRDs in use
• Notifications configured (Slack/PagerDuty for OutOfSync/Degraded)
• ArgoCD itself managed by ArgoCD (self-managed deployment)
• Git repo access tokens rotated regularly
• Audit logging enabled for ArgoCD API operations

Take Your Platform Engineering Further

GitOps with ArgoCD is the foundation of a mature platform engineering practice. The next steps — internal developer platform, golden path templates, self-service environments — build on it.

→ Platform Engineering service at kubernetes.ae — we design and implement GitOps workflows and internal developer platforms for engineering teams at scale.