🚀 ArgoCD Complete Deep-Dive Guide

1. What is ArgoCD & Why It Exists

The Problem ArgoCD Solves

Before ArgoCD, teams faced these challenges:

Key Characteristics

2. GitOps Principles

ArgoCD is built on four fundamental GitOps principles. Understanding these is essential before diving into the tool itself.

3. ArgoCD Architecture

ArgoCD runs inside your Kubernetes cluster as a set of microservices. Let's look at every component:

Component Deep-Dive

How Data Flows

4. Installation — Step by Step

Prerequisites

• A running Kubernetes cluster (minikube, kind, EKS, GKE, AKS — any)
• kubectl configured and connected to the cluster
• helm (optional, for Helm-based install)

Method 1: Plain YAML Install (Recommended for Learning)

kubectl create namespace argocd

# Install the latest stable version
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Or a specific version (recommended for production)
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/v2.13.0/manifests/install.yaml

kubectl wait --for=condition=Ready pods --all -n argocd --timeout=300s

# Check everything is running
kubectl get pods -n argocd

NAME                                  READY   STATUS    AGE
argocd-application-controller-0       1/1     Running   2m
argocd-dex-server-xxxxx               1/1     Running   2m
argocd-redis-xxxxx                    1/1     Running   2m
argocd-repo-server-xxxxx              1/1     Running   2m
argocd-server-xxxxx                   1/1     Running   2m
argocd-notifications-controller-xxx   1/1     Running   2m

# Option A: Port forward (quick, local dev)
kubectl port-forward svc/argocd-server -n argocd 8080:443

# Option B: Change service to LoadBalancer
kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'

# Option C: Use an Ingress (production)

# The initial password is stored in a Kubernetes secret
kubectl -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath="{.data.password}" | base64 -d

# Username: admin
# Password: (output from above command)

# macOS
brew install argocd

# Linux
curl -sSL -o argocd-linux-amd64 \
  https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
sudo install -m 555 argocd-linux-amd64 /usr/local/bin/argocd
rm argocd-linux-amd64

# Login
argocd login localhost:8080

# Change the default password (strongly recommended)
argocd account update-password

Method 2: Helm Install (Production)

# Add the ArgoCD Helm repo
helm repo add argo https://argoproj.github.io/argo-helm
helm repo update

# Install with custom values
helm install argocd argo/argo-cd \
  --namespace argocd \
  --create-namespace \
  --values values.yaml

Example values.yaml:

# values.yaml
server:
  replicas: 2
  ingress:
    enabled: true
    hostname: argocd.mycompany.com
    tls: true

controller:
  replicas: 1

repoServer:
  replicas: 2

redis-ha:
  enabled: true

configs:
  params:
    server.insecure: false
  rbac:
    policy.csv: |
      g, my-team, role:admin

5. Core Concepts

Application

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  namespace: argocd        # Applications always live in the argocd namespace
spec:
  project: default         # Which ArgoCD project this belongs to

  source:
    repoURL: https://github.com/myorg/my-app-config.git
    targetRevision: main   # Branch, tag, or commit SHA
    path: k8s/overlays/production   # Path within the repo

  destination:
    server: https://kubernetes.default.svc   # Target cluster
    namespace: production                    # Target namespace

  syncPolicy:
    automated:             # Enable auto-sync
      prune: true          # Delete resources removed from Git
      selfHeal: true       # Revert manual changes in cluster
    syncOptions:
      - CreateNamespace=true
      - ApplyOutOfSyncOnly=true

Application States

Project (AppProject)

Projects group applications and restrict what they can do. Think of them as "tenants" within ArgoCD.

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: team-frontend
  namespace: argocd
spec:
  description: "Frontend team applications"

  # Which Git repos are allowed
  sourceRepos:
    - 'https://github.com/myorg/frontend-*'

  # Which clusters/namespaces are allowed
  destinations:
    - server: https://kubernetes.default.svc
      namespace: 'frontend-*'

  # Which Kubernetes resource types are allowed
  clusterResourceWhitelist:
    - group: ''
      kind: Namespace
  namespaceResourceWhitelist:
    - group: 'apps'
      kind: Deployment
    - group: ''
      kind: Service
    - group: 'networking.k8s.io'
      kind: Ingress

  # Role-based access for this project
  roles:
    - name: developer
      description: Frontend developers
      policies:
        - p, proj:team-frontend:developer, applications, get, team-frontend/*, allow
        - p, proj:team-frontend:developer, applications, sync, team-frontend/*, allow

Repository

ArgoCD needs to know how to access your Git repos. Repos can be configured via UI, CLI, or declaratively.

# Via CLI
argocd repo add https://github.com/myorg/my-repo.git \
  --username myuser \
  --password mytoken

# Via Secret (declarative)
apiVersion: v1
kind: Secret
metadata:
  name: my-repo
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  url: https://github.com/myorg/my-repo.git
  username: myuser
  password: ghp_xxxxxxxxxxxxxxxxxxxx
  type: git

Cluster

By default, ArgoCD can deploy to the cluster it runs in. For additional clusters:

# Add an external cluster
argocd cluster add my-eks-context

# This creates a ServiceAccount in the target cluster
# and stores the credentials in ArgoCD

6. Your First ArgoCD Application — Hands-On

Let's deploy a real application step by step. We'll deploy an Nginx web app using a Git repository.

Step 1: Create the Git Repository Structure

my-app-config/
├── base/
│   ├── deployment.yaml
│   ├── service.yaml
│   └── kustomization.yaml
└── overlays/
    ├── dev/
    │   ├── kustomization.yaml
    │   └── replica-patch.yaml
    └── prod/
        ├── kustomization.yaml
        └── replica-patch.yaml

base/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-app
  labels:
    app: nginx-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-app
  template:
    metadata:
      labels:
        app: nginx-app
    spec:
      containers:
        - name: nginx
          image: nginx:1.25.3
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 250m
              memory: 256Mi

base/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: nginx-app
spec:
  selector:
    app: nginx-app
  ports:
    - port: 80
      targetPort: 80
  type: ClusterIP

base/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml
  - service.yaml

overlays/dev/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: dev
resources:
  - ../../base
patches:
  - path: replica-patch.yaml

overlays/dev/replica-patch.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-app
spec:
  replicas: 1

Step 2: Create the ArgoCD Application

Option A: Using CLI

argocd app create nginx-dev \
  --repo https://github.com/myorg/my-app-config.git \
  --path overlays/dev \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace dev \
  --sync-policy automated \
  --auto-prune \
  --self-heal

Option B: Using YAML (Recommended — it's GitOps all the way down!)

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: nginx-dev
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/myorg/my-app-config.git
    targetRevision: main
    path: overlays/dev
  destination:
    server: https://kubernetes.default.svc
    namespace: dev
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Step 3: Verify Deployment

# Check via CLI
argocd app get nginx-dev

# Output:
# Name:               argocd/nginx-dev
# Sync Status:        Synced ✅
# Health Status:      Healthy 💚
#
# GROUP  KIND        NAME       STATUS  HEALTH
# apps   Deployment  nginx-app  Synced  Healthy
#        Service     nginx-app  Synced  Healthy

# Check via kubectl
kubectl get all -n dev

Step 4: Make a Change (See GitOps in Action)

# Update the image tag in your Git repo
# Edit base/deployment.yaml, change:
#   image: nginx:1.25.3  →  image: nginx:1.26.0

# Commit and push
git add . && git commit -m "Upgrade nginx to 1.26.0" && git push

# ArgoCD detects the change within 3 minutes (or instantly with webhooks)
# Watch it happen
argocd app get nginx-dev --refresh

7. Sync Policies, Strategies & Waves

Manual vs Automated Sync

syncPolicy: {}  # No "automated" key

# Trigger manually
argocd app sync my-app

syncPolicy:
  automated:
    prune: true
    selfHeal: true
    allowEmpty: false

Sync Options

Sync Waves & Hooks

Sync waves control the order of resource creation. Lower wave numbers are synced first.

# Wave 0: Namespace (created first)
apiVersion: v1
kind: Namespace
metadata:
  name: my-app
  annotations:
    argocd.argoproj.io/sync-wave: "0"
---
# Wave 1: ConfigMap & Secrets
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  annotations:
    argocd.argoproj.io/sync-wave: "1"
---
# Wave 2: Database
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  annotations:
    argocd.argoproj.io/sync-wave: "2"
---
# Wave 3: Application (depends on database)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  annotations:
    argocd.argoproj.io/sync-wave: "3"
---
# Wave 4: Ingress (after app is healthy)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app
  annotations:
    argocd.argoproj.io/sync-wave: "4"

Resource Hooks

# Example: Database migration before deploy
apiVersion: batch/v1
kind: Job
metadata:
  name: db-migrate
  annotations:
    argocd.argoproj.io/hook: PreSync
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
  template:
    spec:
      containers:
        - name: migrate
          image: myorg/migrate:latest
          command: ["./migrate", "up"]
      restartPolicy: Never

8. Helm, Kustomize & Jsonnet Integration

Helm Charts

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: prometheus
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://prometheus-community.github.io/helm-charts
    chart: kube-prometheus-stack
    targetRevision: 55.5.0
    helm:
      releaseName: monitoring
      values: |
        grafana:
          enabled: true
          adminPassword: supersecret
        prometheus:
          prometheusSpec:
            retention: 30d
            storageSpec:
              volumeClaimTemplate:
                spec:
                  resources:
                    requests:
                      storage: 50Gi
      parameters:
        - name: alertmanager.enabled
          value: "true"
  destination:
    server: https://kubernetes.default.svc
    namespace: monitoring

Kustomize

spec:
  source:
    repoURL: https://github.com/myorg/my-app-config.git
    targetRevision: main
    path: overlays/production
    kustomize:
      images:
        - myorg/my-app:v2.1.0
      commonLabels:
        environment: production
      namePrefix: prod-

Plain YAML (Directory of Manifests)

spec:
  source:
    repoURL: https://github.com/myorg/my-app-config.git
    targetRevision: main
    path: manifests/
    directory:
      recurse: true
      include: '*.yaml'
      exclude: 'test-*'

Multiple Sources (ArgoCD 2.6+)

Combine a Helm chart with values from a different Git repo:

spec:
  sources:
    - repoURL: https://prometheus-community.github.io/helm-charts
      chart: kube-prometheus-stack
      targetRevision: 55.5.0
      helm:
        valueFiles:
          - $values/monitoring/values-production.yaml
    - repoURL: https://github.com/myorg/config-values.git
      targetRevision: main
      ref: values

9. Multi-Cluster & Multi-Tenancy

Managing Multiple Clusters

# Add clusters via CLI
argocd cluster add dev-eks-context --name dev-cluster
argocd cluster add staging-gke-context --name staging-cluster
argocd cluster add prod-aks-context --name prod-cluster

# Deploy to a specific cluster
spec:
  destination:
    server: https://prod-aks.example.com
    namespace: production

Multi-Tenancy with Projects

# Team A: restricted to their namespaces
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: team-a
  namespace: argocd
spec:
  sourceRepos:
    - 'https://github.com/myorg/team-a-*'
  destinations:
    - server: '*'
      namespace: 'team-a-*'
  clusterResourceWhitelist: []

10. RBAC & SSO (Security)

RBAC Configuration

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-rbac-cm
  namespace: argocd
data:
  policy.default: role:readonly
  policy.csv: |
    # Admins can do everything
    p, role:admin, applications, *, */*, allow
    p, role:admin, clusters, *, *, allow
    p, role:admin, repositories, *, *, allow
    p, role:admin, projects, *, *, allow

    # Developers can view and sync, but not delete
    p, role:developer, applications, get, */*, allow
    p, role:developer, applications, sync, */*, allow
    p, role:developer, applications, action/*, */*, allow
    p, role:developer, logs, get, */*, allow

    # Frontend team — only their project
    p, role:frontend-dev, applications, *, team-frontend/*, allow

    # Map SSO groups to roles
    g, my-org:platform-team, role:admin
    g, my-org:developers, role:developer
    g, my-org:frontend, role:frontend-dev

SSO with GitHub

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
data:
  url: https://argocd.mycompany.com
  dex.config: |
    connectors:
      - type: github
        id: github
        name: GitHub
        config:
          clientID: $dex.github.clientID
          clientSecret: $dex.github.clientSecret
          orgs:
            - name: my-org
              teams:
                - platform-team
                - developers
                - frontend

SSO with Okta/OIDC

data:
  url: https://argocd.mycompany.com
  oidc.config: |
    name: Okta
    issuer: https://mycompany.okta.com/oauth2/default
    clientID: $oidc.okta.clientID
    clientSecret: $oidc.okta.clientSecret
    requestedScopes:
      - openid
      - profile
      - email
      - groups

11. ApplicationSets — Dynamic App Generation

ApplicationSets automatically generate ArgoCD Applications from templates. Instead of manually creating 50 Application manifests, you write ONE ApplicationSet.

Generator: Git Directory

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: microservices
  namespace: argocd
spec:
  generators:
    - git:
        repoURL: https://github.com/myorg/microservices-config.git
        revision: main
        directories:
          - path: services/*
  template:
    metadata:
      name: '{{path.basename}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/myorg/microservices-config.git
        targetRevision: main
        path: '{{path}}'
      destination:
        server: https://kubernetes.default.svc
        namespace: '{{path.basename}}'
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

Generator: List

spec:
  generators:
    - list:
        elements:
          - cluster: dev
            url: https://dev.k8s.example.com
            values:
              replicas: "1"
          - cluster: staging
            url: https://staging.k8s.example.com
            values:
              replicas: "2"
          - cluster: prod
            url: https://prod.k8s.example.com
            values:
              replicas: "5"
  template:
    metadata:
      name: 'my-app-{{cluster}}'
    spec:
      source:
        path: 'overlays/{{cluster}}'
      destination:
        server: '{{url}}'

Generator: Matrix (Combining)

Deploy every microservice to every cluster:

spec:
  generators:
    - matrix:
        generators:
          - git:
              repoURL: https://github.com/myorg/config.git
              directories:
                - path: services/*
          - list:
              elements:
                - cluster: dev
                  url: https://dev.k8s.example.com
                - cluster: prod
                  url: https://prod.k8s.example.com
  template:
    metadata:
      name: '{{path.basename}}-{{cluster}}'

Generator: Pull Request

Create preview environments for every open PR:

spec:
  generators:
    - pullRequest:
        github:
          owner: myorg
          repo: my-app
          tokenRef:
            secretName: github-token
            key: token
          labels:
            - preview
  template:
    metadata:
      name: 'preview-{{number}}'
    spec:
      source:
        repoURL: https://github.com/myorg/my-app.git
        targetRevision: '{{branch}}'
        path: k8s/preview
        kustomize:
          images:
            - 'myorg/my-app:pr-{{number}}'
      destination:
        server: https://kubernetes.default.svc
        namespace: 'preview-{{number}}'

12. Notifications & Webhooks

Notifications to Slack

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-notifications-cm
  namespace: argocd
data:
  service.slack: |
    token: $slack-token

  template.app-sync-succeeded: |
    slack:
      attachments: |
        [{
          "title": "{{.app.metadata.name}} synced!",
          "color": "#18be52",
          "fields": [{
            "title": "Sync Status",
            "value": "{{.app.status.sync.status}}",
            "short": true
          }, {
            "title": "Repository",
            "value": "{{.app.spec.source.repoURL}}",
            "short": true
          }]
        }]

  template.app-health-degraded: |
    slack:
      attachments: |
        [{
          "title": "{{.app.metadata.name}} is DEGRADED!",
          "color": "#f4c030"
        }]

  trigger.on-sync-succeeded: |
    - when: app.status.operationState.phase in ['Succeeded']
      send: [app-sync-succeeded]

  trigger.on-health-degraded: |
    - when: app.status.health.status == 'Degraded'
      send: [app-health-degraded]

  subscriptions: |
    - recipients:
        - slack:platform-alerts
      triggers:
        - on-sync-succeeded
        - on-health-degraded

Git Webhooks (Faster Detection)

By default, ArgoCD polls Git every 3 minutes. Webhooks make detection instant:

# In GitHub repo → Settings → Webhooks:
# URL:    https://argocd.mycompany.com/api/webhook
# Secret: (your webhook secret)
# Events: Push events

# Configure in ArgoCD
apiVersion: v1
kind: Secret
metadata:
  name: argocd-secret
  namespace: argocd
stringData:
  webhook.github.secret: my-webhook-secret

13. Health Checks & Resource Hooks

Built-in Health Checks

Custom Health Check (Lua Script)

# argocd-cm ConfigMap
data:
  resource.customizations.health.certmanager.io_Certificate: |
    hs = {}
    if obj.status ~= nil then
      if obj.status.conditions ~= nil then
        for i, condition in ipairs(obj.status.conditions) do
          if condition.type == "Ready" and condition.status == "True" then
            hs.status = "Healthy"
            hs.message = condition.message
            return hs
          end
        end
      end
    end
    hs.status = "Progressing"
    hs.message = "Waiting for certificate to be issued"
    return hs

Custom Resource Actions

data:
  resource.customizations.actions.apps_Deployment: |
    discovery.lua: |
      actions = {}
      actions["restart"] = {["disabled"] = false}
      return actions
    definitions:
      - name: restart
        action.lua: |
          local os = require("os")
          if obj.spec.template.metadata == nil then
            obj.spec.template.metadata = {}
          end
          if obj.spec.template.metadata.annotations == nil then
            obj.spec.template.metadata.annotations = {}
          end
          obj.spec.template.metadata.annotations["kubectl.kubernetes.io/restartedAt"] = os.date("!%Y-%m-%dT%H:%M:%SZ")
          return obj

14. Secrets Management

The biggest challenge in GitOps: you can't store secrets in Git! Here are the approaches:

helm install sealed-secrets \
  sealed-secrets/sealed-secrets \
  -n kube-system

kubeseal --format yaml \
  < secret.yaml \
  > sealed-secret.yaml
# Safe to commit!

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: my-secret
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: ClusterSecretStore
  target:
    name: my-app-secret
  data:
    - secretKey: db-password
      remoteRef:
        key: prod/my-app/db
        property: password

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
  annotations:
    avp.kubernetes.io/path: "secret/data/myapp"
stringData:
  password: <password>
# AVP replaces at render time

sops --encrypt --in-place secret.yaml

# In Git:
data:
  password: ENC[AES256_GCM,...]

# KSOPS plugin decrypts
# before applying

15. Diff Strategies & Ignore Differences

spec:
  ignoreDifferences:
    # Ignore replicas (managed by HPA)
    - group: apps
      kind: Deployment
      jsonPointers:
        - /spec/replicas

    # Ignore annotations added by webhooks
    - group: apps
      kind: Deployment
      jqPathExpressions:
        - .metadata.annotations["sidecar.istio.io/inject"]

    # Ignore on all Services
    - group: ""
      kind: Service
      jsonPointers:
        - /spec/clusterIP
        - /metadata/annotations

Global Ignore (All Applications)

# argocd-cm ConfigMap
data:
  resource.compareoptions: |
    ignoreAggregatedRoles: true
  resource.customizations.ignoreDifferences.all: |
    managedFieldsManagers:
      - kube-controller-manager
      - kube-scheduler
    jsonPointers:
      - /metadata/managedFields

16. Disaster Recovery & HA

HA Architecture

kubectl apply -n argocd \
  -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/ha/install.yaml

Backup & Restore

# Export all ArgoCD resources
argocd admin export -n argocd > argocd-backup.yaml

# Restore from backup
argocd admin import -n argocd < argocd-backup.yaml

# Alternative: Velero
velero backup create argocd-backup --include-namespaces argocd

17. ArgoCD with CI/CD Pipelines

The Golden Pattern: Separate CI and CD

GitHub Actions Example

# .github/workflows/ci.yaml
name: CI Pipeline
on:
  push:
    branches: [main]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run tests
        run: npm test
      - name: Build and push Docker image
        run: |
          docker build -t myorg/my-app:${{ github.sha }} .
          docker push myorg/my-app:${{ github.sha }}
      - name: Update config repo
        run: |
          git clone https://x-access-token:${{ secrets.TOKEN }}@github.com/myorg/my-app-config.git
          cd my-app-config/overlays/production
          kustomize edit set image myorg/my-app=myorg/my-app:${{ github.sha }}
          git config user.name "CI Bot"
          git config user.email "ci@myorg.com"
          git add . && git commit -m "Deploy ${{ github.sha }}" && git push

ArgoCD Image Updater (Alternative)

# Annotate your Application
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app
  annotations:
    argocd-image-updater.argoproj.io/image-list: myapp=myorg/my-app
    argocd-image-updater.argoproj.io/myapp.update-strategy: semver
    argocd-image-updater.argoproj.io/myapp.allow-tags: regexp:^v\d+\.\d+\.\d+$
    argocd-image-updater.argoproj.io/write-back-method: git

18. Monitoring & Observability

Prometheus Metrics

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: argocd-metrics
  namespace: argocd
spec:
  selector:
    matchLabels:
      app.kubernetes.io/part-of: argocd
  endpoints:
    - port: metrics

Key Metrics to Monitor

Alerting Rules

groups:
  - name: argocd
    rules:
      - alert: ArgoCD_AppOutOfSync
        expr: argocd_app_info{sync_status="OutOfSync"} == 1
        for: 30m
        labels:
          severity: warning
        annotations:
          summary: "App {{ $labels.name }} out of sync for 30m"

      - alert: ArgoCD_AppDegraded
        expr: argocd_app_info{health_status="Degraded"} == 1
        for: 10m
        labels:
          severity: critical
        annotations:
          summary: "App {{ $labels.name }} is degraded"

      - alert: ArgoCD_SyncFailed
        expr: increase(argocd_app_sync_total{phase="Failed"}[1h]) > 3
        labels:
          severity: critical

Grafana: Import Dashboard ID 14584 for a complete ArgoCD overview.

19. Troubleshooting Common Issues

spec:
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jsonPointers:
        - /spec/template/metadata/annotations

# Check repo server logs
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-repo-server --tail=100

# Test locally
helm template my-chart ./charts/my-chart -f values.yaml
kustomize build overlays/production

repoServer:
  resources:
    limits:
      cpu: 2
      memory: 2Gi
# Plus:
syncOptions:
  - ApplyOutOfSyncOnly=true

# Check ArgoCD RBAC
argocd admin settings rbac can role:developer sync applications '*/*'

# Check K8s RBAC
kubectl auth can-i create deployments \
  --as=system:serviceaccount:argocd:argocd-application-controller \
  -n target-namespace

General Debugging Commands

argocd app get my-app              # App details
argocd app diff my-app             # View sync diff
argocd app get my-app --refresh    # Force refresh from Git
argocd app get my-app --hard-refresh  # Invalidate cache

# Logs
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-application-controller -f
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-server -f
kubectl get events -n argocd --sort-by='.lastTimestamp'

20. Best Practices & Production Checklist

Repository Structure

# App repo (source code + Dockerfile)
myorg/my-app/
├── src/
├── Dockerfile
└── .github/workflows/ci.yaml   # CI only

# Config repo (Kubernetes manifests)
myorg/my-app-config/
├── base/
│   ├── deployment.yaml
│   ├── service.yaml
│   └── kustomization.yaml
└── overlays/
    ├── dev/
    ├── staging/
    └── production/

Production Checklist

App of Apps Pattern

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: root-app
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/myorg/argocd-apps.git
    targetRevision: main
    path: apps/
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd

# apps/ directory:
# ├── nginx-dev.yaml
# ├── nginx-prod.yaml
# ├── prometheus.yaml
# ├── cert-manager.yaml
# └── external-secrets.yaml

21. ArgoCD vs Other CD Tools

22. Real-World Scenario: Full Production Setup

Repository Layout

myorg/platform-config/
├── argocd/                          # ArgoCD itself
│   ├── base/
│   └── overlays/production/
├── apps/                            # App-of-Apps root
│   ├── dev-apps.yaml
│   ├── staging-apps.yaml
│   └── prod-apps.yaml
├── projects/                        # AppProjects
│   ├── team-backend.yaml
│   ├── team-frontend.yaml
│   └── platform.yaml
└── services/                        # Service configs
    ├── auth-service/
    │   ├── base/
    │   └── overlays/ (dev/staging/production)
    ├── payment-service/
    └── order-service/

Production ApplicationSet

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: production-services
  namespace: argocd
spec:
  generators:
    - git:
        repoURL: https://github.com/myorg/platform-config.git
        revision: main
        directories:
          - path: services/*/overlays/production
  template:
    metadata:
      name: 'prod-{{path[1]}}'
      labels:
        environment: production
    spec:
      project: platform
      source:
        repoURL: https://github.com/myorg/platform-config.git
        targetRevision: main
        path: '{{path}}'
      destination:
        server: https://prod.k8s.mycompany.com
        namespace: '{{path[1]}}'
      syncPolicy:
        syncOptions:
          - CreateNamespace=true
          - ApplyOutOfSyncOnly=true
          - PruneLast=true
      ignoreDifferences:
        - group: apps
          kind: Deployment
          jsonPointers:
            - /spec/replicas

Production Deployment Workflow

Rollback in Action

# Option 1: Git revert (recommended)
git revert HEAD && git push

# Option 2: ArgoCD rollback
argocd app rollback prod-auth-service

# Option 3: Sync to a specific commit
argocd app sync prod-auth-service --revision abc123def