I wanted one project where a git push ends up as a running pod with no manual steps, then the same thing on real AWS. I built it as a beginner, step by step, and wrote down what actually broke. Code and screenshots: https://github.com/RobinSishodia/aws-eks-gitops-argocd
What it does
- I push a change to GitHub.
- Jenkins polls the repo (every 2 minutes), builds a Docker image and pushes it to Docker Hub.
- Jenkins commits the new image tag into
k8s/deployment.yaml. - ArgoCD sees the manifest change and syncs it to Kubernetes (automated sync, prune and self-heal).
I proved it end to end by changing the heading in the Flask app and pushing. A few minutes later the new heading was served by freshly rolled-out pods, with no click after the push.
Repo layout
app/ Flask app + Dockerfile k8s/ deployment.yaml, service.yaml argocd-app.yaml ArgoCD Application (path: k8s, automated sync) Jenkinsfile build, push, update manifest terraform/ VPC + EKS (modules) docs/ screenshots The Jenkins pipeline
The pipeline has three stages: build the image, push it, then update the manifest and push that commit back to GitHub. The tag is the Jenkins build number.
pipeline { agent any environment { IMAGE = 'robinsishodia/gitops-demo' TAG = "${env.BUILD_NUMBER}" } stages { stage('CI') { when { not { expression { sh(script: 'git log -1 --pretty=%B', returnStdout: true).contains('[skip ci]') } } } stages { stage('Build image') { steps { sh 'docker build -t $IMAGE:$TAG app/' } } stage('Push image') { /* docker login with the 'dockerhub' credential, then docker push */ } stage('Update manifest') { steps { // GitHub credential injected with withCredentials sh ''' sed -i "s#image: .*#image: $IMAGE:$TAG#" k8s/deployment.yaml git add k8s/deployment.yaml git commit -m "ci: deploy $IMAGE:$TAG [skip ci]" || exit 0 git push origin HEAD:main ''' } } } } } } The when block is the loop guard, explained below. Credentials live in Jenkins, never in the repo.
The ArgoCD side
ArgoCD watches the k8s/ folder and keeps the cluster equal to Git. ArgoCD polls Git about every 3 minutes. If you do not want to wait, force a refresh:
kubectl -n argocd annotate application gitops-demo argocd.argoproj.io/refresh=hard --overwrite That is how I saw the old pods (12 minutes old) replaced by new ones within about a minute.
Terraform and EKS
I rebuilt the same cluster on AWS with Terraform: a VPC (2 AZs, public and private subnets, a single NAT gateway) and an EKS cluster with a managed node group of 2 nodes. I installed ArgoCD on it, applied the same Application manifest, and watched it come up Synced and Healthy. Then I ran terraform destroy (56 resources), checked in the console and CLI that nothing was left, and deleted the IAM access key, so the lab cost nothing beyond free-plan credits.
The workflow I used for safety: terraform plan -out=tfplan, read the plan, then terraform apply tfplan.
What went wrong, and the fixes
1. Jenkins rebuilt its own commits
The pipeline pushes a commit, so polling would trigger another build, forever. Jenkins core does not honour [skip ci], so I wrapped the stages in a when guard that skips a build whose latest commit message contains it (see the Jenkinsfile above). I also tested it: one build ran fully, the next was skipped.
A self-inflicted detail: I once wrote the literal text [skip ci] inside my own commit message while explaining the guard, and Jenkins correctly skipped a build I wanted. I fixed it with an empty commit with a neutral message.
2. AWS Free plan instance types
The node group failed with t3.medium not being free-tier eligible. On the Free plan, only certain instance types are allowed. I listed them with the EC2 CLI filtered by free-tier eligibility and switched to c7i-flex.large.
3. Nodes that never joined the cluster
This one cost the most time. The node group sat on "Still creating" for hours. The cause: in version 21 of the terraform-aws-modules EKS module, the self-managed default add-ons are not bootstrapped, so vpc-cni, kube-proxy and CoreDNS were never installed and the nodes had no networking. The fix was to declare them:
addons = { coredns = {} eks-pod-identity-agent = { before_compute = true } kube-proxy = {} vpc-cni = { before_compute = true } } I also added timeouts = { create = "25m" } on the node group so a stuck build fails after 25 minutes instead of running overnight.
4. Small things that cost me time
- Port 8082 was already used by Jenkins, so I used 8090 for the app and 9443 for ArgoCD port-forwards.
- My Terraform plan files were not covered by
*.tfplanin.gitignore; I addedtfplan*before committing. -
kubectlhad two contexts (EKS and kind). Always checkkubectl config current-contextbefore running commands.
What I learned
- GitOps is mostly discipline: the cluster changes only through Git.
- Any pipeline that commits to its own repo needs a loop guard.
- Read the module docs for what is not installed by default.
- Always destroy lab infrastructure, verify it, and delete the access keys.
What is next
A real GitHub webhook instead of polling (Jenkins runs on my laptop, so it needs a public tunnel or a hosted Jenkins), then monitoring with Prometheus and Grafana.
If you spot something I could do better, I would like to hear it in the comments.