
I built a small tool that looks at broken Kubernetes pods and explains what is wrong. It reads the cluster, collects the evidence, and writes a report with a suggested fix. It cannot change anything. That is the whole point of the design.
Then I broke six apps on purpose and ran it against them. This post covers how it works and what actually happened.
kubectl get pods → evidence (status, events, logs, limits) → deterministic rules → LLM explains → Markdown report
OOMKilled means out of memory, ImagePullBackOff means a bad image, FailedScheduling means the pod can’t be placed, and so on.It supports a local model through Ollama, or Claude through the Anthropic API.
There are two separate guards, so a bug in one doesn’t open the door.
In the code. Every cluster call goes through one function that only allows read verbs:
READ_ONLY_VERBS = {"get", "describe", "logs", "top", "auth", "version", "api-resources"}
Anything else raises WriteAttemptBlocked, and there are tests for it.
In the cluster. KubeMedic logs in as its own ServiceAccount. Its ClusterRole only allows get, list and watch, and gives no access to Secrets. Here is what the cluster said when I asked what that identity can do:
get pods yes
list events yes
get pods/log yes
delete pods no
patch deployments no
create pods no
get secrets no
Even if the code guard had a bug, the cluster would refuse the write.
I created a local cluster with kind and deployed six workloads, each broken in a different way:
| App | What I broke | Pod status |
|---|---|---|
| memory-hog | allocates 150 MB with a 64Mi limit | OOMKilled |
| payments-api | required DATABASE_URL missing, exits 1 | Error / CrashLoop |
| web-frontend | image nginx:1.99.99-doesnotexist | ImagePullBackOff |
| report-worker | references a Secret that doesn’t exist | CreateContainerConfigError |
| catalog-svc | liveness probe on port 8080, nginx listens on 80 | CrashLoopBackOff |
| ml-batch | requests 512Gi of memory | Pending |
Run with Ollama and llama3.2:3b on my laptop:
The explanations were useful, but not perfect:
This is why the rules decide what is wrong and the model only explains it. The diagnosis stays correct even when the model’s wording is shaky.