Kubernetes pod gets recreated when deleted

Question

I have started pods with command

$ kubectl run busybox --image=busybox --restart=Never --tty -i --generator=run-pod/v1

Something went wrong, and now I can't delete this Pod.

I tried using the methods described below but the Pod keeps being recreated.

$ kubectl delete pods  busybox-na3tm
pod "busybox-na3tm" deleted
$ kubectl get pods
NAME                                     READY     STATUS              RESTARTS   AGE
busybox-vlzh3                            0/1       ContainerCreating   0          14s

$ kubectl delete pod busybox-vlzh3 --grace-period=0


$ kubectl delete pods --all
pod "busybox-131cq" deleted
pod "busybox-136x9" deleted
pod "busybox-13f8a" deleted
pod "busybox-13svg" deleted
pod "busybox-1465m" deleted
pod "busybox-14uz1" deleted
pod "busybox-15raj" deleted
pod "busybox-160to" deleted
pod "busybox-16191" deleted


$ kubectl get pods --all-namespaces
NAMESPACE   NAME            READY     STATUS              RESTARTS   AGE
default     busybox-c9rnx   0/1       RunContainerError   0          23s

Answer 1

Many answers here tells to delete a specific k8s object, but you can delete multiple objects at once, instead of one by one:

kubectl delete deployments,jobs,services,pods --all -n <namespace>

In my case, I'm running OpenShift cluster with OLM - Operator Lifecycle Manager. OLM is the one who controls the deployment, so when I deleted the deployment, it was not sufficient to stop the pods from restarting.

Only when I deleted OLM and its subscription, the deployment, services and pods were gone.

First list all k8s objects in your namespace:

$ kubectl get all -n openshift-submariner

NAME                                       READY   STATUS    RESTARTS   AGE
pod/submariner-operator-847f545595-jwv27   1/1     Running   0          8d  
NAME                                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/submariner-operator-metrics   ClusterIP   101.34.190.249   <none>        8383/TCP   8d
NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/submariner-operator   1/1     1            1           8d
NAME                                             DESIRED   CURRENT   READY   AGE
replicaset.apps/submariner-operator-847f545595   1         1         1       8d

OLM is not listed with get all, so I search for it specifically:

$ kubectl get olm -n openshift-submariner

NAME                                                      AGE
operatorgroup.operators.coreos.com/openshift-submariner   8d
NAME                                                             DISPLAY      VERSION
clusterserviceversion.operators.coreos.com/submariner-operator   Submariner   0.0.1 

Now delete all objects, including OLMs, subscriptions, deployments, replica-sets, etc:

$ kubectl delete olm,svc,rs,rc,subs,deploy,jobs,pods --all -n openshift-submariner

operatorgroup.operators.coreos.com "openshift-submariner" deleted
clusterserviceversion.operators.coreos.com "submariner-operator" deleted
deployment.extensions "submariner-operator" deleted
subscription.operators.coreos.com "submariner" deleted
service "submariner-operator-metrics" deleted
replicaset.extensions "submariner-operator-847f545595" deleted
pod "submariner-operator-847f545595-jwv27" deleted

List objects again - all gone:

$ kubectl get all -n openshift-submariner
No resources found.

$ kubectl get olm -n openshift-submariner
No resources found.

Answer 2

You need to delete the deployment, which should in turn delete the pods and the replica sets https://github.com/kubernetes/kubernetes/issues/24137

To list all deployments:

kubectl get deployments --all-namespaces

Then to delete the deployment:

kubectl delete -n NAMESPACE deployment DEPLOYMENT

Where NAMESPACE is the namespace it's in, and DEPLOYMENT is the name of the deployment.

In some cases it could also be running due to a job or daemonset. Check the following and run their appropriate delete command.

kubectl get jobs

kubectl get daemonsets.app --all-namespaces

kubectl get daemonsets.extensions --all-namespaces

Answer 3

When the pod is recreating automatically even after the deletion of the pod manually, then those pods have been created using the Deployment. When you create a deployment, it automatically creates ReplicaSet and Pods. Depending upon how many replicas of your pod you mentioned in the deployment script, it will create those number of pods initially. When you try to delete any pod manually, it will automatically create those pod again.

Yes, sometimes you need to delete the pods with force. But in this case force command doesn’t work.