Kubernetes default QoS for whole namespace

Question

Is there any method of setting default QoS for all of the nodes from given namespace? Or just set QoS on namespace level instead of pod level. I mean the situation where every existing and every newly created pod will have this default setting for given namespace.

Eg. All pods in namespace kube-system or super-important got QoS level "guaranted".

I want to set the QoS in such a way that in case of any problems with the resources pods from kube-system (or other important groups/namespaces), they be removed last, later than the less important ones.

P.S. I got advice about admission-controllers but i didn't see correlation to QoS.

Answer 1

It's possible. Some basic information about QoS:

Kubernetes uses QoS classes to make decisions about scheduling and evicting Pods.

For a Pod to be given a QoS class of Guaranteed:

• Every Container in the Pod must have a memory limit and a memory request, and they must be the same.
• Every Container in the Pod must have a CPU limit and a CPU request, and they must be the same.

A Pod is given a QoS class of Burstable if:

• The Pod does not meet the criteria for QoS class Guaranteed. At least one Container in the Pod has a memory or CPU request.

For a Pod to be given a QoS class of BestEffort:

• The Containers in the Pod must not have any memory or CPU limits or requests.

Here is an example of how to set the Guaranteed QoS for all pods in the namespace qos-test.

Let's create a namespace qos-test:

$ kubectl create namespace qos-test

Next, let's create a LimitRange object YAML file (CPU and Memory should be present, limits and requests should be the same):

$ cat <<EOF > limitRange.yaml
apiVersion: v1
kind: LimitRange
metadata:
  name: mem-limit-gua
spec:
  limits:
  - default:
      cpu: 100m
      memory: 512Mi
    defaultRequest:
      cpu: 100m
      memory: 256Mi
    type: Container
EOF

Then let’s apply it to the namespace qos-test:

$ kubectl create -f limitRange.yaml --namespace=qos-test

Now, let's create a pod (CPU or Memory requests and limits must not be present in Pod spec):

$ cat <<EOF > default-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: default-pod
spec:
  containers:
  - name: default-ctr
    image: nginx
EOF

$ kubectl create -f default-pod.yaml --namespace=qos-test

Finally, let's check what we've got:

$ kubectl get namespaces
...
$ kubectl get limitrange --all-namespaces -o wide
...
$ kubectl get limitrange -o yaml -n qos-test
...
$ kubectl get pod default-pod -o yaml -n qos-test

apiVersion: v1
kind: Pod
metadata:
  ...
  name: default-pod
  namespace: qos-test
  ...
spec:
  containers:
  - image: nginx
    imagePullPolicy: Always
    name: default-ctr
    resources:
      limits:
        cpu: 100m
        memory: 256Mi
      requests:
        cpu: 100m
        memory: 256Mi
    ...
status:
  ...
  hostIP: 10.0.2.15
  phase: Running
  podIP: 172.17.0.10
  qosClass: Guaranteed
  ...

As you can see default-pod got the qosClass: Guaranteed.

To create pods with different QoS please refer to this part of documentation:

• Configure Quality of Service for Pods

You can find more details about configuring default limits for a Namespace in this article:

• Configure Default Memory Requests and Limits for a Namespace