count k8s cluster cpu/memory usage with prometheus

Question

I want to count k8s cluster cpu/memory usage (not k8s pod usage) with prometheus, so that i can show in grafana.

I use sum (container_memory_usage_bytes{id="/"}) to get k8s cluster used memory, and topk(1, sum(kube_node_status_capacity_memory_bytes) by (instance)) to get whole k8s cluster memory, but they can not divide since topk function does not return value but vector.

How can i do this?

Answer 1

I have installed Prometheus on google Cloud through the gcloud default applications. The dashboards automatically got deployed with the installation. The following queries are what was used for memory and CPU usage of the cluster:

CPU usage by namespace:

sum(irate(container_cpu_usage_seconds_total[1m])) by (namespace)

Memory usage (no cache) by namespace:

sum(container_memory_rss) by (namespace)

CPU request commitment:

sum(kube_pod_container_resource_requests_cpu_cores) / sum(node:node_num_cpu:sum)

Memory request commitment:

sum(kube_pod_container_resource_requests_memory_bytes) / sum(node_memory_MemTotal)

Answer 2

My main qustion is that topk(1, sum(kube_node_status_capacity_memory_bytes) by (instance)) can not return a value, but now i find that use sum() to covert it can work, whole query as following:

sum(sum (container_memory_usage_bytes{id="/"})by (instance))/sum(topk(1, sum(kube_node_status_capacity_memory_bytes) by (instance)))*100