How to rename label within a metric in Prometheus

2020-07-06 03:18发布

问题:

I have a query:

node_systemd_unit_state{instance="server-01",job="node-exporters",name="kubelet.service",state="active"} 1

I want the label name being renamed (or replaced) to unit_name ONLY within the node_systemd_unit_state metric. So, desired result is:

node_systemd_unit_state{instance="server-01",job="node-exporters",unit_name="kubelet.service",state="active"} 1

There are many other metrics with a label name name in the node-exporters job. That's why I can't use relabel config across the job.

回答1:

You can have more than one source label, so:

- source_labels: [__name__, name]
  regex: "node_systemd_unit_state;(.+)"
  target_label: unit_name

If the label name isn't matching up with other metrics/exporters, you should file a bug with them. Relabelling like this should only be a temporary solution while a proper fix is being pursued.



回答2:

you can use the label_replace function in promQL, but it also add the label, don't replace it

label_replace(
  <vector_expr>, "<desired_label>", "$1", "<existing_label>", "(.+)"
)

label_replace(
node_systemd_unit_state{instance="server-01",job="node-exporters",name="kubelet.service",state="active"},
"unit_name","$1","name", "(.+)"
)

So, to avoid the repetition you can add:

sum(label_replace(
    node_systemd_unit_state{instance="server-01",job="node-exporters",name="kubelet.service",state="active"},
    "unit_name","$1","name", "(.+)"
    )
)by(unit_name)


回答3:

I got tired of all the fragmented documentation and I feel I provided a better answer in this post here: https://medium.com/@texasdave2/replace-and-remove-a-label-in-a-prometheus-query-9500faa302f0

Replace is not a true REPLACE

Your goal is to simply replace the old label name “old_job_id” with a new label name “new_task_id”. Prometheus label_replace will really “add” the new label name. It will preserve the old label name as well… So, that could be a problem, it’s not a true “replace in place”.

So if you want to “add” your new label name and “remove” the old label name, you need to do this:

sum without (old_job_id) (label_replace(metric, "new_task_id", "$1", "old_job_id", "(.*)"))

Here’s how this reads:

  • sum without (old_job_id) will remove the old label name from the query output

  • metric is your metric, like “node_filesystem_avail_bytes”

  • “new_task_id” is where you would put your new label name

  • “$1” is regex for using the string in new label name, don’t change this

  • “old_job_id” is where you’ll put your old label, the one you want to get rid of (.*……. that mess is regex that will replace the whole label name