Micrometer - Prometheus Gauge displays NaN

问题:

I am trying to generate Prometheus metrics with using Micrometer.io with Spring Boot 2.0.0.RELEASE.

When I am trying to expose the size of a List as Gauge, it keeps displaying NaN. In the documentation it says that;

It is your responsibility to hold a strong reference to the state object that you are measuring with a Gauge.

I have tried some different ways but I could not solve the problem. Here is my code with some trials.

import io.micrometer.core.instrument.*;
import io.swagger.backend.model.Product;
import io.swagger.backend.service.ProductService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;

import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;

@RestController
@RequestMapping("metrics")
public class ExampleController {

    private AtomicInteger atomicInteger = new AtomicInteger();

    private ProductService productService;
    private final Gauge productGauge;

    @Autowired
    public HelloController(ProductService productService,
                           MeterRegistry registry) {

        this.productService = productService;

        createGauge("product_gauge", productService.getProducts(), registry);
    }

    private void createGauge(String metricName, List<Product> products,
                                    MeterRegistry registry) {

        List<Product> products = productService.getProducts();

        // #1
        // this displays product_gauge as NaN
        AtomicInteger n = registry.gauge("product_gauge", new AtomicInteger(0));
        n.set(1);
        n.set(2);

        // #2
        // this also displays product_gauge as NaN
        Gauge
            .builder("product_gauge", products, List::size)
            .register(registry);

        // #3
        // this displays also NaN
        testListReference = Arrays.asList(1, 2);
        Gauge
            .builder("random_gauge", testListReference, List::size)
            .register(registry);

        // #4
        // this also displays NaN
        AtomicInteger currentHttpRequests = registry.gauge("current.http.requests", new AtomicInteger(0));
    }

    @GetMapping(path = "/product/decrement")
    public Counter decrementAndGetProductCounter() {
        // decrement the gague by one
    }
}

Is there anyone who can help with this issue? Any help would be appreciated.

回答1:

In all cases, you must hold a strong reference to the observed instance. When your createGauge() method is exited, all function stack allocated references are eligible for garbage collection.

For #1, pass your atomicInteger field like this: registry.gauge("my_ai", atomicInteger);. Then increment/decrement as you wish. Whenever micrometer needs to query it, it will as long as it finds the reference.

For #2, pass your productService field and a lambda. Basically whenever the gauge is queried, it will call that lambda with the provided object: registry.gauge("product_gauge", productService, productService -> productService.getProducts().size());

(No guarantee regarding syntax errors.)

回答2:

I wasn't able to use @panser solution 'cause I'm using gauges with labels. My solution involved the creation of com.google.common.util.concurrent.AtomicDouble cache with io.micrometer.core.instrument.Tag's key and values as map key, heres goes:

    private static final Map<String, AtomicDouble> GAUGE_CACHE = new HashMap<>();

    public void handleGauge(String name, List<Tag> tags, double value) {
        String gaugeKey = this.gaugeKey(name, tags);
        if (!GAUGE_CACHE.containsKey(gaugeKey)) {
            GAUGE_CACHE.put(gaugeKey, new AtomicDouble());
        }
        Objects.requireNonNull(this.registry.gauge(name, tags, GAUGE_CACHE.get(gaugeKey))).set(value);
    }

    private String gaugeKey(String name, List<Tag> tags) {
        return name + ":" + tags.stream().map(tag -> tag.getKey() + tag.getValue()).collect(Collectors.joining(""));
    }

That worked pretty well for my needs, hopefully help other people.

回答3:

I had the same issue with Micrometer.io gauges when I used your method #1 meterRegistry.gauge("myGauge", new AtomicDouble()). I am using Scala by the way. I noticed that after I created about 50 gauges, the new gauges after that displayed NaN.

Instead I used:

val atomicDouble = new AtomicDouble()
Gauge
  .builder("myGauge", atomicDouble, new AtomicDoubleToDoubleFunction)
  .strongReference(true)
  .register(meterRegistry)

with

class AtomicDoubleToDoubleFunction extends ToDoubleFunction[AtomicDouble] {
  override def applyAsDouble(value: AtomicDouble): Double = value.doubleValue()
}

This fixed the NaN issue, and all of my gauges appear correctly. I found the .strongReference(true) example from https://www.codota.com/code/java/classes/io.micrometer.core.instrument.Gauge .

回答4:

my example for gauge using

private final AtomicLong countTryUsers = new AtomicLong(0);
Metrics.gauge("app.countTry", countTryUsers);

public void updateCountTryUsers(Long countTryUsersDb){
   countTryUsers.set(countTryUsersDb);
}

so I register app.countTry just once, and then just update AtomicLong countTryUsers over custom method updateCountTryUsers()