How can I monitor memory and CPU usage by spark ap

2020-08-01 05:33发布

After running my spark application, I want to monitor its memory and cpu usage to evaluate its performance but couldn't find any option. Is it possible to monitor it? How can I monitor memory and CPU usage by spark application?

1条回答
Rolldiameter
2楼-- · 2020-08-01 05:52

There are a few options:

  1. Ganglia is one
  2. If you're running on your own cluster, HDP or Cloudera both have real time CPU & memory consumption charts.
  3. If you want specific JVM metrics, then I'd recommend FlameGraph, though it's not real time.
  4. There's also Grafana, it's extremely powerful, you can track many metrics with it, and it's real time.
查看更多
登录 后发表回答