How to get resource utilized memory and CPU by each hive query which is running under single application Id

Question

I am running some bunch of queries using hql files. Usually these all queries will run under a single application Id. How to find out resource utilized by each query which is running under single appication Id.

Answer 1

Hi@sivachandran,

You can monitor the metrics of your cluster. Generally to monitor the sources we use monitoring tools like Grafana, ELK, etc. You can learn the concepts of JMX monitoring for the hive.

Comments

Thanks @MD,

I am using Hive version 1.2 hope that JMX is not available. I want to store the number of core and memory used for that particular query into a hive table. So I want to know where i can get those info.

---

Hi@sivachandran,

In other options, you may find all this information in the log files. Try to find with executor ID.