How to monitor if LogStash is fully loaded?

Question

I am using LogStash to collect the usage logs of my service. I am wondering how could I know if a LogStash is fully loaded and should add more servers to handle the logs? I don't want to miss any of the logs.

Any suggestion would be helpful, thanks in advance:)

Answer 1

First, Logstash is only parsing the logs and send the logs event to a place to store, ex: Elasticsearch!

If you are using elasticsearch as your logs storage, you can try to install marvel. It is a plugin of elastcisearch, after you have install it, you want use it to view your log service status, ex: disk space, CPU, memory usage. So you want use it to monitor your service and know whether the service is fully loaded.

Updated:

As discuss in the comments, your output is file. What you need is an broker. Logstash default message queue is 20 events. So, when the queue is full, as your say, some logs will be drop. So, you need an broker to buffer your logs. The recommandate broker for logstash is Redis. You can run two Logstash instances, one is receive the logs from server, (input is tcp and output is redis), the other one is your indexer (input is redis and output is file).