We are having ELK setup with Kibana version 5.6.10. We are facing a time mismatch in displaying logs from different servers.
We are fetching log from 8 IIS server and parsing via Logstash to Elastic search Kibana. While filtering logs for past hour we could notice only 2 server logs were displayed. We have checked filebeat configuration in each IIS servers and found same configuration setup; also verified IIS log time format and other configurations. We could see indexing is happening properly in Elastic Search but while filtering the display option for an hour only throwing results for 2 servers. If we filter for four hours we can see multiple servers with the different time value in the display.
Would like to know anyone facing a similar issue and welcoming solution for it.
I have had the same issue. The issue is with the time zone. Kibana works on UTC by default. Please check if the time zone in your ES docs is same as that of Kibana. You can do so by (kibana) Management Tab ->Advance Settings ->dateFormat:tz
If the time is zone is different, please use 'Today' in kibana time window to check your recent documents.
Alternatively, you can also index your timestamp field with UTC timezone(or your desired time zone) in ES. Then set up kibana with the same timezone as ES to check your documents.
The issue is of the timezone. The server of which the logs are not being displayed are most probably in a different timezone than the timezone of Kibana. This is an issue of Kibana, it doesn't work on global. Here is the issue reported on GIT. You can keep track of this.
https://discuss.elastic.co/t/kibana-timestamp-in-browser-local-time-but-incoming-logs-utc/57501
https://github.com/elastic/kibana/issues/1600
