I deployed Splunk Forwarder in my kubernetes cluster using this blog.
http://jasonpoon.ca/2017/04/03/kubernetes-logging-with-splunk/
I have 4 files at
/opt/splunk/etc/system/local
inputs.conf
server.conf
limits.conf
outputs.conf
my inputs.conf looks like this.
[default]
host = testtest
[monitor:///usr/local/tomcat/logs]
whitelist=test.log|.log_WHITELIST_|test
index= abc
sourcetype=log4j
[splunktcp://9997]
compressed = false
I am not able to get the log in Splunk UI. when I am deploying it as side car. but If I use same docker image locally but putting dummy logs file. i can see the logs. Then why it is not working with kubernates?
I have checked the splunkd.log as well.
/opt/splunk/var/log/splunk # tail splunkd.log
12-07-2018 10:43:38.793 +0000 INFO TailingProcessor - Adding watch on path: /opt/splunk/var/spool/splunk.
12-07-2018 10:43:38.793 +0000 INFO TailingProcessor - Adding watch on path: /usr/local/tomcat/logs.
12-07-2018 10:43:38.795 +0000 INFO loader - Limiting REST HTTP server to 21845 sockets
12-07-2018 10:43:38.795 +0000 INFO loader - Limiting REST HTTP server to 657 threads
12-07-2018 10:43:38.798 +0000 INFO TailReader - Registering metrics callback for: batchreader0
12-07-2018 10:43:38.798 +0000 INFO TailReader - Starting batchreader0 thread
12-07-2018 10:43:38.798 +0000 INFO TailReader - Registering metrics callback for: tailreader0
12-07-2018 10:43:38.798 +0000 INFO TailReader - Starting tailreader0 thread
12-07-2018 10:43:38.850 +0000 INFO TcpOutputProc - Connected to idx=52.204.198.184:9997 using ACK.
12-07-2018 10:44:08.358 +0000 WARN AuthenticationManagerSplunk - Seed file is not present. Defaulting to generic username/pass pair.
Any suggestion how to fix this. Stuck here for days. Do I have to open any port under kubernates? although I can ping splunk server from the splunk container.