Hello Splunk Community !!
I've configured Splunk REST API in our environment and I was able to see data when I initially configured it but it's unable to do so frequently. I've tired using Polling Intervals but it didn't workout (default is 60 seconds). Currently, the setup looks like this for the REST API APP
1. Installed the APP on Heavy Forwarder and it's able to forward events to indexers and I can see those in Search Head UI
2. Edited props.conf file to include LINE_BREAKER and TIME_STAMP fields such that it can split events and assign the timestamp of the event based on the data received.
inputs.conf
[rest://CFTest]
auth_type = none
endpoint = https://api.cloudflare.com/client/v4/zones/CFZONE/logs/received?start=$start_time$&end=$end_time$&fields=RayID,ClientIP,EdgeStartTimestamp,ClientRequestHost×tamps=rfc3339
http_header_propertys = X-Auth-Email=XXX@XXX.com,X-Auth-Key=XXXX
http_method = GET
index_error_response_codes = 1
response_type = json
sequential_mode = 0
sourcetype = cloudflare
streaming_request = 0
cookies = __cfduid=d2a7b8efd7e8cefe148fdb2a95369cf9d1522783367
disabled = 0
index = incapsula
polling_interval =
backoff_time = 60
When I was investigating why it's pulling logs infrequently then I came across this information in splunkd.log. Interestingly logs are pulled during the timestamp whenever I see this in the splunkd.log and after that timestamp I can't see it.
04-04-2018 03:39:21.959 +0000 INFO ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/rest_ta/bin/rest.py
04-04-2018 03:39:21.959 +0000 INFO ExecProcessor - interval: run once
04-04-2018 03:39:21.959 +0000 INFO ExecProcessor - interval="5 3 * * *" is a valid cron schedule
If I edit inputs.conf which I have listed above, then again I can see logs around that particular timestamp. I don't know how this schedule is decided and how to change it based on our requirement which is to pull every minute.
Please let me know if anyone came across this situation in your environment and what steps you took to resolve the issue.
Thanks
Venky
↧