Quantcast
Channel: Questions in topic: "heavy-forwarder"
Viewing all articles
Browse latest Browse all 727

Why isn't the Splunk REST API able to pull data frequently?

$
0
0
Hello Splunk Community !! I've configured Splunk REST API in our environment and I was able to see data when I initially configured it but it's unable to do so frequently. I've tired using Polling Intervals but it didn't workout (default is 60 seconds). Currently, the setup looks like this for the REST API APP 1. Installed the APP on Heavy Forwarder and it's able to forward events to indexers and I can see those in Search Head UI 2. Edited props.conf file to include LINE_BREAKER and TIME_STAMP fields such that it can split events and assign the timestamp of the event based on the data received. inputs.conf [rest://CFTest] auth_type = none endpoint = https://api.cloudflare.com/client/v4/zones/CFZONE/logs/received?start=$start_time$&end=$end_time$&fields=RayID,ClientIP,EdgeStartTimestamp,ClientRequestHost&timestamps=rfc3339 http_header_propertys = X-Auth-Email=XXX@XXX.com,X-Auth-Key=XXXX http_method = GET index_error_response_codes = 1 response_type = json sequential_mode = 0 sourcetype = cloudflare streaming_request = 0 cookies = __cfduid=d2a7b8efd7e8cefe148fdb2a95369cf9d1522783367 disabled = 0 index = incapsula polling_interval = backoff_time = 60 When I was investigating why it's pulling logs infrequently then I came across this information in splunkd.log. Interestingly logs are pulled during the timestamp whenever I see this in the splunkd.log and after that timestamp I can't see it. 04-04-2018 03:39:21.959 +0000 INFO ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/rest_ta/bin/rest.py 04-04-2018 03:39:21.959 +0000 INFO ExecProcessor - interval: run once 04-04-2018 03:39:21.959 +0000 INFO ExecProcessor - interval="5 3 * * *" is a valid cron schedule If I edit inputs.conf which I have listed above, then again I can see logs around that particular timestamp. I don't know how this schedule is decided and how to change it based on our requirement which is to pull every minute. Please let me know if anyone came across this situation in your environment and what steps you took to resolve the issue. Thanks Venky

Viewing all articles
Browse latest Browse all 727

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>