I believe I have managed to get myself confused and would like to request assistance in resolving a conundrum I have with my SPLUNK infrastructure.
I have a new SPLUNK Clustered Environment with 1 Search Head, 1 Deployment / License server, 1 Cluster manager and 2 Indexers. I also have a corporate network and my DMZ network. I am using the SPLUNK TA for Windows as well as the SPLUNK TA for SYSMON.
On my SPLUNK infrastructure, I have the SPLUNK TA for Windows and SPLUNK TA for SYSMON installed on the Cluster Manager, The Deployment Server, and the Search Head. The cluster manager deploys a "master app" to each of the Indexers negating the need for me to manually do it. I have created a new application called "mydomain_windows_events". I have created 4 folders in this application (default, bin, local and meta). I have created inputs.conf file cloned off the SPLUNK TA for Windows. I have modified the inputs.conf file to capture events I would like.
For my corporate connected machine, it has a Universal Forwarder installed to it that sends directly to the CM and the data transmits successfully and all my data is indexed properly and extracts the fields as required.
I also have a SPLUNK Heavy Forwarder in our DMZ network that is also a Deployment server (similar application configured as non-DMZ above). All the DMZ Windows devices will send their logs to the Heavy Forwarder to then be forwarded to a test standalone SPLUNK Enterprise instance. I confirmed that my standalone Enterprise instance receives the logs fine and all of the fields are extracted properly. However, I then configured my SPLUNK Heavy Forwarder via "Forwarding and Receiving" to send to my clustered indexers (I added indexer1:9997, indexer2:9997) and I am no longer getting the extract fields as before.
What have I done incorrectly? I think this may be my understanding as to whether I should configure my Heavy Forwarder to send to the CM via port 8089 as per my non-DMZ configuration? Or whether I should be sending the Heavy Forwarder directly to the indexers. I suspect this is where the problem is. As I said the data gets there but I am not getting the extraction and just a block of logs.
The sourcetype on both the extracted and unextracted show the same. For example, when viewing when extraction has worked, the sourcetype reads "WinEventLog". When viewing when the extraction has NOT worked the sourcetype reads "WinEventLog". So it appears the same. Is this a problem with cooked data (i.e. not raw)? I read the SPLUNK article about Heavy Forwarders and it states that "By default, forwarders send cooked data (universal forwarders send unparsed data and heavy forwarders send parsed data.)". So what do I do with my heavy forwarder so that the data being received by my clustered environment shows correctly?
↧