jenkins-master-project | Jenkins Plugin : Master Project | Identity Management library

Found the solutions.

I just had to use services/search/jobs/export end point.

Let's find out what was our Jenkins host (which contained the Jenkins job), What was the job's name (this can be parsed / grepped from the source path for the log file) and what was Jenkins's job's actual console output (_raw). Also, lets limit our search for finding the data within last 30 minutes (i.e. earliest=-30m).

There are actually 3 ways to do it.

1) By passing user name and password at command line.

2) By generating a SESSION TOKEN that we can use in any future CURL command in the headers.

3) By generating a --cookie "${COOKIE}" ID and using that. This one is preferred method of all as it replicates cookie values to any backend servers Splunk uses. Cookie name to be used: splunkd_8081

The last 2 solutions depends upon the first method for using a user's credentials to create either the SESSION or COOKIE ID.

Solution 1:

1) Here we'll use our splunk server

2) Pass username and password in the command line

3) Provide Splunk options for finding / fetching Splunk data (for Jenkins logs containing specific lines) and also doing a little extra regex match (so that it'll return exact Jenkins build# for the source path rather than showing 3 more lines for source for the same console output. Jenkins latestBuild, latestSuccessfulBuild etc are symlinks which point to a numbered build and we don't want to list these symlinked source entries in our output so I'm using a regex pattern to find source path where it contains a NUMBERED build# just before the log file in the source path).

4) Then, I'm using | to filter out only 3 fields: host, source, and _raw (that Splunk would return). host contains, which Jenkins server had the Jenkins job. source contains Jenkins job name, build# etc info in it's value. _raw variable contains the Jenkins job's console output (Few lines close to the string / line that we are trying to search for in the console output of a Jenkins job).

NOTE: All these 3 fields are available inside a dictionary variable result so I'm just outputting that.

5) Then, I'm providing the output in json format (you can also use csv). Finally I'm using jq to filter out the information.

NOTE: If you use jq -r ".result._raw" (i.e. _raw field inside dictionary variable result then it'll give you LINE by LINE output for the console output (rather than giving you one blob with \n embedded in it). You can also use sed 's/\\n/\n/g' but jq -r ".result._raw" was easy enough).

Commands ran: