Jenkins Pipeline - how to get logs from parallel b

Question

Is it possible, and if yes: how?, to get the log output for each parallel step separately?

I.e.:

def projectBranches = [:]
        for (int i = 0; i < projects.size(); i++) {
            def _i = i
            projectBranches[_i] = {
                someFunction(_i)
            }
        }

        parallel projectBranches

Is it now possible to get the log for each of projectBranches[_i]?

Answer 1

You could get your nodes by use of Jenkins REST API: job/test/1/api/json?depth=2

Result should contain something like:

{"_class":"org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode","actions":[{},{},{}],"displayName":"Branch: 0","iconColor":"blue","id":"13","parents":["3"],"running":false,"url":"job/test/1/execution/node/13/"},
{"_class":"org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode","actions":[{},{},{}],"displayName":"Allocate node : Start","iconColor":"blue","id":"23","parents":["13"],"running":false,"url":"job/test/1/execution/node/23/"},
{"_class":"org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode","actions":[{},{}],"displayName":"Allocate node : Body : Start","iconColor":"blue","id":"33","parents":["23"],"running":false,"url":"job/test/1/execution/node/33/"},
{"_class":"org.jenkinsci.plugins.workflow.cps.nodes.StepAtomNode","actions":[{},{}],"displayName":"Print Message","iconColor":"blue","id":"37","parents":["33"],"running":false,"url":"job/test/1/execution/node/37/"}

So for your case you are interested in the child with type StepAtomNode of your Branch with name given (0-9 for this case). From this you could obtain console output address by simply adding log to the address (like: job/test/1/execution/node/37/log).

Now this is where it gets a bit ugly, you need to parse the html to get the actual log from the

<pre class="console-output">log here
</pre>

Answer 2

I needed to access logs from within the pipeline code
so I implemented the algorithm proposed by キキジキ (really helpful) with a few adjustments (to add branchName prefix on each line to be able to get the whole log and still figure out which branch corresponds to each line ; and to support nested branches, which I needed) in https://github.com/gdemengin/pipeline-logparser :

• to get logs programmatically

  • to get the full logs with branch prefix (similar to what currentBuild.rawBuild.log returned before version 2.2.5 of workflow-job plugin. but in version 2.26 they got rid of the branch information and I could not find any built-in function with the same information)
    String logs = logparser.getLogsWithBranchInfo()

    [Pipeline] Start of Pipeline
    [Pipeline] parallel
    [Pipeline] { (Branch: branch1)
    [Pipeline] { (Branch: branch2)
    [Pipeline] }
    [Pipeline] echo
    [branch1] in branch1
    [Pipeline] sleep
    [branch1] Sleeping for 1 sec
    [Pipeline] echo
    [branch2] in branch2
    [Pipeline] sleep
    [branch2] Sleeping for 1 sec

  • get the logs from 'branch2' only
    String logsBranch2 = logparser.getLogsWithBranchInfo(filter: ['branch2'])

    [branch2] in branch2
    [branch2] Sleeping for 1 sec

• to archive logs (in as $JOB_URL/<runId>/artifacts) to have them available as a link for later use

  • to archive the full logs (with branch prefix)
    logparser.archiveLogsWithBranchInfo('consoleText.txt')

  • to archive the logs from branch2 only
    logparser.archiveLogsWithBranchInfo('logsBranch2.txt', [filter: ['branch2']])

Answer 3

I found a way to achieve that, but you need to access the build folder directly (for example using currentBuild.rawBuild.getLogFile().getParent()).

• Parse the xml files (or the single flowNodeStore.xml file) inside the workflow directory:
  • Build a hierarchy of nodes using the <id> and <parentIds> values.
  • If <branchName> is defined associate it to the current node and recursively to all nodes that have this node as parent. If a node has multiple parents assign no branch value to it.
• Read the log file as byte[].
• Read each line of log-index to find log ranges to assign to each node. The format of a line can be one of the following:
  • offset nodeId -> start of new node range, end of the previous (if present).
  • offset: end of current node range.
• Convert the byte range back to a utf8 string (new String(range, "UTF-8")).
  • You might want to strip away all embedded codes with something like replaceAll("\u001B.*?\u001B\\[0m", "")