Jenkins, parameterized builds and the ability to s

Question

(I've seen some suggestions here and elsewhere involving the NodeLabel plugin to solve similar problems, but have yet to find a solid example of how to actually USE NodeLabel for this purpose.)

Right now a gerrit trigger starts a build based on a given branch in the git repo of one of my projects.

This creates ONE parent job P that uses the Parameterized Trigger plugin to spawn two more project builds (identified by two labels) on two slaves with two different OSes (X1 and Y1).

Tomorrow, I have a new version of Y1 to build for called Y2, and X1 is deprecated. I can identify the valid targets for this version of my code within the code's git repo.

At that point I have a parent (P) and three node types (X1, Y1 Y2) to deal with. Because of branching I also must support the ability to build both legacy code (for X1 and Y1) OR new code (Y2 only).

I could create yet another job Y2 (which is identical to Y1 except for the node type it restricts the build to using a static label), and then during the execution of each child job I could decide whether to abort the job early (deprecated target) or perform that build. However, every time I build I now get four git clones adding redundant load and traffic to the git server: one for the parent and one for each target, including the two I could have determined to be obsolete during the first "Execute shell" step!

This might not sound like much, but when we get to OS Y9 we could have up to ten jobs being run for a given branch, eight of which are unneeded... this is not tenable. Though eventually older OS targets would be pruned, several versions need to be available at any given time even if only one is required for the given build.

Today, the first build step of my build on Jenkins for any target uses "Execute shell" to gather useful information to pass to the child jobs.

The child jobs are, however, hard-coded in the "Trigger/call builds on other projects --> Projects to build" field (along with other important settings such as how and when they block!) If instead I could put a variable in there (or somehow refer to a variable I set programmatically), I could simply set that variable in the shell step before it, and then it'd be populated with exactly the sub-projects it requires! The subprojects "know" what nodes/groups of nodes they are allowed to execute on.

In short, my project executes a script, that script sets a variable, that variable defines the sub-projects to call, much as if I edited the config for this build (e.g., MY_PROJ_TO_BUILD="X1 Y1" populates the "Projects to build" field). The next step continues by building those sub-projects in the usual way. That's all I need it to do and certainly without losing any current functionality and without directly hacking Jenkins. (Since these are triggering off of gerrit, having multiple versions of gerrit and/or Jenkins running isn't a viable option either.)

NodeLabel has been mentioned but I've yet to see an example of how to use it effectively for this kind of problem (the examples I've seen are extremely vague). So, if you have an example that fits the bill or a pointer to one I'd be glad to see it. Likewise, if you have a different idea/solution I'd be glad to hear it. I can imagine some reasons why what I wish worked (variable sub in the "projects to build" field) doesn't and perhaps cannot, but surely running making lots of unnecessary clones just to determine a small subset of targets are required for a given branch's build isn't optimal either.

(Note: this is for one project/repo... there are dozens I'm dealing with actually, and I have to say creating several identical-but-for-the-node-type-they-run-on subprojects for each one is itself a farce... but it's the only way I've found up to now that ensures backward build compatibility when several targets may need to be build for a given branch build.)

Answer 1

Just one idea, I'm not sure if it would work/help in your situation...

Trigger the child jobs via URL access rather than through Jenkins. The project script can access the URL(s) for the child job(s) to kick off.