Blog

The latest expert opinions, articles, and guides for the Java professional.

Using Jenkins Build Pipelines to Chain and Visualize Jobs

Jenkins is a popular server-based automation tool that is used to implement continuous integration pipelines. Jenkins Build Pipelines can be further configured for continuous delivery by using Pipeline plugins in both declarative and scripted DSLs.

Continuous Delivery and Job Chaining

Job chaining in Jenkins is the process of automatically starting other job(s) after the execution of a job. This approach lets you build multi-step Jenkins build pipelines or trigger the rebuild of a project if one of its dependencies is updated. These chained jobs form the core of Jenkins continuous delivery pipelines.

In this article, we will look at a couple of plugins for Jenkins job chaining and see how to use them to build and visualize these Jenkins pipelines.

Out of the Box Solution – Using Jenkins Build Other Projects

Jenkins has a built-in feature to build other projects. It is in the Post-build Actions section. You can specify the projects that you want to build after this project is built (you can trigger more than one). So whenever project A is built you will trigger the building of project B. You can also specify the conditions when the other jobs are built. Most often you are interested in continuing with the build pipeline only if the job is successful but your mileage might vary.

Jenkins Build Pipeline: Build other projects trigger

One thing to remember here is that Jenkins Build Triggers have two configurable locations. You can configure project A and specify a post action as in the previous screenshot. Another option is to configure this from project B and say “build this project B only after project A is built”. You don’t have to fill out both, just change one and the other is updated. See the next screenshot for the second option.

Jenkins Build Triggers: Build after others solution

Jenkins Build Pipeline Plugin

Update : As of 9/30/2019 The Jenkins Build Pipeline plugin currently has an unaddressed stored XSS vulnerability. It is still used, as evident by the 40,000 plus downloads in August 2019.

Jenkins Build Pipeline Plugin is one interesting plugin. The main features of this plugin is to provide visualization of the build pipeline and also to provide manual trigger for continuous delivery purposes. The configuration is a separate Post Build action where you can configure which projects should be built after project A. By default the triggering is actually done manually by the end user! If you want certain steps of the Jenkins pipeline to be automatic then you have to use the built-in job chaining (see the Out of the Box Solution for more details).

Jenkins Build Pipeline Configuration example

The Jenkins Build Pipeline plugin offers a very good visualization of the build pipeline. By configuring a new Jenkins view and choosing which job is the first job in the Jenkins pipeline you can get a visualization of the whole build pipeline. In the screenshot, be sure to note that one of those steps is manual and the result are automatic. The manual one can be triggered from the very same view.

Jenkins Build Pipeline example

Jenkins Parameterized Trigger Plugin

Update: The last major release for this plugin was in December 2013. The plugin had almost 90,000 reported downloads in August 2019.

The Jenkins Parameterized Trigger Plugin is another triggering plugin but with a twist: this plugin lets you configure more aspects of the Jenkins triggering logic. It covers the basic Out of the Box Solution features and adds many more. The most important one is the option to trigger the next build with Jenkins pipeline parameters. For example, by defining SOURCE_BUILD_NUMBER=${BUILD_NUMBER} you are able to use the variable $SOURCE_BUILD_NUMBER in project B. This way you can, for example, use the artifact built in the previous job to be fetched from your central artifact repository using the ${BUILD_NUMBER}.

Jenkins Trigger Parameterized Build



Downstream Buildview Plugin

The Downstream Buildview Plugin plugin that does not do job chaining itself, but provides a means to visualize the Jenkins build pipeline. It is similar to the Build Pipeline View but more dynamic. You can click on any item in the build history and have its build pipeline visualized.

Jenkins Downstream Build Pipeline Visualization

The Jenkins Pipeline

Jenkins Pipeline, or Pipeline, is an overarching collection of plugins that support and enable implementation of continuous delivery pipelines. Jenkins build pipelines are defined via declarative or scripted pipelines and can be created in Vanilla Jenkins, Blue Ocean or SCM.

This plugin “suite” is considered to be a more complete and dependable way to implement continuous delivery with Jenkins (which previously has been viewed as more of a duct-tape solution).

Pipeline has a long list of dependent plugins that can be used to further increase functionality.

Conclusion

The main feature that makes Jenkins so good is that there is always an app plugin for what you need. Of course, the same fact also highlights its biggest weakness. It is rather difficult to choose the correct plugin and very often you need a couple of plugins to achieve your goal. The same is true for job chaining and visualization.

The job chaining features that we covered in this post all provide the minimum functionality – triggering other jobs. The Parameterized Trigger plugin is the most feature-rich, but lacks the manual triggering. The Jenkins Build Pipeline plugin only offers manual triggering and you need to figure out automatic triggering yourself (using the built-in feature for example).

From the visualization side, the Jenkins Build Pipeline plugin is definitely the best looking. At the same time, the plugin does not support passing parameters (the latest alpha build is a bit better) and once the build pipeline gets long it gets a bit ugly. We do like the part of defining a separate view and then always being on top of your build pipeline. The Downstream Build View plugin gives you great flexibility and insight to job chaining, but does not enforce any kind of process.

So, there are the Jenkins plugins that we use at JRebel for job chaining and visualization. Do you use the same tools? If not, can you recommend any others? Which are your favorites? Please leave comments below!

Responses (18)

  1. Avatar  

    Billy Foss

    May 9, 2012 @ 3:01 pm

    We use the pipeline view to show the full cycle from branching to build to deploying on test systems.  It gives a nice summary view of a full branch/build process.  I will be trying the Downstream Buildview Plugin to get a more dynamic view of other jobs that chain.
    Some may want to consider using the Promote step to allow review of build results that can be approved and then trigger the following steps in a chain.  For example, the Promote can allow certain users to approve a build, which can trigger copying that build to a QA server for further testing.

  2. Avatar  

    Toomas Römer

    May 10, 2012 @ 7:24 am

    I did not know about the promote plugin but will check it out. Sounds like a great addition.

  3. Avatar  

    Grégory Boissinot

    May 10, 2012 @ 7:56 pm

    A build chain in Jenkins can also be achieve with the XTrigger Jenkins plugin (https://wiki.jenkins-ci.org/display/JENKINS/XTrigger+Plugin)

  4. Avatar  

    Java Developer

    May 18, 2012 @ 3:01 pm

    The pipeline plugin looks really interesting. We are going to checkout that.  Thanks for the article Toomas,

  5. Avatar  

    Toomas Römer

    May 21, 2012 @ 8:14 am

    The comments mangled your URL a little (404 because of the extra bracket). Here is a shortened link http://bit.ly/JJlhqO I guess now I know 4 ways for chaining :)

  6. Avatar  

    irfan

    November 7, 2012 @ 9:58 am

    i have around 10 jobs in my view and i want to run the remaining 9 jobs after the build of first job. Instead of mentioning the names of the remaining 9 jobs as comma separated is there any other way to give lije some * function or some thing.

  7. Avatar  

    Toomas Römer

    November 7, 2012 @ 10:09 am

    I think not.

  8. Avatar  

    irfan

    November 8, 2012 @ 6:53 am

    Thanks for the reply Toomas. There is one more thing i would like to know, the email notification thing only works under the negative scenarios and a successful build after a fail build so is there any way to get the e-mail notification for success scenarios..

  9. Avatar  

    Toomas Römer

    November 11, 2012 @ 7:23 pm

    I think you cannot do it with vanilla installation but check out the Email-ext plugin, https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin

  10. Avatar  

    Robson Peixoto

    January 29, 2014 @ 3:23 pm

    Hi. How can I use Build Pipeline Plugin with parameters? I’d like to use the same BUILD_NUMBER on job trigged by Build Pipeline Plugin

  11. Avatar  

    Satish

    June 24, 2014 @ 6:25 am

    Hi team,
    while building the application it is taking more time, so I want to stop the running build instance, how to do that. Can you please help me on this.

  12. Avatar  

    Gorovdude

    October 21, 2014 @ 6:39 am

    Another nice plugin is the MultiJob Plugin https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin

  13. Avatar  

    jack

    November 4, 2014 @ 4:24 pm

    Which one is the best for you ?

  14. Avatar  

    Greg

    September 15, 2015 @ 3:17 am

    The pipeline plugin was amazing, until someone wanted to save a little money. Imagine:

    build-pipeline =

    execute step 1
    execute step 2, parameters x, y, z
    execute step 3a
    execute step 4
    execute step 5

    Now someone wants:

    execute step 1
    execute step 2, parameters x, y’, z”
    execute step 3b
    execute step 4

    Good luck with that; I learned the hard way this weekend. If you’re going to chain together jobs, SKIP the point-and-click and install the workflow: aggregator plugin, instead, then learn enough groovy to do what you need to do. I just built pre-defined jobs, and ripped out the post-build actions / conditional steps on all of my jobs. If you examine each job’s xml file, I still have a lot of code duplication, but I’ll rip that out eventually, replacing it with the job dsl plugin (NOT the workflow dsl).

  15. Avatar  

    Ace Han

    April 20, 2016 @ 4:05 am

    Hello there, there is another interesting plugin https://github.com/jenkinsci/coordinator-plugin

    Which is no code required to define jobs running sequence.Much easier to get the whole status of one build

    And I found a walk through demo here
    http://www.tothenew.com/blog/jenkins-coordinator-plugin/

    Hope you will enjoy it

  16. Avatar  

    Altan Cabal

    May 23, 2016 @ 8:40 pm

    Good article. Thank you!

  17. Avatar  

    Azim

    June 28, 2016 @ 10:49 am

    Hi,

    Is it possible to run a post node step just like a post build step. Example:

    node (“${label}”) {
    // build steps

    // post build steps. Should run irrespective of failure in build steps. Example if I want to update GitHub commit status here.
    }

    Thanks

  18. Avatar  

    Ganesh

    August 8, 2016 @ 7:52 am

    There is a completely another CI system which has been supporting these pipeline concept for quite some time. But, not much community support and cohesion is there for this but it is also nice CI tool. https://www.go.cd/

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.