Jenkins parallel pipeline

I was very excited to know about the ‘parallel’ feature in Jenkins Pipeline, but, there are many gotchas while makingĀ  use of the pipeline feature (many of which are documented here: Jenkins Pipeline Example). After trying and reading a few different solutions, following worked for me (notice in screenshot that the browser jobs run in parallel !)

pipeline {
  agent any
  stages {
    stage('Build') {
      steps {
        echo 'hello'
      }
    }
    stage('Test') {
      steps {
        script {
            def jobs = [:]
            def browsers = ["Chrome", "Firefox"]
            for (int i = 0; i < browsers.size(); i++) {
                def browser = browsers.get(i)
                def jobName = "printing $browser"
                jobs[jobName] = doJob(browser)
            }
            parallel jobs
        }
      }
    }
    stage('Deploy') {
      steps {
        echo 'deploying'
      }
    }
  }
}

def doJob(browser) {
    return {
        node {
            echo "testing in $browser"
        }
    }
}

casperjs output to html

Documenting what I had to do.

Used XSLT from here:
nosetest xslt

Problems and Fixes:

  1. Firefox was inserting “transformiix” as the root element, this caused the DOCTYPE to be spit out. I fixed by adding:
    doctype-public="-//W3C//DTD HTML 4.0//EN"/
    to 
    <xsl:output

    And removed the doctype declartion in the above xslt

  2. My version of casperjs was setting xml namespace to

    xmlns="http://www.w3.org/1999/xhtml

    After unsuccessfully trying to match the namespace in the xslt, I gave up and removed the namespace from the xunit xml, by

    sed -i 's@ xmlns="http://www.w3.org/1999/xhtml" @ @' "output.xml"
  3. Inserted the style into xml so can be rendered in the browser

    sed -i 's@ encoding="UTF-8"?>@ encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="nosetests.xslt" ?>@' "output.xml"
  4. Removed timestamp info from the xslt as it was in UTC timezone and could be very confusing when looking at the results (attempts to convert to my TZ in client were unsuccessful)

Checking for the existence of domains from commandline in Bash

while read line; do wget --quiet --spider --timeout=10 $line; if [ $? -ne 0 ]; then echo $line; fi; done < "/tmp/domains.txt" 
 
/tmp/domains.txt would hold the list of domain names separated by newline.
Note this is just a quick hack and not entirely reliable.

PHP as a replacement for sed/awk

I had gotten sick of not being able to do what I wanted to do with sed/awk – partly because I am not too familiar with those – and, was investigating replacements for those. Many people seem to be using perl and since I wasn’t too familiar with Perl as well, I started thinking about using PHP – which I do know. And, that served my needs brilliantly.

This is a simplistic version of my problem & a solution:
I can do the following to get the values for specific params (cid and oid in this case) for requests made, from my access logs:

k7@local:echo "http://example.com?cid=123&oid=435" | php -R 'preg_match("#.*cid=([^&]+)&oid=([^&]+)#",$argn, $matches);if(!empty($matches)) {echo $matches[1], " ", $matches[2], "n";}'
123 435

Regular Expression to check for comma separated list of Integers

//get rid of all kinds of spaces
$IDs = preg_replace('/s+/', '', $inputIDs);
if(!preg_match(''/^\d+$|^(\d+,)+\d+$/'', $IDs)) {
echo "Invalid Input";
}

You could use the validated IDs above in a SQL like the this:

SELECT * FROM t1 WHERE id IN($IDs)

Disclaimer: I am not going to be responsible for any kind of SQL injection resulting from using the snippet above.
updated 20-Mar-2013