A Jenkins Pipeline Tutorial That Times Every Stage

Every Jenkins pipeline tutorial shows the same twelve-line Jenkinsfile and stops at "and now it builds". None of them tells you what it cost. This one is the same twelve lines, run on a real Jenkins — jenkins/jenkins:lts-jdk21, which reports itself as 2.568.3 on JDK 21.0.12.1 — with the numbers read back out of Jenkins' own API afterwards.

Three of those numbers were worth the exercise: the plugin you install pulls in fifty-five others, the stage timings add up to more than the build took, and a single sh step costs about 280 milliseconds before it runs anything at all.

Getting in, and the password nobody warns you about

A fresh container reaches its login page in 7 seconds and then refuses to do anything useful, because Jenkins has generated a one-time password and put it somewhere you are not looking. It is in the container log, framed in asterisks:

Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:

2b590171dda74533a5432a7b8d9bdc29

This may also be found at: /var/jenkins_home/secrets/initialAdminPassword

The file is 33 bytes — thirty-two hex characters and a newline — and it is -rw-r-----, owned by jenkins. Both copies agreed, which is worth knowing because people reach for the file when they have lost the log:

docker exec <container> cat /var/jenkins_home/secrets/initialAdminPassword

Everything below skips this by starting Jenkins with -Djenkins.install.runSetupWizard=false, which is how you get an API to talk to. On a real instance you want the wizard, because it is also where the admin account gets created.

Two plugins asked for, fifty-six installed

"Pipeline" is not built in. In the plugin manager it is workflow-aggregator, and the official Docker image ships a CLI for installing plugins without a browser:

FROM jenkins/jenkins:lts-jdk21
RUN jenkins-plugin-cli --plugins workflow-aggregator:latest git:latest

Two names in, 56 .jpi files out, 46 MB of them. That fan-out is not a warning — it is what makes the pipeline DSL work — but it does explain why a Jenkins upgrade is never a small thing, and why the plugin page of an old instance reads like an archaeology site.

It also does not include what you probably expect. workflow-aggregator gives you the DSL and no stage view: the wfapi endpoint every timing below comes from returned 404 until pipeline-stage-view was installed separately. If your Jenkins shows the coloured stage grid, that is a different plugin doing it, and the wizard's suggested set is what usually brings it along.

The pipeline

Eight stages, four of them doing nothing but sleeping, because the point is the shape rather than the work:

pipeline {
  agent any
  stages {
    stage('Checkout') { steps { sh 'sleep 1; echo "pretend clone"' } }
    stage('Build')    { steps { sh 'sleep 3; echo "pretend build"' } }
    stage('Test in parallel') {
      parallel {
        stage('unit')        { steps { sh 'sleep 4; echo unit' } }
        stage('integration') { steps { sh 'sleep 6; echo integration' } }
        stage('lint')        { steps { sh 'sleep 2; echo lint' } }
      }
    }
    stage('Package') { steps { sh 'sleep 1; echo "pretend jar"' } }
  }
  post {
    always  { echo "post/always ran" }
    success { echo "post/success ran" }
    failure { echo "post/failure ran" }
  }
}

agent any is the line to understand first. It allocates an executor and a workspace — here on the controller's own built-in node, which is convenient and is also why nobody should run production builds this way. The syntax reference is precise about the ordering: an agent is allocated and then options like timeout apply, and "the time to allocate the agent is not included in the limit set by the timeout option" — so a build stuck waiting for a free executor is not a build that will time out.

What each stage cost

Jenkins keeps this and will hand it over: /job/<name>/<build>/wfapi/describe.

run #1: SUCCESS · queued 10 ms · total 14,225 ms
  Checkout                    SUCCESS     1,282 ms
  Build                       SUCCESS     3,570 ms
  Test in parallel            SUCCESS        41 ms
  unit                        SUCCESS     4,462 ms
  integration                 SUCCESS     6,811 ms
  lint                        SUCCESS     2,172 ms
  Package                     SUCCESS     1,224 ms
  Declarative: Post Actions   SUCCESS        34 ms

Read the arithmetic before the numbers. The stages add up to 19,596 ms and the build took 14,225 ms — the stages total more than the run that contains them. That is not an error, it is the proof that parallel did what it says: unit, integration and lint overlap, so their 13,445 ms of combined time only costs the 6,811 ms of the slowest one. The enclosing Test in parallel stage shows 41 ms because it is only the container that starts the three branches.

That also means you cannot add stage times to get a build time on any pipeline that uses parallel, which is a mistake worth not making in front of a dashboard.

The sleeps total 11 seconds; the build took 14.2. The missing 3.2 seconds is Jenkins itself, and the next section is where most of it went.

Every sh step costs about 280 ms

This is the measurement that changes how you write a Jenkinsfile. Two stages, identical work — twenty no-ops — expressed two ways:

stage('twenty trivial sh steps') {
  steps { script { for (int i = 0; i < 20; i++) { sh 'true' } } }
}
stage('one sh doing the same nothing twenty times') {
  steps { sh 'for i in $(seq 1 20); do true; done' }
}

Three runs, and it barely moved:

run 1:  20 separate sh steps  5,737 ms (287 ms each)   one sh  301 ms   19x
run 2:  20 separate sh steps  5,623 ms (281 ms each)   one sh  293 ms   19x
run 3:  20 separate sh steps  5,635 ms (282 ms each)   one sh  298 ms   19x

Nineteen times slower for exactly the same work. A sh step is not a shell command; it is a Jenkins step that writes a script file into the workspace, spawns a shell to run it, streams the output back, and records the step in the flow graph so the stage view can draw it. That bookkeeping is about 280 ms, every time, whether the command takes a millisecond or an hour.

Which gives a rule you can apply without measuring anything: group commands into one sh, and use separate steps only where you want them separately reported. A twenty-line build reduced to five sh blocks saves four seconds per run for free — and on a pipeline that runs two hundred times a day, that is not nothing.

Tip

The corollary matters more than the saving. If a stage's own overhead is 280 ms per step, then a stage's timing tells you about your pipeline's structure as much as about your build. Before optimising a slow stage, check how many steps it has.

The workspace is not a clean room

agent any gave the build a directory, and the same one every time. A pipeline that printed its location, listed what it found and left a marker behind, run three times:

build 1   workspace: /var/jenkins_home/workspace/ws
          left by a previous build: (nothing)
          markers now: 1
build 2   left by a previous build:  marker-1.txt
          markers now: 2
build 3   left by a previous build:  marker-1.txt  marker-2.txt
          markers now: 3

The workspace is reused, not recreated. Build 3 could read a file build 1 wrote, and on a real project that file is a stale jar, a node_modules from a branch you no longer have checked out, or a report the next build then publishes as its own.

Two things follow. A build that passes may be passing because of something left over, which is the hardest class of CI bug to see — it reproduces on the server and never on a laptop. And a git clone in your pipeline is not doing what you think if the directory already has a clone in it; the checkout scm step handles that properly, a hand-written clone often does not.

The fix is one step, in the place that always runs:

post { always { cleanWs() } }

Except that one does not work here, and the other does. cleanWs() needs the Workspace Cleanup plugin, which is one more that does not arrive with workflow-aggregator:

java.lang.NoSuchMethodError: No such DSL method 'cleanWs' found among steps
[archive, bat, build, catchError, checkout, deleteDir, dir, echo, error,
 fileExists, git, input, isUnix, library, load, mail, milestone, node,
 parallel, powershell, properties, pwd, readFile, retry, script, sh, sleep,
 stage, stash, step, timeout, tool, unstash, withCredentials, withEnv, ...]

deleteDir() is in that list, comes from workflow-basic-steps, and did the job — before: 1 files, after deleteDir: 0 files.

That error is also the best step reference in Jenkins. Misspell any step and the failure prints every step this instance actually has, which beats searching a plugin's documentation for whether a method exists on your version.

When a stage fails

The interesting part of a pipeline is what it does when something breaks. Stage Two exits 3:

failjob: FAILED · total 2,005 ms
  One                         SUCCESS      389 ms
  Two                         FAILED       366 ms
  Three                       FAILED        44 ms
  Declarative: Post Actions   SUCCESS       34 ms

Three things in that, and one of them is a trap.

The remaining stages do not run. The console is explicit — Stage "Three" skipped due to earlier failure(s) — and the 44 ms is Jenkins deciding that, not the stage doing anything.

But the stage view calls it FAILED. Three never executed and is reported with the same status as Two, which actually failed. On a long pipeline that is genuinely misleading: a wall of red stages usually means one real failure and a row of skipped ones, and only the console log tells you which is which.

The post section still runs. post/always and post/failure both executed, post/success did not, and the build ended:

POST-ALWAYS
POST-FAILURE
ERROR: script returned exit code 3
Finished: FAILURE

The docs describe always as running "regardless of the completion status of the Pipeline's or stage's run", and that is exactly what it is for: cleanup, artefact archiving and notifications belong there rather than in a last stage, because a last stage is the thing that does not run.

What to write, given all that

The pipeline worth copying out of this is short:

• One agent declaration at the top, and a real agent rather than the controller once you are past learning.
• Stages named after what a human wants to see on a dashboard, not after every command.
• The file committed to the repository it builds, which is the whole argument of Using a Jenkinsfile — the inline script above is a measuring convenience, not a pattern.
• Commands grouped into as few sh steps as the reporting allows — the 280 ms is per step, not per line.
• parallel for anything independent, and the knowledge that your stage times will then stop adding up.
• Cleanup in post { always }, never in a final stage.

If you are comparing this with the hosted alternative, the same pipeline shape on GitHub Actions is measured the same way and the per-step overhead there is different in kind, because a step is a process rather than a Jenkins object. If the thing being built is a container, what each Dockerfile choice costs is the other half of the same build. And if the pipeline's job is to push to a cluster, Docker against Kubernetes is where that decision lives.

[!TAKEAWAY] Ask Jenkins for its own numbers — wfapi/describe on any build — before you believe anything about your pipeline's speed. It reported 14,225 ms for a build whose stages claim 19,596 ms, which is how parallel work looks from the outside, and 281 ms of overhead for a step that ran true, which is how a twenty-command stage becomes six seconds of nothing.