Installation of Jenkins – and similar tools – used to be a tedious job. And once done, the lifecycle management was only done when absolutely necessary. When first Docker and later Kubernetes were introduced, everything got better and easier.

Initial installation using helm made it even better. Now you have a running instance of Jenkins in minutes.

❯ helm dependency update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "ingress-nginx" chart repository
...Successfully got an update from the "jenkins" chart repository
...Successfully got an update from the "stable" chart repository
...Successfully got an update from the "nexus-jenkins" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 1 charts
Downloading jenkins from repo https://********************/repository/jenkins-community/
Deleting outdated charts
❯ helm install -n devns-mb jenkinsci-jenkins17 .
NAME: jenkinsci-jenkins17
LAST DEPLOYED: Tue Nov 29 20:36:29 2022
NAMESPACE: devns-mb
STATUS: deployed
REVISION: 1
❯ kubectl get pods
NAME                                   READY   STATUS     RESTARTS   AGE
jenkinsci-jenkins17-jenkins17-main-0   0/2     Init:0/1   0          7s

But (!), when you think you are done, you are partly right. You are still missing vital tooling.

Additional plugins

If you have a backup strategy, this can be a good starting point. Installing plugins and upgrading can be done directly from the Jenskins gui. You are notified from the dashboard, urging to action.

For core updates as well as installation and updating of plugins the gui comes in handy. There is a caveat though. Nothing is versioned, only the helm chart. The rest is done manually and if lucky, someone has documented all the used plugins, tool setting, etcetera.

This is where Jenkins CasC – Configuration as Code – comes in. This plugin is implemented in the Jenkinsci helm chart by default, so you only have to start using it.

One example is explained in the paragraph below.

Pipeline Libraries

When using Kubernetes, setting up the nodes and pod templates in Jenkins is quite challenging. The basic setup is in the cloud configuration. Filling many settings by hand. You will end up with a raw yaml for the pod, which you have to copy paste into git, if you want to have the setup versioned.

Of course, it would be much easier to have this pod configuration in a git repository. This can be established by using the Global shared library functionality. To use an external library to recycle code. This library has to be setup in the ‘Configure System’ section of the Jenkins gui. Here you put the name and branch to be used.

If the ‘Load implicitly’ checkbox is marked, it will be downloaded before every Jenkins job run. If not, you must include it in every pipeline explicitly.

As mentioned, this library setup can be done from the JCasC setup in the helm chart. It is setup in the values.yaml with an ‘unclassified’ header. JCasC is located under the controller section of the helm chart.

You find the result in the Docker pod as a separate configuration file.

When you have setup the external library, put the pod template code in the ‘/resources/podTemplates’ directory of this git repository. Now you can start using it, by implementing it in the Jenkins file of your project.

As the pod template is stored in git, it is now easier to use multiple specific templates for specific circumstances.

Credentials loophole

You may have noticed the credentials ‘jenkins_to_gitlab’ in the JCasC code sample above. These credentials are not hardcoded in the values file of the helm chart, but stored in a Kubernetes secret.

Hardcoding the username and password is a bad practice, so even if you try it, it will not work. Thanks to the way Jenkins is installed and hashed each time a ‘helm install’ is performed. The outcome is different every time.

So, every time you do an export of the JCasC setup – via the manage/configuration-as-code/viewExport endpoint – a different value is gained. You can manually fix it by changing the password by hand in the Jenkins gui. Not very user friendly.

To create a solid fix, we can use a Kubernetes (sealed) secret. This is achieved in two steps.

First, the reference to a secret is stored in the JCasC part of the helm chart’s value file.


Then a templates directory is created in the helm chart directory.

The template directory contains the yaml file with the secret:

When installing the helm chart, the secret is automatically installed in the cluster. The secret is now ready for use.

Conclusion

As you can see, al lot of progress is made in making tools better prepared to keep track of updates and changes. These changes are now easily stored in a version control system. All this to help make your system more resilient and better to maintain.

Enjoy!