By David Fiser
Jenkins is a popular open-source automation server for software development teams. Used for managing the development side in DevOps, the main purpose of Jenkins is to perform tasks, called jobs, such that software project builds are automatically developed in the CI/CD process.
Jenkins has a distributed architecture: A master machine manages a group of agents (aka slaves), which are Java executables running on remote machines and that execute build jobs. Jenkins is also based on a modular architecture, and most of its features are implemented inside plugins that extend its core functionality, for example, post-build tasks.
The convenience offered by Jenkins also covers security aspects through its matrix-based model, not to mention how it’s easier to just pull an official Jenkins image from Docker Hub. However, working under default settings and enabling Jenkins’ matrix-based security might lead developers to assume that it’s already a secure setup. Unfortunately, we discovered that this can lead to potential security problems.
In our analysis, we observed that a user account with less privilege can gain administrator rights over the automation server if jobs are built on the master machine (i.e., the main Jenkins server), a setup enabled by default. An exploit for this can be easily written using shell spawn — a default build step. If an exploit is successfully deployed, an attacker can perform remote code execution (RCE) on the master, which can result to Jenkins being completely overwritten.
An unsecure or lax configuration of these settings makes servers susceptible to attacks that abuse the said design feature, and we have already disclosed our findings to the Jenkins project. They responded that what we reported is not a security vulnerability, referencing their user handbook and wiki.
A look into Jenkins’ default security settings
According to Jenkins’ security page, many security options in version 2.0 were enabled by default to ensure that Jenkins environments remain secure, unless certain protections are explicitly disabled by the administrator.
Security can be configured using the Configure Global Security page. By default, Security Realm (authentication) protects Jenkins’ own user database. It’s worth noting that the Master stores the user database and under that setting, logged-in users have access to everything.
Figure 1. Jenkins’ matrix-based security model
Under Authorization, options such as matrix-based security and project-based matrix authorization strategy can be enabled to allow the administrator to limit access to certain Jenkins features.
Executing shell spawn on the master
A loosely configured setting is at risk of being taken advantage of by a malicious actor. Before illustrating how that can happen, first, let’s look into the steps that take place in performing a job. The following is basically an analogy of the CI/CD pipeline:
- Triggering the build
- Source Code Management (SCM) – polling the source code
- Setting the build environment
- Performing build steps (configuring compiler, etc.)
- Post-build actions (running tests, deployment, etc.)
The scripting capabilities of build steps can be extended with plugins installed within Jenkins. One of the default features within build steps, called Execute shell, is what we will focus on.
As opposed to implementing every single possible feature of the build script within the web application, scripting a build process – a common practice – can be done to speed up the process. However, executing shell spawn inside the web application poses security risks, especially if user permissions aren’t properly assigned.
Allowing Jenkins users to have unnecessary permissions is concerning as it can potentially lead to security being compromised. For instance, a plugin called Script Security helps address that risk since it can limit the execution of untrusted Groovy scripts using a whitelist-based sandbox. Groovy script can only be executed with admin approval or if it’s inside a sandbox. The purpose of using a sandbox is to avoid situations where the user is able to interfere with the Jenkins instance and call internal functions, e.g., Jenkins.getInstance().
In contrast, the Execute shell feature is not restricted and doesn’t need to be approved by the admin, allowing users with less privilege to spawn a shell and execute their scripts. This poses security risks if jobs are executed on the master, which, as mentioned, is set by default. The master stores Jenkins configurations, application binaries, job definitions, and user database. Since jobs are executed under the same user as the Jenkins application and the job working directory is inside JENKINS_HOME, configuration file access using a relative file path will be permitted, allowing Jenkins files to be read, written, and executed.
Possible attack scenarios
To illustrate how relying on Jenkins’ default settings can pose security risks, let’s look at the following scenario. Jenkins is installed under a setting that has suggested plugins, Matrix-based security model, and no configuration for the agent. In this setup, a user with less privilege is logged in with the following permissions:
Figure 2. User permissions that can allow exploit code execution on the master
With the above user permissions, an attacker can create and/or modify a malicious job by spawning a shell script. In this setup, the availability of tools the attacker can use is limited to the platform. If Jenkins is running on the Linux platform, e.g., an official Jenkins Docker container, where the environment includes binaries like echo, sed, Python, and wget, among others, an attacker can have a variety of options to conduct RCE. The following actions can be carried out:
- The attacker can achieve full control of Jenkins.
- The attacker can leak user password hashes, Jenkins configuration files, job configuration files, secrets or badly stored credentials.
- The attacker can change user passwords.
- The attacker can kill the Jenkins instance.
- The attacker can replace Jenkins binaries with his/her own.
Proof of Concept (PoC)
An attacker could use sed and execute the following shell command:
sed -i ‘s#<useSecurity>true</useSecurity>#<useSecurity>false</useSecurity>#g’ ../../config.xml3.
Figure 3. Exploiting shell access on the master
This will unmark the Enable security checkbox in the Configure Global Security page, providing all users admin access once Jenkins reloads its configuration files from the disk.
Figure 4. Upon the execution of the exploit code, Jenkins’ security will be disabled
Security recommendations and mitigation steps
Admins are fully responsible for the security and proper configuration of Jenkins. Generally, the principle of least privilege should be implemented; limiting access to the bare minimum permissions needed by users to perform tasks can reduce the risk of account abuse or hijacking. If services and software are not utilized by users, limiting, replacing, or even completely disabling them should be considered.
The risks involved in relying on default security settings are not limited to the abuse of shell but also include the execution of malicious code on the master. So we recommend, more specifically, the execution of jobs on the master node to be disabled. In comparison, using shell spawn to maliciously manipulate configurations on slave environments, e.g., containers, will not transpire since they are isolated from the master. The Jenkins project, for their part, has already issued a warning and a recommendation related to this authorization issue. To limit job execution permissions on the master, they advised admins to use the Authorize Project plugin.
To further prevent potential abuse, shell, among other plugins, should be disabled if it is not necessary for a user. The shell execution can be limited by setting Shell executable to /bin/false at the Configure System page. This way, the /bin/false binary will be executed, but an argument passed to the shell script will make the app exit and discard any input.
Figure 5. How to disable Shell executable
Organizations can take advantage of Trend Micro DevOps security solutions, which help in baking security into development processes via APIs to improve development cycles and reduce human touch points and errors. Such security solutions can also reduce disruption of development schedules and workflows with protection for images, containers, and hosts by quickly closing the security feedback loop.