Skip to content

Feature/136 trivy integration #139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 61 commits into from
Jan 7, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
c2897fb
Add Trivy implementation structure; #136
robertauer Nov 18, 2024
251cdf1
Add invalid image name test; #136
robertauer Nov 18, 2024
ec14c12
Fail pipeline on failing unit tests
robertauer Nov 18, 2024
8cc5d40
Implement scanImage method; #136
robertauer Nov 19, 2024
cc9b3e6
Handle null references; #136
robertauer Nov 20, 2024
5a97314
Move parameters to sensible locations; #136
robertauer Nov 20, 2024
0341304
try to fix pipeline error
alexander-dammeier Nov 20, 2024
6d6dbb7
Fix parameter call
robertauer Nov 21, 2024
d5b41b1
#136 fix groovy compile error with constant
alexander-dammeier Nov 21, 2024
77ad25b
Fix NullPointerException
robertauer Nov 21, 2024
c30624e
Use custom exit code on found vulnerabilities; 136
robertauer Nov 21, 2024
a65c6e2
Implement report conversion function; #136
robertauer Nov 21, 2024
a28ad47
Save all trivy report files; #136
robertauer Nov 21, 2024
096a0aa
Add HTML format conversion; #136
robertauer Nov 21, 2024
c933342
Fail on unsupported scan format; #136
robertauer Nov 25, 2024
7777d4d
Trivy ignore file is working
robertauer Nov 25, 2024
358b7e2
Enable report file name setting; #136
robertauer Nov 25, 2024
190fb08
Avoid rate limits of default Trivy database source; #136
robertauer Nov 25, 2024
7bd4562
Set build status according to strategy; #136
robertauer Nov 25, 2024
4183ba2
Document Trivy functionality; #136
robertauer Nov 25, 2024
5bf7fdf
Adapt tests to new Trivy constructor; #136
robertauer Nov 26, 2024
a32591f
Add method to scan Dogu images; #136
robertauer Nov 27, 2024
122c87d
Remove newlines from ssh output; #136
robertauer Nov 27, 2024
326db5c
#136 implement trivy integration test
robertauer Nov 27, 2024
52f5ddf
Merge branch 'develop' into feature/136_trivy_integration
alexander-dammeier Nov 27, 2024
91bf4ca
#136 add comments to integration test
alexander-dammeier Nov 27, 2024
e6736d9
#136 use javac compiler as default
alexander-dammeier Nov 27, 2024
f651b32
Adapt old trivy function to new implementation; #136
robertauer Nov 28, 2024
5e5703e
Revert "#136 use javac compiler as default"
alexander-dammeier Nov 28, 2024
dc31bd4
Update changelog; #136
robertauer Nov 28, 2024
6cbd37d
Merge branch 'feature/136_trivy_integration' of github.com:cloudogu/c…
robertauer Nov 28, 2024
a4651bd
#136 compile to java8
alexander-dammeier Nov 28, 2024
57db423
Merge branch 'feature/136_trivy_integration' into feature/136_fix_gro…
alexander-dammeier Nov 28, 2024
37ce2f9
#136 add tests for trivy scans
alexander-dammeier Nov 28, 2024
94ef45f
#136 refactor tests
alexander-dammeier Nov 28, 2024
a8cbc91
Update src/com/cloudogu/ces/cesbuildlib/TrivySeverityLevel.groovy
robertauer Dec 10, 2024
e1d0334
Clarify additional flags defaults; #136
robertauer Dec 10, 2024
a7feef0
Merge branch 'feature/136_trivy_integration' of github.com:cloudogu/c…
robertauer Dec 10, 2024
c8484a0
Do not potentially overwrite the original trivy report file; #136
robertauer Dec 10, 2024
12c2a30
Use class variable instead of hardcoding the directory; #136
robertauer Dec 10, 2024
69b9209
Fix path; #136
robertauer Dec 10, 2024
cd604e8
Remove outdated comments; #136
robertauer Dec 10, 2024
174645f
Make trivy image source adjustable; #136
robertauer Dec 10, 2024
15fe1a9
Enable custom formats for Trivy report conversion; #136
robertauer Dec 11, 2024
aecb7fd
Do not overwrite first report file on second conversion; #136
robertauer Dec 11, 2024
d2cb3a7
Remove faulty code
robertauer Dec 19, 2024
e4ea857
Let the user decide which file extension to use; #136
robertauer Dec 19, 2024
790c557
Select specific severity levels in formatted report; #136
robertauer Dec 19, 2024
2bf5665
Remove unused variable assignment; #136
robertauer Dec 19, 2024
60eb8f1
#136 - refactor saveFormattedTrivyReport
meiserloh Dec 20, 2024
5baa5a1
Merge remote-tracking branch 'origin/develop' into feature/136_trivy_…
meiserloh Dec 20, 2024
59946ee
#136 - change Test to Junit5
meiserloh Dec 20, 2024
bcb0de2
#136 - add tests for saveFormattedTrivyReport
meiserloh Jan 2, 2025
640b2a6
#136 - remove constructor call from parameters to (hopefully) resolve…
meiserloh Jan 6, 2025
5b53198
#136 - Mark function as NonCPS, because of the use of Iterators
meiserloh Jan 6, 2025
095f4b7
Use duplicated strings to prevent VerifyError
robertauer Jan 6, 2025
cc742c0
#136 - Downgrade Junit & Mockito to be inline with the used JUnit ver…
meiserloh Jan 6, 2025
47d53c6
Merge remote-tracking branch 'origin/feature/136_trivy_integration' i…
meiserloh Jan 6, 2025
08fe966
#136 - apply review suggestions
meiserloh Jan 7, 2025
5af485c
Highlight technical text
robertauer Jan 7, 2025
a8f00bf
Describe how to retrieve the dogu image to scan
robertauer Jan 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@
/*.iml
/.idea/
.mvn/**
**/trivyReport.json
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0


## [Unreleased]
### Added
- Add Trivy class for scanning container images with Trivy
- Combines the functionality of the findVulnerabilitiesWithTrivy function and the Trivy class of the dogu-build-lib

### Deprecated
- findVulnerabilitiesWithTrivy function is deprecated now. Please use the new Trivy class.

### Changed
- [#140] Update Maven-Build-Dependencies
- JUnit 5
Expand Down
6 changes: 3 additions & 3 deletions Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ node('docker') {

def cesBuildLib = libraryFromLocalRepo().com.cloudogu.ces.cesbuildlib

def mvn = cesBuildLib.MavenWrapperInDocker.new(this, 'adoptopenjdk/openjdk11:jdk-11.0.10_9-alpine')
def mvn = cesBuildLib.MavenWrapperInDocker.new(this, 'eclipse-temurin:11.0.25_9-jdk-alpine')
mvn.useLocalRepoFromJenkins = true
def git = cesBuildLib.Git.new(this)

Expand Down Expand Up @@ -40,7 +40,7 @@ node('docker') {
}

stage('Unit Test') {
mvn 'test -Dmaven.test.failure.ignore=true'
mvn 'test'
// Archive Unit and integration test results, if any
junit allowEmptyResults: true, testResults: '**/target/failsafe-reports/TEST-*.xml,**/target/surefire-reports/TEST-*.xml'
}
Expand Down Expand Up @@ -70,4 +70,4 @@ def libraryFromLocalRepo() {
// Checks out to workspace local folder named like the identifier.
// We have to pass an identifier with version (which is ignored). Otherwise the build fails.
library(identifier: 'ces-build-lib@snapshot', retriever: legacySCM(scm))
}
}
207 changes: 177 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ Jenkins Pipeline Shared library, that contains additional features for Git, Mave
- [Markdown](#markdown)
- [DockerLint (Deprecated)](#dockerlint-deprecated)
- [ShellCheck](#shellcheck)
- [Trivy](#trivy)
- [Steps](#steps)
- [mailIfStatusChanged](#mailifstatuschanged)
- [isPullRequest](#ispullrequest)
Expand Down Expand Up @@ -1240,6 +1241,179 @@ shellCheck(fileList) // fileList="a.sh b.sh" execute shellcheck on a custom list

See [shellCheck](vars/shellCheck.groovy)

# Trivy

Scan container images for vulnerabilities with Trivy.

## Create a Trivy object

```groovy
Trivy trivy = new Trivy(this)
// With specific Trivy version
Trivy trivy = new Trivy(this, "0.57.1")
// With specific Trivy image
Trivy trivy = new Trivy(this, "0.57.1", "images.mycompany.test/trivy")
// With explicit Docker registry
Docker docker = new Docker(this)
docker.withRegistry("https://my.registry.invalid", myRegistryCredentialsID)
Trivy trivy = new Trivy(this, "0.57.1", "aquasec/trivy", docker)
```

## Scan image with Trivy

Scan an image with Trivy by calling the `scanImage` function.

```groovy
Trivy trivy = new Trivy(this)
boolean imageIsSafe = trivy.scanImage("ubuntu:24.04")
if (!imageIsSafe){
echo "This image has vulnerabilities!"
}
```

### Set the severity level for the scan

You can set the severity levels of the vulnerabilities Trivy should scan for as a parameter of the scan method:

```groovy
Trivy trivy = new Trivy(this)
trivy.scanImage("ubuntu:24.04", TrivySeverityLevel.ALL)
trivy.scanImage("ubuntu:24.04", "CRITICAL,LOW")
```

For the available pre-defined severity levels see [TrivySeverityLevel.groovy](src/com/cloudogu/ces/cesbuildlib/TrivySeverityLevel.groovy)

### Set the pipeline strategy

To define how the Jenkins pipeline should behave if vulnerabilities are found, you can set certain strategies:
- TrivyScanStrategy.IGNORE: Ignore the vulnerabilities and continue
- TrivyScanStrategy.UNSTABLE: Mark the job as "unstable" and continue
- TrivyScanStrategy.FAIL: Mark the job as failed

```groovy
Trivy trivy = new Trivy(this)
trivy.scanImage("ubuntu:24.04", TrivySeverityLevel.ALL, TrivyScanStrategy.UNSTABLE)
```

### Set additional Trivy flags

To set additional Trivy command flags, use the `additionalFlags` parameter:

```groovy
Trivy trivy = new Trivy(this)
trivy.scanImage("ubuntu:24.04", TrivySeverityLevel.ALL, TrivyScanStrategy.UNSTABLE, "--db-repository public.ecr.aws/aquasecurity/trivy-db")
```

Note that the flags `--db-repository public.ecr.aws/aquasecurity/trivy-db --java-db-repository public.ecr.aws/aquasecurity/trivy-java-db`
are set by default to avoid rate limiting of Trivy database downloads. If you set `additionalFlags` by yourself, you are overwriting
these default flags and have to make sure to include them in your set of additional flags, if needed.

### Set the Trivy report file name

If you want to run multiple image scans in one pipeline, you can set distinct file names for the report files:

```groovy
Trivy trivy = new Trivy(this)
trivy.scanImage("ubuntu:20.04", TrivySeverityLevel.ALL, TrivyScanStrategy.UNSTABLE, "", "trivy/ubuntu20.json")
trivy.scanImage("ubuntu:24.04", TrivySeverityLevel.ALL, TrivyScanStrategy.UNSTABLE, "", "trivy/ubuntu24.json")
// Save report by using the same file name (last parameter)
trivy.saveFormattedTrivyReport(TrivyScanFormat.HTML, "UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL", "ubuntu20.04report", "trivy/ubuntu20.json")
```

## Save Trivy report in another file format

After calling the `scanImage` function you can save the scan report as JSON, HTML or table files.

```groovy
Trivy trivy = new Trivy(this)
trivy.scanImage("ubuntu:24.04")
trivy.saveFormattedTrivyReport(TrivyScanFormat.TABLE)
trivy.saveFormattedTrivyReport(TrivyScanFormat.JSON)
trivy.saveFormattedTrivyReport(TrivyScanFormat.HTML)
```

You may filter the output to show only specific severity levels (default: `"UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL"`):

```groovy
Trivy trivy = new Trivy(this)
trivy.scanImage("ubuntu:24.04")
trivy.saveFormattedTrivyReport(TrivyScanFormat.TABLE, "CRITICAL")
trivy.saveFormattedTrivyReport(TrivyScanFormat.JSON, "UNKNOWN,LOW,MEDIUM")
```

You may also use any other supported [Trivy format](https://trivy.dev/v0.57/docs/references/configuration/cli/trivy_convert/) or a custom template from a file in your workspace.

```groovy
Trivy trivy = new Trivy(this)
trivy.scanImage("ubuntu:24.04")
trivy.saveFormattedTrivyReport("cosign-vuln", "UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL", "ubuntu24.04cosign.txt")
trivy.saveFormattedTrivyReport("template --template @myTemplateFile.xyz", "UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL", "ubuntu24.04myTemplate.txt")
```

## Scan Dogu image with Trivy

This section describes how to get a Dogu image from the testing CES instance and scan it with Trivy.

### Get Dogu image from CES instance

Make sure to have a `build` stage in your Dogu test pipeline which builds the Dogu image, e.g. via
the `ecoSystem.build("/dogu")` command.
After the build stage you will be able to copy the Dogu image to your local Jenkins worker via
the `ecoSystem.copyDoguImageToJenkinsWorker("/dogu")` command.

### Scan Dogu image

The `scanDogu()` function lets you scan a Dogu image without typing its full name. The method reads the image name
and version from the dogu.json inside the directory you point it to via its first argument.
The default directory is the current directory.

```groovy
// Preparation
ecoSystem.copyDoguImageToJenkinsWorker("/dogu")
Trivy trivy = new Trivy(this)

// Scan the Dogu image
trivy.scanDogu()
// Explicitly set directory that contains the dogu code (dogu.json)
trivy.scanDogu("subfolder/test1/jenkins")
// Set scan options just like in the scanImage method
trivy.scanDogu(".", TrivySeverityLevel.ALL, TrivyScanStrategy.UNSTABLE, "", "trivy/mydogu.json")
trivy.saveFormattedTrivyReport(TrivyScanFormat.TABLE)
trivy.saveFormattedTrivyReport(TrivyScanFormat.JSON)
trivy.saveFormattedTrivyReport(TrivyScanFormat.HTML)
```

## Ignore / allowlist

If you want to ignore / allow certain vulnerabilities, please use a `.trivyignore` file.

Provide the file in your repo `/` directory where you run your job, e.g.:

```shell
.gitignore
Jenkinsfile
.trivyignore
```

[Offical documentation](https://trivy.dev/v0.57/docs/configuration/filtering/#by-finding-ids)
```ignorelang
# Accept the risk
CVE-2018-14618

# Accept the risk until 2023-01-01
CVE-2019-14697 exp:2023-01-01

# No impact in our settings
CVE-2019-1543

# Ignore misconfigurations
AVD-DS-0002

# Ignore secrets
generic-unwanted-rule
aws-account-id
```

# Steps

## mailIfStatusChanged
Expand Down Expand Up @@ -1293,7 +1467,9 @@ For example, if running on `http(s)://server:port/jenkins`, `server` is returned

Returns true if the build is successful, i.e. not failed or unstable (yet).

## findVulnerabilitiesWithTrivy
## findVulnerabilitiesWithTrivy (Deprecated)

This function is deprecated. Use [Trivy](#trivy) functionality instead.

Returns a list of vulnerabilities or an empty list if there are no vulnerabilities for the given severity.

Expand Down Expand Up @@ -1330,36 +1506,7 @@ node {
}
```

### Ignore / allowlist

If you want to ignore / allow certain vulnerabilities please use a .trivyignore file
Provide the file in your repo / directory where you run your job
e.g.:
```shell
.gitignore
Jenkinsfile
.trivyignore
```

[Offical documentation](https://aquasecurity.github.io/trivy/v0.41/docs/configuration/filtering/#by-finding-ids)
```ignorelang
# Accept the risk
CVE-2018-14618

# Accept the risk until 2023-01-01
CVE-2019-14697 exp:2023-01-01

# No impact in our settings
CVE-2019-1543

# Ignore misconfigurations
AVD-DS-0002

# Ignore secrets
generic-unwanted-rule
aws-account-id

```

If there are vulnerabilities the output looks as follows.

Expand Down
37 changes: 35 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,12 @@
<artifactId>groovy-all</artifactId>
<version>2.5.23</version>
<type>pom</type>
<exclusions>
<exclusion> <!-- declare the exclusion here -->
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-test</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
Expand All @@ -61,7 +67,14 @@
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<version>5.14.2</version>
<version>3.6.28</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>3.6.28</version>
<scope>test</scope>
</dependency>

Expand All @@ -75,7 +88,14 @@
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.11.3</version>
<version>5.4.2</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
<version>5.4.2</version>
<scope>test</scope>
</dependency>

Expand All @@ -86,6 +106,19 @@
<scope>test</scope>
</dependency>


<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.18.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
<version>1.27.1</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
Expand Down
2 changes: 1 addition & 1 deletion src/com/cloudogu/ces/cesbuildlib/Docker.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -401,7 +401,7 @@ class Docker implements Serializable {
def userName = sh.returnStdOut('whoami')
String jenkinsUserFromEtcPasswd = sh.returnStdOut "cat /etc/passwd | grep $userName"

if (jenkinsUserFromEtcPasswd.isEmpty()) {
if (jenkinsUserFromEtcPasswd == null || jenkinsUserFromEtcPasswd.isEmpty()) {
script.error 'Unable to parse user jenkins from /etc/passwd.'
}
return jenkinsUserFromEtcPasswd
Expand Down
2 changes: 1 addition & 1 deletion src/com/cloudogu/ces/cesbuildlib/Sh.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ class Sh implements Serializable {
// Trim to remove trailing line breaks, which result in unwanted behavior in Jenkinsfiles:
// E.g. when using output in other sh() calls leading to executing the sh command after the line breaks,
// possibly discarding additional arguments
.trim()
?.trim()
}
}
Loading