Skip to content

Commit b9b15ad

Browse files
mrrajanmrrajan
committed
Scan SBOM feature test scenarios
Signed-off-by: mrrajan <[email protected].>
1 parent 596e5fe commit b9b15ad

File tree

1 file changed

+291
-0
lines changed

1 file changed

+291
-0
lines changed

‎e2e/tests/ui/features/scan-sbom.feature‎

Lines changed: 291 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,291 @@
1+
Feature: Scan SBOM - To Generate Vulnerability Report for SBOM
2+
As an RHTPA user
3+
I want to be able to scan an SBOM so that I can review the vulnerabilities within the SBOM without having to ingest
4+
5+
Background: Authentication
6+
Given User is authenticated
7+
8+
Scenario: Verify Generate Vulnerability Report Screen
9+
When User Navigates to SBOMs List page
10+
When User Clicks Generate Vulnerability Button
11+
Then The Application should navigate to Generate Vulnerability Report screen
12+
Then The Page should contain Browse files option and instruction to Drag and drop files
13+
14+
Scenario: Generate Vulnerability Report for unsupported SBOM file extensions
15+
Given User Navigated to Generate Vulnerability Report screen
16+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
17+
Then The report generation failed with error "Report failed"
18+
Then "The "<fileName>" file could not be analyzed. The file might be corrupted or an unsupported format" message should be displayed
19+
Then "Try another file" button should be displayed
20+
When User Clicks on "Try another file" button
21+
Then Application navigates to Generate Vulnerability Report screen
22+
Examples:
23+
| fileName | filePath|
24+
| <tarfile> | |
25+
26+
Scenario: Generate Vulnerability Report for unsupported SBOM format
27+
Given User Navigated to Generate Vulnerability Report screen
28+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
29+
Then The report generation failed with error "Report failed"
30+
Then "The "<fileName>" file could not be analyzed. The file might be corrupted or an unsupported format" message should displayed
31+
Then "Try another file" button should be displayed
32+
Examples:
33+
| fileName | filePath|
34+
| <SPDX 2.2> | |
35+
| <CycloneDX 1.4> | |
36+
| <CycloneDX 1.5> | |
37+
38+
Scenario: Generate Vulnerability Report For SBOM without any vulnerabilities
39+
Given User Navigated to Generate Vulnerability Report screen
40+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
41+
Then "No Vulnerabilities found" message should be displayed
42+
Then "The "<fileName>" was analyzed and found no vulnerabilities report" message should be displayed
43+
Then "Try another file" button should be displayed
44+
When User Clicks on "Try another file" button
45+
Then Application navigates to Generate Vulnerability Report screen
46+
Examples:
47+
| fileName | filePath|
48+
| <CycloneDX> | |
49+
| <SPDX> | |
50+
51+
Scenario: Cancel Generate vulnerability report
52+
Given User Navigated to Generate Vulnerability Report screen
53+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
54+
Then The Page should have a spinner with "Generating vulnerability report" message and "Cancel Report" option while processing the SBOM
55+
When User Clicks on "Cancel Report" link
56+
Then Application navigates to Generate Vulnerability Report screen
57+
Examples:
58+
| fileName | filePath|
59+
| <BigSBOMFile> | |
60+
61+
Scenario: Generate Vulnerability Report for supported SBOM file extensions
62+
Given User Navigated to Generate Vulnerability Report screen
63+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
64+
Then On the successful report generation the Application should render Vulnerability Report for the SBOM
65+
Examples:
66+
| fileName | filePath|
67+
| <json> | |
68+
| <bz2> | |
69+
70+
Scenario: Verify Generate Vulnerability Report Screen
71+
Given User Navigated to Generate Vulnerability Report screen
72+
When User Clicks on Browse files Button
73+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
74+
Then On the successful report generation the Application should render Vulnerability Report for the SBOM
75+
Then The title should be Vulnerability report with text "This is a temporary vulnerability report"
76+
Then The Vulnerabilities list should be filtered by 'Affected' status by default
77+
Then Filtering drop down should be visible with drop down values "Status" and "Importer"
78+
Then Clear filters option should be visible and enabled
79+
Then Tooltip on the "Published" column should display "The date when informartion about this vulnerability was first made available"
80+
Then Tooltip on the "Updated" column should display "The date when information about this vulnerability was most recently changed"
81+
Then "Actions" button should be visible with dropdown options "Generate new report" and "Download CSV"
82+
Examples:
83+
| fileName | filePath|
84+
| <CycloneDX> | |
85+
| <SPDX> | |
86+
87+
Scenario: Verify Vulnerabilities on Generate Vulnerability Report for an SBOM
88+
Given User Navigated to Generate Vulnerability Report screen
89+
When User Clicks on Browse files Button
90+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
91+
Then On the successful report generation the Application should render Vulnerability Report for the SBOM
92+
Then The Vulnerabilities on the Vulnerability ID column should match with "<Vulnerabilities>"
93+
Examples:
94+
| fileName | filePath| Vulnerabilities |
95+
| <CycloneDX> | | <vuln list> |
96+
| <SPDX> | | <vuln list> |
97+
98+
Scenario: Verify Vulnerability Details on Generate Vulnerability Report for an SBOM
99+
Given User Navigated to Generate Vulnerability Report screen
100+
When User Clicks on Browse files Button
101+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
102+
Then On the successful report generation the Application should render Vulnerability Report for the SBOM
103+
Then The "Description" column of the "<Vulnerability>" should match with "<severity:Importer>"
104+
Then The "Severity" column of the "<Vulnerability>" should match with "<severity:Importer>"
105+
Then The "Status" column of the "<Vulnerability>" should match with "<status>"
106+
Then The "Affected packages" column of the "<Vulnerability>" should match with "<affectedcount>"
107+
Then The "Published" column of the "<Vulnerability>" should match with "<Published>"
108+
Then The "Updated" column of the "<Vulnerability>" should match with "<Updated>"
109+
Examples:
110+
| fileName | filePath| Vulnerability | Description | severity:Importer | status | Published | Updated |
111+
| <CycloneDX> | | <vuln ID> | <Description> | <severity: source> | <csaf_status> | <date> | <date> |
112+
| <SPDX> | | <vuln ID> | <Description> | <severity: source> | <csaf_status> | <date> | <date> |
113+
114+
Scenario: Verify Affected package list on Generate Vulnerability Report for an SBOM
115+
Given User Navigated to Generate Vulnerability Report screen
116+
When User Clicks on Browse files Button
117+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
118+
When User Clicks on Affected package count button of the "<Vulnerability>"
119+
Then Affected Package list should expand
120+
Then The "Type" column of the "<Vulnerability>" affected package should match with "<Type>"
121+
Then The "Namespace" column of the "<Vulnerability>" affected package should match with "<Namespace>"
122+
Then The "Name" column of the "<Vulnerability>" affected package should match with "<Name>"
123+
Then The "Path" column of the "<Vulnerability>" affected package should match with "<Path>"
124+
Then The "Qualifiers" column of the "<Vulnerability>" affected package should match with "<Qualifiers>"
125+
Examples:
126+
| fileName | filePath| Vulnerability | Type | Namespace | Name | Path | Qualifiers |
127+
| <CycloneDX> | | <vuln ID> | <Type> | <Namespace> | <Name>| <Path>|<Qualifiers>|
128+
| <SPDX> | | <vuln ID> | <Type> | <Namespace> | <Name>| <Path>|<Qualifiers>|
129+
130+
Scenario: Verify Filtering on Generate Vulnerability Report for an SBOM
131+
Given User Navigated to Generate Vulnerability Report screen
132+
When User Clicks on Browse files Button
133+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
134+
When User Applies "<filter>" filter with "<value>" on the Vulnerability Report
135+
Then Applied "<filter>" should be visible with "<value>" on the filter bar
136+
Then The Vulnerabilities on the Vulnerability ID column should match with "<Vulnerabilities>"
137+
When User Clicks on "Clear filters" option
138+
Then All the applied filters should be cleared
139+
Examples:
140+
| fileName | filePath| filter | value | Vulnerabilities |
141+
| <CycloneDX> | | Status | <value> | <vuln list> |
142+
| <CycloneDX> | | Status | <value> | <vuln list> |
143+
| <SPDX> | | Status | <value> | <vuln list> |
144+
| <SPDX> | | Status | <value> | <vuln list> |
145+
| <CycloneDX> | | Importer | <value> | <vuln list> |
146+
| <SPDX> | | Importer | <value> | <vuln list> |
147+
| <SPDX> | | Vulnerability ID | <value> | <vuln list> |
148+
149+
Scenario: Verify Severity on Generate Vulnerability Report for an SBOM
150+
Given User Navigated to Generate Vulnerability Report screen
151+
When User Clicks on Browse files Button
152+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
153+
When User Applies "<filter>" filter with "<value>" on the Vulnerability Report
154+
Then Applied "<filter>" should be visible with "<value>" on the filter bar
155+
Then The Vulnerabilities on the Vulnerability ID column should match with "<Vulnerabilities>"
156+
Then The "Severity" of the "<Vulnerability>" should match with "<severity:importer>"
157+
Examples:
158+
| fileName | filePath| filter | value | Vulnerabilities | Vulnerability | severity:importer |
159+
| <CycloneDX> | | Severity | Low | <vuln list> | <vuln ID> | <severity: importer> |
160+
| <CycloneDX> | | Severity | Medium | <vuln list> | <vuln ID> | <severity: importer> |
161+
| <CycloneDX> | | Severity | High | <vuln list> | <vuln ID> | <severity: importer> |
162+
| <CycloneDX> | | Severity | Critical | <vuln list> | <vuln ID> | <severity: importer> |
163+
| <SPDX> | | Severity | Low | <vuln list> | <vuln ID> | <severity: importer> |
164+
| <SPDX> | | Severity | Medium | <vuln list> | <vuln ID> | <severity: importer> |
165+
| <SPDX> | | Severity | High | <vuln list> | <vuln ID> | <severity: importer> |
166+
| <SPDX> | | Severity | Critical | <vuln list> | <vuln ID> | <severity: importer> |
167+
168+
Scenario: Verify Multiple filtering on Generate Vulnerability Report for an SBOM
169+
Given User Navigated to Generate Vulnerability Report screen
170+
When User Clicks on Browse files Button
171+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
172+
When User Applies "<filter1>" filter with "<value1>" on the Vulnerability Report
173+
When User Applies "<filter2>" filter with "<value2>" on the Vulnerability Report
174+
Then Applied "<filter1>" should be visible with "<value1>" on the filter bar
175+
Then Applied "<filter2>" should be visible with "<value2>" on the filter bar
176+
Then The Vulnerabilities on the Vulnerability ID column should match with "<Vulnerabilities>"
177+
Then The "Severity" of the "<Vulnerability>" should match with "<severity:importer>"
178+
Examples:
179+
| fileName | filePath| filter1 | value1 | filter2 | value2 | Vulnerabilities | Vulnerability | severity:importer |
180+
| <CycloneDX> | | Severity | Low | Status | Affected | <vuln list> | <vuln ID> | <severity: importer> |
181+
| <CycloneDX> | | Severity | Medium | Importer | OSV | <vuln list> | <vuln ID> | <severity: importer> |
182+
| <SPDX> | | Severity | High | Status | Fixed | <vuln list> | <vuln ID> | <severity: importer> |
183+
| <SPDX> | | Severity | Critical | Importer | CVE | <vuln list> | <vuln ID> | <severity: importer> |
184+
185+
Scenario: Verify Actions on Generate Vulnerability Report for an SBOM
186+
Given User Navigated to Generate Vulnerability Report screen
187+
When User Clicks on Browse files Button
188+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
189+
When User Clicks on "Actions" button
190+
Then The "Actions" dropdown should have options "Generate new report" and "Download CSV"
191+
When User Clicks on "Generate new report" option from the "Actions" dropdown
192+
Then Application navigates to Generate Vulnerability Report screen
193+
Examples:
194+
| fileName | filePath|
195+
| <CycloneDX> | |
196+
| <SPDX> | |
197+
198+
Scenario: Verify Download CSV on Generate Vulnerability Report for an SBOM
199+
Given User Navigated to Generate Vulnerability Report screen
200+
When User Clicks on Browse files Button
201+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
202+
When User Clicks on "Actions" button
203+
Then The "Actions" dropdown should have options "Generate new report" and "Download CSV"
204+
When User Clicks on "Download CSV" option from the "Actions" dropdown
205+
Then The Vulnerability report CSV file should be downloaded
206+
Examples:
207+
| fileName | filePath|
208+
| <CycloneDX> | |
209+
| <SPDX> | |
210+
211+
Scenario: Verify Download and Leave on Generate Vulnerability Report for an SBOM
212+
Given User Navigated to Generate Vulnerability Report screen
213+
When User Clicks on Browse files Button
214+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
215+
When User Clicks on "<Vulnerability>" from the Vulnerability ID column
216+
Then A modal window should open with "Leave Vulnerability Report?" message
217+
When User Clicks on "Download and Leave" button from the modal window
218+
Then The Vulnerability report CSV file should be downloaded
219+
Then Application navigates to Vulnerability Explorer screen
220+
Examples:
221+
| fileName | filePath| Vulnerability |
222+
| <CycloneDX> | | <vuln ID> |
223+
| <SPDX> | | <vuln ID> |
224+
225+
Scenario: Verify Leave without Downloading on Generate Vulnerability Report for an SBOM
226+
Given User Navigated to Generate Vulnerability Report screen
227+
When User Clicks on Browse files Button
228+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
229+
When User Clicks on "<Vulnerability>" from the Vulnerability ID column
230+
Then A modal window should open with "Leave Vulnerability Report?" message
231+
When User Clicks on "Leave without Downloading" button from the modal window
232+
Then Application navigates to Vulnerability Explorer screen
233+
Examples:
234+
| fileName | filePath| Vulnerability |
235+
| <CycloneDX> | | <vuln ID> |
236+
| <SPDX> | | <vuln ID> |
237+
238+
Scenario: Verify Cancel on Leave Vulnerability Report modal window
239+
Given User Navigated to Generate Vulnerability Report screen
240+
When User Clicks on Browse files Button
241+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
242+
When User Clicks on "<Vulnerability>" from the Vulnerability ID column
243+
Then A modal window should open with "Leave Vulnerability Report?" message
244+
When User Clicks on "Cancel" button from the modal window
245+
Then Application should remain on the Generate Vulnerability Report screen
246+
Examples:
247+
| fileName | filePath| Vulnerability |
248+
| <CycloneDX> | | <vuln ID> |
249+
| <SPDX> | | <vuln ID> |
250+
251+
Scenario: Verify Pagination on Generate Vulnerability Report for an SBOM
252+
Given User Navigated to Generate Vulnerability Report screen
253+
When User Clicks on Browse files Button
254+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
255+
Then Pagination of Vulnerability list works
256+
Examples:
257+
| fileName | filePath|
258+
| <BigSBOMFile> | |
259+
260+
Scenario: Generate Vulnerability Report for BigSBOMFile
261+
Given User Navigated to Generate Vulnerability Report screen
262+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
263+
Then On the successful report generation the Application should render Vulnerability Report for the SBOM
264+
Examples:
265+
| fileName | filePath|
266+
| <BigSBOMFile> | |
267+
268+
Scenario: Generate Vulnerability Report with Drag and Drop
269+
Given User Navigated to Generate Vulnerability Report screen
270+
When User Drags and Drops SBOM "<fileName>" from "<filePath>" to the Drop area
271+
Then On the successful report generation the Application should render Vulnerability Report for the SBOM
272+
Examples:
273+
| fileName | filePath|
274+
| <CycloneDX> | |
275+
| <SPDX> | |
276+
| <json> | |
277+
| <bz2> | |
278+
| <BigSBOMFile> | |
279+
280+
Scenario: Verify Sorting on Generate Vulnerability Report for an SBOM
281+
Given User Navigated to Generate Vulnerability Report screen
282+
When User Clicks on Browse files Button
283+
When User Selects SBOM "<fileName>" from "<filePath>" on the file explorer dialog window
284+
When User Sorts the "<column>" column in "<order>" order
285+
Then The Values on the "<column>" column should be sorted in "<order>" order
286+
Examples:
287+
| fileName | filePath| column | order |
288+
| <fileName> | | Severity | Ascending|
289+
| <fileName> | | Severity | Descending|
290+
291+
# Placeholders like <fileName>, <filePath>, <column>, and <order> should be replaced in the Examples table above

0 commit comments

Comments
 (0)