Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions e2e/tests/ui/features/@search/search.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
Feature: Search
As a Devsecops Engineer
I want to perform searching across vulnerabilities, SBOMs and packages, specific searches for CVE IDs, SBOM titles, package names and show results that are easy to navigate to the specific item of interest.

Background:
Given User is authenticated
And User is on the Search page

Scenario: User visits search page without filling anything
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We definitely shouldn't hardcode numbers of SBOMs, packages etc. This would make the test suite annoying to maintain, as we add more SBOMs with new tests. I'm not sure if this scenario should be used at all, to be honest. The only way I can imagine this could be useful is if we did an API check for number of SBOMs, packages etc. and compared it with the numbers shown on the search page.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. I asked about this before during our daily sync and I was told to hard-code it for now and then later re-write it using the API. The comparison is also not strict, meaning that there should be at least a given number of SBOMs, packages, etc, but a bigger number is also tolerated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matejnesuta Can you please parameterize these values with Examples section? something like this,

  Then a total number of "<Count1>" "<Label1>" should be visible in the tab
  And a total number of "<Count2>" "<Label2>" should be visible in the tab
  And a total number of "<Count3>" "<Label3>" should be visible in the tab
  And a total number of "<Count4>" "<Label4>" should be visible in the tab

Examples:
  | Count1 | Label1  | Count2 | Label2     | Count3 | Label3          | Count4 | Label4     |
  | 17     | SBOMs   | 5537   | Packages   | 29     | Vulnerabilities | 57     | Advisories |

Then a total number of 17 "SBOMs" should be visible in the tab
And a total number of 5537 "Packages" should be visible in the tab
And a total number of 29 "Vulnerabilities" should be visible in the tab
And a total number of 57 "Advisories" should be visible in the tab

Scenario Outline: User toggles the "<types>" list and manipulates the list
When User selects the Tab "<types>"
Then the "<types>" list should have specific filter set
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my opinion, the wording of this step is not clear. We could make this step more useful (and reusable), if the step was called something like Then the "<types>" list should have the <filter> filter set. This way it would be immediately visible what the filter is.

And the "<types>" list should be sortable
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably make sure each column is sortable. BUT I think that this may be redundant, as this is already being tested here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though on second thought it may be a good idea to still test this, as BDD scenarios are likely to run separately from the tests written by Carlos, but the test step could re-use the methods defined in his page objects.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be fair, I am kind of inclined to rewriting this, so the test just checks relevant aria-sort label.

I am currently not sure, whether the sorting is performed on frontend or backend, burt I feel like it should be tested either with some sort of React component test, or an API test. I tried to test it using an E2E test, but I have not been able to make it not flake in the Github CI.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The methods verifySorting validates the frontend - It is capable of validating different data types (date, cve,..) as well.

And the "<types>" list should be limited to 10 items
And the user should be able to switch to next "<types>" items
And the user should be able to increase pagination for the "<types>"
Comment on lines +19 to +21
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these steps seems to Pagination related which can be verified with a single step definition using verifyPagination method under ToolbarTable.ts helper file.

And First column on the search results should have the link to "<types>" explorer pages

Examples:
|types|
|SBOMs|
# |Packages|
|Vulnerabilities|
|Advisories|

Scenario Outline: Download Links on the "<types>" Search Result list
When User selects the Tab "<types>"
Then Tab "<types>" is visible
And Download link should be available for the "<types>" list

Examples:
|types|
|SBOMs|
|Advisories|

Scenario Outline: Autofill shows results matched on <input>
When user starts typing a "<input>" in the search bar
Then the autofill dropdown should display items matching the "<input>"
And the results should be limited to 5 suggestions

Examples:
|input|
|quarkus|
|CVE-2022|
|policies|

Scenario: Search bar should not preview anything when no matches are found
And user starts typing a "non-existent name" in the search bar
Then The autofill drop down should not show any values

Scenario Outline: User searches for a specific "<type>"
When user types a "<type-instance>" in the search bar
And user presses Enter
And User selects the Tab "<types>"
Then the "<types>" list should display the specific "<type-instance>"
And the list should be limited to 10 items or less
And the user should be able to filter "<types>"
And user clicks on the "<type-instance>" "<type>" link
And the user should be navigated to the specific "<type-instance>" page

Examples:
|type|types|type-instance|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have just type or types?

|SBOM|SBOMs|quarkus-bom|
|CVE|Vulnerabilities|CVE-2022-45787|
|Package|Packages|quarkus|
|Advisory|Advisories|CVE-2022-45787|
275 changes: 275 additions & 0 deletions e2e/tests/ui/features/@search/search.step.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,275 @@
import { SearchPage } from "../../helpers/SearchPage";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With Carlos' page object PR merged, we should probably consider these classes outdated and use the ones from that merge request if possible to avoid redundancy.

import { ToolbarTable } from "../../helpers/ToolbarTable";
import { Tabs } from "../../helpers/Tabs";
import { DetailsPage } from "../../helpers/DetailsPage";
import { createBdd } from "playwright-bdd";
import { expect } from "@playwright/test";

export const { Given, When, Then } = createBdd();

/**
* This function returns table identifier and column, which contains link to the details page
* @param type Catogory of the data to get the table identifier and column
*/
function getTableInfo(type: string): [string, string] {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Helper functions should not be defined in the step definition file. Please use the methods defined for the page objects as mentioned above, if applicable, and if you have to extend them, please do it there.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree

switch (type) {
case "SBOMs":
case "SBOM":
return ["sbom-table", "Name"];
case "Advisories":
case "Advisory":
return ["advisory-table", "ID"];
case "Vulnerabilities":
case "CVE":
return ["Vulnerability table", "ID"];
case "Packages":
case "Package":
return ["Package table", "Name"];
default:
throw new Error(`Unknown type: ${type}`);
}
}

function getPaginationId(type: string): string {
switch (type) {
case "Vulnerabilities":
return "vulnerability-table-pagination-top";
case "Advisories":
return "advisory-table-pagination-top";
case "Packages":
return "package-table-pagination-top";
case "SBOMs":
return "sbom-table-pagination-top";
default:
throw new Error(`Unknown type: ${type}`);
}
}

function getColumns(type: string): string[] {
switch (type) {
case "Vulnerabilities":
return ["ID", "CVSS", "Date published"];
case "Advisories":
return ["ID", "Revision"];
case "Packages":
return ["Name", "Namespace", "Version"];
case "SBOMs":
return ["Name", "Created on"];
default:
throw new Error(`Unknown type: ${type}`);
}
}

Given("User is on the Search page", async ({ page }) => {
const searchPage = new SearchPage(page);
await searchPage.open();
});

Then(
"Download link should be available for the {string} list",
async ({ page }, type: string) => {
const table = new ToolbarTable(page, getTableInfo(type)[0]);
await table.verifyDownloadLink(type);
},
);

When(
"user starts typing a {string} in the search bar",
async ({ page }, searchText: string) => {
const searchPage = new SearchPage(page);
await searchPage.typeInSearchBox(searchText);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure methods for something like this already exist in the repo. Again, please, let's avoid redundant code and use what can be reused.

},
);

Then("The autofill drop down should not show any values", async ({ page }) => {
const searchPage = new SearchPage(page);
await searchPage.autoFillIsNotVisible();
});

When(
"user types a {string} in the search bar",
async ({ page }, searchText: string) => {
const searchPage = new SearchPage(page);
await searchPage.typeInSearchBox(searchText);
},
);

When("user presses Enter", async ({ page }) => {
await page.keyboard.press("Enter");
});

Then(
"the {string} list should display the specific {string}",
async ({ page }, type: string, name: string) => {
const tabs = new Tabs(page);
await tabs.verifyTabIsSelected(type);
const info = getTableInfo(type);
const table = new ToolbarTable(page, info[0]);
await table.verifyColumnContainsText(info[1], name);
},
);

Then(
"the list should be limited to {int} items or less",
async ({ page }, count: number) => {
const table = new ToolbarTable(page, "sbom-table");
await table.verifyTableHasUpToRows(count);
},
);

Then(
"user clicks on the {string} {string} link",
async ({ page }, arg: string, type: string) => {
const info = getTableInfo(type);
const table = new ToolbarTable(page, info[0]);
await table.openDetailsPage(arg, info[1]);
},
);

Then(
"the user should be navigated to the specific {string} page",
async ({ page }, arg: string) => {
const detailsPage = new DetailsPage(page);
await detailsPage.verifyPageHeader(arg);
},
);

Then(
"the user should be able to filter {string}",
async ({ page }, arg: string) => {
const table = new ToolbarTable(page, getTableInfo(arg)[0]);
if (arg === "SBOMs") {
await table.filterByDate("12/22/2025", "12/22/2025");
await table.verifyColumnDoesNotContainText("Name", "quarkus-bom");
await table.clearFilter();
await table.verifyColumnContainsText("Name", "quarkus-bom");
} else if (arg === "Vulnerabilities") {
await page.getByLabel("Critical").click();
await table.verifyColumnDoesNotContainText("ID", "CVE-2022-45787");
await table.clearFilter();
await table.verifyColumnContainsText("ID", "CVE-2022-45787");
} else if (arg === "Packages") {
await page.getByLabel("OCI").click();
await table.verifyColumnDoesNotContainText("Name", "quarkus");
await table.clearFilter();
await table.verifyColumnContainsText("Name", "quarkus");
} else if (arg === "Advisories") {
await table.filterByDate("12/22/2025", "12/22/2025");
await table.verifyColumnDoesNotContainText("ID", "CVE-2022-45787");
await table.clearFilter();
await table.verifyColumnContainsText("ID", "CVE-2022-45787");
Comment on lines +142 to +160
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these values should be from Examples section of the feature file - Never hard code data into the low level or mid level keywords.

}
},
);

Then(
"the {string} list should have specific filter set",
async ({ page }, arg: string) => {
if (arg === "Vulnerabilities") {
await expect(page.locator("h4").getByText("CVSS")).toBeVisible();
await expect(page.locator("h4").getByText("Created on")).toBeVisible();
await expect(
page.locator('input[aria-label="Interval start"]'),
).toBeVisible();
await expect(
page.locator('input[aria-label="Interval end"]'),
).toBeVisible();
} else if (arg === "Advisories") {
await expect(page.locator("h4").getByText("Revision")).toBeVisible();
await expect(
page.locator('input[aria-label="Interval start"]'),
).toBeVisible();
await expect(
page.locator('input[aria-label="Interval end"]'),
).toBeVisible();
} else if (arg === "Packages") {
await expect(page.getByRole("heading", { name: "Type" })).toBeVisible();
await expect(
page.getByRole("heading", { name: "Architecture" }),
).toBeVisible();
} else if (arg === "SBOMs") {
await expect(page.getByText("Created onFrom To")).toBeVisible();
}
},
);

Then("the {string} list should be sortable", async ({ page }, arg: string) => {
var columns: string[] = getColumns(arg);
var id: string = getPaginationId(arg);

const table = new ToolbarTable(page, getTableInfo(arg)[0]);
await table.verifySorting(`xpath=//div[@id="${id}"]`, columns);
});

Then(
"the {string} list should be limited to {int} items",
async ({ page }, type: string, count: number) => {
const info = getTableInfo(type);
const table = new ToolbarTable(page, info[0]);
const tableTopPagination = `xpath=//div[@id="${getPaginationId(type)}"]`;
await table.selectPerPage(tableTopPagination, "10 per page");
await table.verifyTableHasUpToRows(count);
},
);

Then(
"the user should be able to switch to next {string} items",
async ({ page }, arg: string) => {
var id: string = getPaginationId(arg);
const info = getTableInfo(arg);
const table = new ToolbarTable(page, info[0]);
await table.verifyPagination(`xpath=//div[@id="${id}"]`);
},
);

Then(
"the user should be able to increase pagination for the {string}",
async ({ page }, arg: string) => {
const info = getTableInfo(arg);
const table = new ToolbarTable(page, info[0]);
var id: string = getPaginationId(arg);
const tableTopPagination = `xpath=//div[@id="${id}"]`;
await table.verifyPagination(`xpath=//div[@id="${id}"]`);
await table.goToFirstPage(tableTopPagination);
await table.selectPerPage(tableTopPagination, "20 per page");
await table.goToFirstPage(tableTopPagination);
await table.verifyTableHasUpToRows(20);
Comment on lines +226 to +236
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step seems to be redundant as "the user should be able to switch to next {string} items" has table.verifyPagination() method to verify the same.

},
);

Then(
"First column on the search results should have the link to {string} explorer pages",
async ({ page }, arg: string) => {
const info = getTableInfo(arg);
const table = new ToolbarTable(page, info[0]);
await table.verifyColumnContainsLink(info[1], arg);
},
);

Then(
"a total number of {int} {string} should be visible in the tab",
async ({ page }, count: number, arg: string) => {
await page.waitForLoadState("networkidle");
const tabs = new Tabs(page);
await tabs.verifyTabHasAtLeastResults(arg, count);
},
);

Then(
"the autofill dropdown should display items matching the {string}",
async ({ page }, arg: string) => {
const searchPage = new SearchPage(page);
await searchPage.autoFillHasRelevantResults(arg);
},
);

Then(
"the results should be limited to {int} suggestions",
async ({ page }, arg: number) => {
const searchPage = new SearchPage(page);
expect(await searchPage.totalAutoFillResults()).toBeLessThanOrEqual(
arg * 4,
);
await searchPage.expectCategoriesWithinLimitByHref(arg);
},
);
Loading
Loading