Manual Test Plan for 8.3.0 release

[simitt]

When picking up a test case, please add your name to this overview beforehand and tick the checkbox when finished. Testing can be started when the first build candidate (BC) is available.

• Smoke Testing ESS setup: Given a lack of automated tests running on ESS, we need to conduct some manual testing on Cloud to ensure no regressions are introduce.

  • Create an APM & Fleet Server deployment in 7.17 and then upgrade to 8.3.0 (BC) and verify that
    • The APM Server starts up and can ingest data without throwing any errors, data show up in the APM UI.
    • When enabling metrics collection in the Cloud UI, the Stack Monitoring UI shows data for the APM Server.
    • The cloud troubleshooting logs and metrics are accessible as expected.
  • Create an APM & Fleet Server deployment in 7.17 and then upgrade to 8.3.0 (BC) and switch to managed mode and verify that
    • The APM Server starts up and can ingest data without throwing any errors, data show up in the APM UI.
    • When enabling metrics collection in the Cloud UI, the Stack Monitoring UI shows data for the APM Server.
    • The cloud troubleshooting logs and metrics are accessible as expected.
  • Create an Ingest Server deployment in 8.2 and then upgrade to 8.3.0 (BC) and verify that
    • The APM Server starts up and can ingest data without throwing any errors, data show up in the APM UI.
    • When enabling metrics collection in the Cloud UI, the Stack Monitoring UI shows data for the APM Server.
    • The cloud troubleshooting logs and metrics are accessible as expected.
  • Create an Ingest Server deployment in 8.3.0 (BC) and verify that
    • The APM Server starts up and can ingest data without throwing any errors, data show up in the APM UI.
    • When enabling metrics collection in the Cloud UI, the Stack Monitoring UI shows data for the APM Server.
    • The cloud troubleshooting logs and metrics are accessible as expected.

• Testing the new k8s webhook before making it available as tech preview, by following the README.

When picking up a test case from the github board, add yourself as assignee on the PR before you start testing:

• Testing new features and bug fixes as outlined in 8.3.0 user facing test plan
• Testing apmbench related changes according to 8.3.0 apmbench test plan

Link any found regressions here.

[axw]

I've created an 8.3.0-BC4 deployment. The smoke test went as expected, except that the stack monitoring ES output metrics are all zero: #8383

[simitt]

8.3 has been released - closing this