Refactor E2E tests

[fanny-jiang]

What does this PR do?

A brief description of the change being made with this pull request.

Motivation

What inspired you to submit this pull request?

Additional Notes

• github.com/DataDog/datadog-agent/test/new-e2e go module is currently pinned to an older version, so some CI checks will fail. Version will be bumped before merging. See below for testing locally

Minimum Agent Versions

Are there minimum versions of the Datadog Agent and/or Cluster Agent required?

• Agent: vX.Y.Z
• Cluster Agent: vX.Y.Z

Describe your test plan

To run locally:

From the test/e2e directory, run go get github.com/DataDog/datadog-agent/test/new-e2e@1cc7604f4b6b35a230b3dc1226d35b86034a832a

You may also need to add this replace block to the test/e2e/go.mod file and use the development version generated by the go get command:

replace (
	github.com/DataDog/datadog-agent/comp/core/tagger/types => github.com/DataDog/datadog-agent/comp/core/tagger/types <dev_version>
	github.com/DataDog/datadog-agent/comp/otelcol/ddflareextension/dev =>  github.com/DataDog/datadog-agent/comp/otelcol/ddflareextension/dev <dev_version>
	github.com/DataDog/datadog-agent/comp/otelcol/ddflareextension/def => github.com/DataDog/datadog-agent/comp/otelcol/ddflareextension/def <dev_version>
)

• Run TestLocalKindSuite tests (creates local kind environment):
  • From the /test/e2e directory: go test ./... -run TestLocalKindSuite -count=1 -v -timeout=0s
• Run TestAWSKindSuite tests (creates aws kind environment):
  • From the /test/e2e directory: aws-vault exec <aws_profile> -- make e2e-tests

Notes:

• To keep pulumi stacks, prepend the env vars, E2E_SKIP_DELETE_ON_FAILURE=true E2E_DEV_MODE=true
• Don't use make e2e-tests-keep-stacks

Checklist

• PR has at least one valid label: bug, enhancement, refactoring, documentation, tooling, and/or dependencies
• PR has a milestone or the qa/skip-qa label

[fanny-jiang]

Note: these lines are for testing in the CI. They'll be removed when test/e2e/go.mod is updated with the new version when it is available.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 48.94%. Comparing base (0ca2c91) to head (3f8fdb0).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1593   +/-   ##
=======================================
  Coverage   48.94%   48.94%           
=======================================
  Files         227      227           
  Lines       20636    20636           
=======================================
  Hits        10101    10101           
  Misses      10010    10010           
  Partials      525      525           

Flag	Coverage Δ
unittests	48.94% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0ca2c91...3f8fdb0. Read the comment docs.