split sandbox between linked and isolated modes #6730

rochdev · 2025-10-23T15:30:19Z

What does this PR do?

Split sandbox between linked and isolated modes.

Motivation

The sandbox was originally created for integration tests with applications mimicking a real app with a real install of dd-trace. But for a lot of our tests, for example ESM tests, we don't really need that level of isolation, and linking to dd-trace is good enough.

Additional Notes

The main change is in integration-tests/helpers/index.js, everything else is just changing all usage of createSandbox to either isolatedSandbox or linkedSandbox.

Supersedes #6640

github-actions · 2025-10-23T15:30:57Z

Overall package size

Self size: 12.89 MB
Deduped: 115.59 MB
No deduping: 117.8 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.7.0 | 35.02 MB | 35.02 MB | | @datadog/native-appsec | 10.3.0 | 20.73 MB | 20.74 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @datadog/pprof | 5.11.1 | 9.96 MB | 10.34 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.4 | 2.95 MB | 5.73 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @opentelemetry/resources | 1.9.1 | 306.54 kB | 1.74 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api-logs | 0.206.0 | 201.39 kB | 1.42 MB | | @opentelemetry/api | 1.9.0 | 1.22 MB | 1.22 MB | | jsonpath-plus | 10.3.0 | 617.18 kB | 1.08 MB | | import-in-the-middle | 1.15.0 | 127.66 kB | 856.24 kB | | lru-cache | 10.4.3 | 804.3 kB | 804.3 kB | | @datadog/openfeature-node-server | 0.1.0-preview.12 | 95.11 kB | 401.68 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | source-map | 0.7.6 | 185.63 kB | 185.63 kB | | pprof-format | 2.2.1 | 163.06 kB | 163.06 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 7.0.5 | 63.38 kB | 63.38 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.3 | 23.74 kB | 23.74 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

codecov · 2025-10-23T15:31:13Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.19%. Comparing base (dbefc61) to head (5eb5b68).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6730      +/-   ##
==========================================
+ Coverage   84.13%   84.19%   +0.05%     
==========================================
  Files         505      509       +4     
  Lines       21038    21123      +85     
==========================================
+ Hits        17701    17785      +84     
- Misses       3337     3338       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

datadog-official · 2025-10-23T15:32:44Z

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 5eb5b68 | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-10-23T17:18:59Z

Benchmarks

Benchmark execution time: 2025-10-23 18:45:02

Comparing candidate commit 5eb5b68 in PR branch linked-sandbox with baseline commit dbefc61 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1605 metrics, 65 unstable metrics.

BridgeAR

Mostly LGTM. I didn't understand the addition of the modules in the versions package.json though.

BridgeAR · 2025-10-24T10:00:14Z

packages/dd-trace/test/plugins/versions/package.json

    "@prisma/client": "6.17.1",
    "@redis/client": "5.8.3",
    "@smithy/smithy-client": "4.9.0",
+    "@types/node": "18.19.130",


I am surprised about this addition. I believe that should not have an impact, since it is already listed in our main package.json.

It's needed by the test app. It was available before because it got hoisted from dd-trace to the test app node_modules which is no longer the case when linking.

BridgeAR · 2025-10-24T10:00:49Z

packages/dd-trace/test/plugins/versions/package.json

    "couchbase": "4.6.0",
    "cypress": "15.4.0",
    "cypress-fail-fast": "7.1.1",
+    "dc-polyfill": "0.1.10",


I am not sure about this change. Why do we need to add that to the versions package.json?

It's needed by the test app. It was available before because it got hoisted from dd-trace to the test app node_modules which is no longer the case when linking.

BridgeAR · 2025-10-24T10:01:57Z

integration-tests/helpers/index.js

+ */
 async function createSandbox (dependencies = [], isGitRepo = false,
-  integrationTestsPaths = ['./integration-tests/*'], followUpCommand) {
+  integrationTestsPaths = ['./integration-tests/*'], followUpCommand, isolated = true) {


We do not really need the default value, since it is explicitely passed through all the time.

I would probably change the arguments order to make the default values arguments at the end.

It's also used by useSandbox, but I guess it could also pass it explicitly.

BridgeAR · 2025-10-24T10:04:11Z

integration-tests/helpers/index.js

-  execHelper(addCommand, addOptions)
+  if (isolated) {
+    execHelper(`npm pack --silent --pack-destination ${folder}`, { env: restOfEnv })
+    execHelper(`yarn add ${deps.concat(`file:${out}`).join(' ')} ${addFlags.join(' ')}`, addOptions)


Suggested change

execHelper(`yarn add ${deps.concat(`file:${out}`).join(' ')} ${addFlags.join(' ')}`, addOptions)

execHelper(`yarn add ${deps.join(' ')} `file:${out}` ${addFlags.join(' ')}`, addOptions)

rochdev · 2025-10-24T14:48:45Z

@BridgeAR Some of your comments around dependencies made me realize that by moving to linking, we lose our only safety net that guarantees that we're not using a dev dependency by accident in an integration. Since they are lazy loaded, the existing integration doesn't otherwise cover them.

Because of this, I think we should keep all integration tests isolated. The problem we were trying to solve also no longer exists after the switch to Bun, not only because it's really fast to begin with, but it's also able to reuse the dependencies that were pulled from the install on the repo itself, so it doesn't need to pull the packages again from internet. With that said and given there are basically no performance benefits and we lose some important aspects of these tests, I would prefer to close this PR and keep the isolated sandbox always.

WDYT?

cc @bengl

rochdev added 2 commits October 23, 2025 11:26

add linked version of sandbox

36d3f8a

switch to linked sandbox for plugin tests

fceece7

rochdev added the semver-patch label Oct 23, 2025

rochdev added 5 commits October 23, 2025 11:35

fix yarn add without dependencies

7b2eada

undo eslint weirdness

56c7c7c

move eveverything to specialized sandboxes

ba8672e

fix sinon sandbox

70b7e12

add jsdoc

56b33af

rochdev added 5 commits October 23, 2025 13:52

fix cookie-parser test

9056616

add missing flags

41e7c12

add missing dc-polyfill to langchain

bec8ad0

fix llmobs sdk

32b1031

switch to isolated sandbox for pg

5eb5b68

rochdev marked this pull request as ready for review October 23, 2025 18:48

rochdev requested review from a team as code owners October 23, 2025 18:48

rochdev requested review from BridgeAR and removed request for a team October 23, 2025 18:48

BridgeAR reviewed Oct 24, 2025

View reviewed changes

rochdev marked this pull request as draft October 24, 2025 14:41

	execHelper(`yarn add ${deps.concat(`file:${out}`).join(' ')} ${addFlags.join(' ')}`, addOptions)
	execHelper(`yarn add ${deps.join(' ')} `file:${out}` ${addFlags.join(' ')}`, addOptions)

split sandbox between linked and isolated modes #6730

Are you sure you want to change the base?

split sandbox between linked and isolated modes #6730

Uh oh!

Conversation

rochdev commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Additional Notes

Uh oh!

github-actions bot commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overall package size

Uh oh!

codecov bot commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

datadog-official bot commented Oct 23, 2025 • edited by datadog-datadog-prod-us1 bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Uh oh!

BridgeAR left a comment

Choose a reason for hiding this comment

Uh oh!

BridgeAR Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

rochdev Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

BridgeAR Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

rochdev Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

BridgeAR Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

rochdev Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

BridgeAR Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

rochdev commented Oct 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rochdev commented Oct 23, 2025 •

edited

Loading

github-actions bot commented Oct 23, 2025 •

edited

Loading

codecov bot commented Oct 23, 2025 •

edited

Loading

datadog-official bot commented Oct 23, 2025 •

edited by datadog-datadog-prod-us1 bot

Loading

pr-commenter bot commented Oct 23, 2025 •

edited

Loading