Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(instrumentation-aws-sdk): update devDeps to modern version of instrumented packages #2723

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

trentm
Copy link
Contributor

@trentm trentm commented Feb 19, 2025

This flips the usage of 'npm test' and 'npm run test-all-versions' so that
the former uses recent versions of the instrumented packages. This allows
updating devDeps to the latest. 'npm test' will skip tests for older
versions of node that aren't supported by the latest version of the
instrumented packages.

Testing all the old versions (of node and instrumented packages) is
left to 'npm run test-all-versions' (TAV).

Refs: #2722

skipping tests on older Node.js versions is awkward

The mechanism proposed here to skip running tests in npm test for older Node.js versions is to use a preload script via mocha --require ... (as is already being used for TS-to-JS transpilation) and a config envvar to state the min supported Node.js version. Effectively this:

SKIP_TEST_IF_NODE_OLDER_THAN=18 mocha --require '../../../scripts/skip-test-if.js' 'test/**/*.test.ts'

That preload "skip-test-if.js" will process.exit(0) to skip tests.
I'll mention some alternatives attempted in comments below.

…strumented packages

This flips the usage of 'npm test' and 'npm run test-all-versions' so that
the former uses recent versions of the instrumented packages. This allows
updating devDeps to the latest. 'npm test' will *skip* tests for older
versions of node that aren't supported by the latest version of the
instrumented packages.

Testing all the old versions (of node and instrumented packages) is
left to 'npm run test-all-versions' (TAV).

Refs: open-telemetry#2722
@trentm trentm self-assigned this Feb 19, 2025
@trentm trentm requested a review from a team as a code owner February 19, 2025 00:54
@github-actions github-actions bot requested a review from blumamir February 19, 2025 00:54
@github-actions github-actions bot requested review from jj22ee and trivikr February 19, 2025 00:54
Copy link

codecov bot commented Feb 19, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.74%. Comparing base (8a5c214) to head (361852f).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2723      +/-   ##
==========================================
+ Coverage   92.41%   92.74%   +0.33%     
==========================================
  Files         171      171              
  Lines        8150     8150              
  Branches     1653     1653              
==========================================
+ Hits         7532     7559      +27     
+ Misses        618      591      -27     

see 2 files with indirect coverage changes

@trentm
Copy link
Contributor Author

trentm commented Feb 19, 2025

alternative attempted: top-level "skip this test file" logic

An alternative I've used before is to have a top-level if-block in a give test file that just exited if the node version was too old for the given version of the module, e.g.:
https://github.com/elastic/apm-agent-nodejs/blob/fc37783b9e5ae51b64cc5d7cd89a2aa35c4aa89f/test/instrumentation/modules/mysql2/mysql.test.js#L19-L25

However, this cannot work with mocha because it runs all test in process.

alternative attempted: load dep in try/catch

I made another attempt to load the devDep in a try/catch, then have each mocha test block this.skip() in beforeEach if the expected module vars weren't loaded.

Example diff
--- a/plugins/node/opentelemetry-instrumentation-aws-sdk/test/s3.test.ts
+++ b/plugins/node/opentelemetry-instrumentation-aws-sdk/test/s3.test.ts
@@ -22,7 +22,17 @@ import { AwsInstrumentation } from '../src';
 import { AttributeNames } from '../src/enums';
 registerInstrumentationTesting(new AwsInstrumentation());

-import { PutObjectCommand, S3Client } from '@aws-sdk/client-s3';
+// Skip tests, rather than crashing, if this import fails.
+let PutObjectCommand: unknown;
+let S3Client: unknown;
+try {
+  const mod = require('@aws-sdk/client-s3');
+  PutObjectCommand = mod.PutObjectCommand;
+  S3Client = mod.S3Client;
+} catch {
+  // Assuming the failure is due to this Node.js and aws-sdk being incompatible.
+}
+
 import * as fs from 'fs';
 import * as nock from 'nock';

@@ -33,6 +43,14 @@ import { expect } from 'expect';
 const region = 'us-east-1';

 describe('S3 - v3', () => {
+  beforeEach(function () {
+    if (!S3Client) {
+      // Skip because, presumably, the current version of node is too low to
+      // to support this version of `@aws-sdk/client-s3`.
+      this.skip();
+    }
+  });
+
   describe('PutObject', () => {
     it('Request span attributes - adds bucket Name', async () => {
       const dummyBucketName = 'ot-demo-test';

This is code-heavy because of the scoping of the loaded vars. Also potentially having to add the beforeEach() logic to multiple top-level describe()-blocks would be labourious.

The main hurdle that stopped me here was when types are imported from the instrumented module, e.g.:

Note that in the diff above I am using require() to load in a try/catch. Assuming I'm not missing something, to import a type and use it in a TS file, I'd need to use the import-statement, and this cannot be in a try/catch. So I'm stuck.

@trentm
Copy link
Contributor Author

trentm commented Feb 19, 2025

^ hup the pkg: label to get TAV tests to run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants