[WIP] Work on using external modules #158

freiksenet · 2024-12-11T16:52:07Z

No description provided.

netlify · 2024-12-11T16:52:27Z

❌ Deploy Preview for grats failed.

Name	Link
🔨 Latest commit	`78cb2c2`
🔍 Latest deploy log	https://app.netlify.com/sites/grats/deploys/67c86c10a986950008762c0e

src/GraphQLAstExtensions.ts

captbaritone · 2024-12-11T19:24:00Z

TODO.md

+  - [ ] reenable global validations
+- [ ] "modular" mode? like no full schema, but parts of schema but with full validation by resolving it?
+- [ ] all tests to add fixtures for metadata/resolver map
+- [ ] pluggable module resolution - too many variables there, use filepath by default, let users customize it


In theory we only need this path in order to perform validation (is that right?). I wonder if there is a TypeScript API which will let us use the same (or most of) TypeScript's implementation of module resolution. I would need to avoid expecting the file to have a .js/.ts/.mjs/etc extension, but maybe something like that is exposed? I suspect that would be sufficient we could find something like that.

I did see this: https://github.com/microsoft/TypeScript/blob/6a00bd2422ffa46c13ac8ff81d7b4b1157e60ba7/src/server/project.ts#L484

Which could be a good place to start looking.

I'm not sure any non-ts files are included in how the typescript resolves it. There is also the whole story of "lib" in package.json and stuff like this. Ultimately there is no "well-known" spot for the GraphQL files, so it feels like it all might just break weirdly. I will experiment on this ofc, but I feel that there might not be an easy solution.

captbaritone · 2024-12-11T19:24:26Z

TODO.md

+- [ ] Read SDL to actually do validation
+  - [ ] reenable global validations
+- [ ] "modular" mode? like no full schema, but parts of schema but with full validation by resolving it?
+- [ ] all tests to add fixtures for metadata/resolver map


I'm onboard with this. Happy to have this as a separate PR if you want to merge that first.

captbaritone · 2024-12-11T19:26:06Z

TODO.md

+- [ ] all imported types (so support interfaces etc)
+- [ ] Read SDL to actually do validation
+  - [ ] reenable global validations
+- [ ] "modular" mode? like no full schema, but parts of schema but with full validation by resolving it?


Copy pasting from chat:

So I'm realizing it's not rational to expect Grats to own emitting the final GraphQLSchema object if you have external types. Those types will have resolvers that Grats can't know about. So, I think if you use @gqlExternal we must assume that Grats is going to be producing a schema slice, and the consumer will be responsible for stiching it togehter with the other schemas. That could be done with resolver maps or graphql-tools schema merge.

In short, I think as soon as you use @gqlExternal you are implicitly opting into modular mode. We will validate against the full schema, but only emit SDL/resolver maps/GraphQLSchema for the part of the schema that Grats owns.

I think I have a setup that should work - you can only have gqlExternal in resolver mode and it does full validation, but only generates for local types.

captbaritone · 2024-12-11T19:30:44Z

src/tests/fixtures/externals/nonGratsPackageWrapper.ts

+import {
+  SomeType as _SomeType,
+  SomeInterface as _SomeInterface,
+} from "./nonGratsPackage.ignore";


One thing I'd like to add eventually is the ability for a fixture file to model multiple files. I did something similar for Relay:

Example: https://github.com/facebook/relay/blob/main/compiler/crates/relay-compiler/tests/relay_compiler_integration/fixtures/client_mutation_extension.input
Implemented here: https://github.com/facebook/relay/blob/main/compiler/crates/graphql-test-helpers/src/project_fixture.rs

I think that would help for tests like this which need to encode relationships between different files.

Will work in a separate PR, together with test fixes.

…port

freiksenet · 2024-12-16T10:31:14Z

@captbaritone Could you approve the workflow for me?

captbaritone · 2024-12-16T14:50:13Z

@captbaritone Could you approve the workflow for me?

Done

captbaritone

Overall looks good! Left some observations/ideas inline, but the overall approach looks good!

captbaritone · 2024-12-16T20:59:59Z

src/Errors.ts

 }
+
+export function noModuleInGqlExternal() {
+  return `Grats: @gqlExternal must include a module name in double quotes. For example: /** @gqlExternal "myModule" */`;


Diagnostics do not need to be prefixed with Grats:

You do have it in tsConfigNotFound, is that a mistake too?

src/Errors.ts

src/Extractor.ts

src/transforms/addImportedSchemas.ts

src/utils/Result.ts

src/transforms/mergeExtensions.ts

src/transforms/addImportedSchemas.ts

captbaritone

Looking pretty good! Left some comments. What would you like to do next? Do you need this to merge to further validate, or should we try to work toward stabilizing it?

I think next steps for that would be:

Documentation (both for the tag and for the overall use case probably as an additional subpage of this section.
An example project in the examples directory. Could probably just copy/paste/tweak this one https://github.com/captbaritone/grats/tree/main/examples/incremental-migration

src/Errors.ts

src/Extractor.ts

src/lib.ts

src/transforms/addImportedSchemas.ts

src/utils/Result.ts

…port

captbaritone

I tried checking this out and trying to make use of it. Along the way I fell into a number of confusing edge cases which didn't have helpful error messages or validations. I've called them out in comments.

But, I was able to get it to work!

captbaritone · 2025-03-07T22:04:47Z

src/Extractor.ts

+    name: NameNode,
+    kind: NameDefinition["kind"],
+  ) {
+    const tag = this.findTag(node, EXTERNAL_TAG);


Nit: In most places we check for the tag before calling this. Can we pass the external tag as an argument here and avoid this error case?

captbaritone · 2025-03-07T22:06:51Z

src/tests/fixtures/externals/fundamentalErrors.ts.expected

+
+/**
+ * @gqlType MyType
+ * @gqlExternal


Should this error since there's no schema path specified? I don't see an error for this in the output below.

captbaritone · 2025-03-07T22:12:43Z

src/Extractor.ts

+      node.type.kind === ts.SyntaxKind.TypeReference &&
+      this.hasTag(node, EXTERNAL_TAG)


What is the user experience if they use @gqlExternal but it's not a type reference? Does it just silently ignore the @gqlExternal? If so, can we avoid that with a custom error message for that case?

captbaritone · 2025-03-07T22:13:07Z

src/Extractor.ts

      // case for operation types such as `Query`, `Mutation`, and `Subscription`
      // where there is not strong convention around.
+    } else if (
+      node.type.kind === ts.SyntaxKind.TypeReference &&


What is the user experience if they use @gqlExternal but it's not a type reference? Does it just silently ignore the @gqlExternal? If so, can we avoid that with a custom error message for that case?

captbaritone · 2025-03-07T22:13:52Z

src/Extractor.ts

+
+    if (
+      node.type.kind === ts.SyntaxKind.TypeReference &&
+      this.hasTag(node, EXTERNAL_TAG)


Same as above. I think we need to handle the case where the user uses @gqlExternal on a non-type reference

captbaritone · 2025-03-07T22:15:19Z

src/transforms/addImportedSchemas.ts

+  const externalDocs: DocumentNode[] = [];
+  const errors: ts.Diagnostic[] = [];
+  for (const schemaPath of importedSchemas) {
+    const text = fs.readFileSync(path.resolve(schemaPath), "utf-8");


Can we handle the case where this path does not exist? Right now it just crashes Grats.

/Users/captbaritone/projects/grats/dist/src/transforms/addImportedSchemas.js:81 finally { if (e_2) throw e_2.error; } ^ Error: ENOENT: no such file or directory, open '/Users/captbaritone/projects/grats/examples/production-app/models/SomeExternalThing' at Object.readFileSync (node:fs:453:20) at addImportedSchemas (/Users/captbaritone/projects/grats/dist/src/transforms/addImportedSchemas.js:54:27) at /Users/captbaritone/projects/grats/dist/src/lib.js:130:90 at ResultPipe.andThen (/Users/captbaritone/projects/grats/dist/src/utils/Result.js:78:35) at /Users/captbaritone/projects/grats/dist/src/lib.js:130:14 at ResultPipe.andThen (/Users/captbaritone/projects/grats/dist/src/utils/Result.js:78:35) at extractSchemaAndDoc (/Users/captbaritone/projects/grats/dist/src/lib.js:93:10) at watchHost.afterProgramCreate (/Users/captbaritone/projects/grats/dist/src/cli.js:97:58) at synchronizeProgram (/Users/captbaritone/projects/grats/node_modules/.pnpm/[email protected]/node_modules/typescript/lib/typescript.js:131052:12) at updateProgram (/Users/captbaritone/projects/grats/node_modules/.pnpm/[email protected]/node_modules/typescript/lib/typescript.js:131244:9) { errno: -2, code: 'ENOENT', syscall: 'open', path: '/Users/captbaritone/projects/grats/examples/production-app/models/SomeExternalThing' } Node.js v20.10.0 ELIFECYCLE Command failed with exit code 1. ```

Whatever error we show here would be a great palace to add context about how path resolution is done. So if they defined the right file but used the wrong relative path, we can help them fix it.

Also interesting: This dependency on non-ts files is problematic for watch mode. Ideally we would rerun Grats if any of these external GraphQL files changed.

captbaritone · 2025-03-07T22:25:17Z

src/tests/fixtures/externals/union.ts

+import { SomeUnion as _SomeUnion } from "./nonGratsPackage.ignore";
+
+/**
+ * @gqlType MyUnion


Shouldn't this need to be @gqlUnion?

In general, it feels like a bug if I define a type here and then get an error "UnknownType MyUnion". I think we need to find a way to validate that the types marked as being external are actually defined in the referenced external schema and match the type kind mentioned here.

Maybe we can add an additional validation pass to do that?

captbaritone · 2025-03-07T22:51:00Z

src/lib.ts

+        .map((doc) => mergeExtensions(ctx, doc))
        // Perform custom validations that reimplement spec validation rules
        // with more tailored error messages.
        .andThen((doc) => customSpecValidations(doc))


I've noticed that this will fail with external types since it notices the type has no fields. Can we move this below addImportedSchemas?

Hmm, actually even disabling this seems to trigger validation errors on line 135 saying the external type has no fields.

WIP

a995481

Fix todo

69bba95

freiksenet commented Dec 11, 2024

View reviewed changes

src/GraphQLAstExtensions.ts Outdated Show resolved Hide resolved

captbaritone reviewed Dec 11, 2024

View reviewed changes

freiksenet added 2 commits December 16, 2024 11:03

Merge remote-tracking branch 'origin/main' into freiksenet/import-sup…

8a1fa3b

…port

Actually usable stuff

a431599

freiksenet added 3 commits December 16, 2024 15:59

Support all types and add tests to them

d9acd95

Add async result pipe

08a8573

Whitespace issues

da25562

captbaritone reviewed Dec 16, 2024

View reviewed changes

freiksenet added 3 commits December 17, 2024 12:10

Add test for empty type

f312118

Minor review comments

58fedca

Cleaner externalModule

fda55a0

captbaritone reviewed Jan 2, 2025

View reviewed changes

freiksenet added 3 commits March 4, 2025 12:24

Merge remote-tracking branch 'origin/main' into freiksenet/import-sup…

1b801c5

…port

Fix tests

f889bfd

Review comments

78cb2c2

captbaritone requested changes Mar 7, 2025

View reviewed changes

captbaritone reviewed Mar 7, 2025

View reviewed changes

		node.type.kind === ts.SyntaxKind.TypeReference &&
		this.hasTag(node, EXTERNAL_TAG)

[WIP] Work on using external modules #158

Are you sure you want to change the base?

[WIP] Work on using external modules #158

Uh oh!

Conversation

freiksenet commented Dec 11, 2024

Uh oh!

netlify bot commented Dec 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ Deploy Preview for grats failed.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

freiksenet commented Dec 16, 2024

Uh oh!

captbaritone commented Dec 16, 2024

Uh oh!

captbaritone left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

captbaritone left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

captbaritone left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

netlify bot commented Dec 11, 2024 •

edited

Loading