[BUG] Importing a schema object in script() lead to a 400 error

[waltoss]

The following GenAIScript configuration leads to a 400 Error when making a request to OpenAI:

import { dialog } from "./schema-magnolia.mts";

script({
  model: "openai:gpt-4o",
  system: ["system.schema"],
  responseSchema: dialog,
  responseType: "json_schema",
});

defSchema("DIALOG_SCHEMA", dialog);

$`
Generate a dialog for Magnolia CMS respecting DIALOG_SCHEMA
`;

The OpenAI API returns the following error:

- Invalid schema for response_format 'result': In context=('properties', '0'), 'default' is not permitted.

The reason is that responseSchema is equal to the string value "dialog" instead of the imported js object

This is due to parsePromptScriptMeta turning the code inside script({...}) into JSON

Do you have a lead to fix that ?

[github-actions]

{
  "feedback": "The issue appears to be due to how the `script()` function is handling the `responseSchema`. The current implementation seems to be converting the schema into a string rather than keeping it as an object. To resolve this, you may need to adjust the part of the code that parses and converts the prompt script metadata to ensure that the `responseSchema` is preserved as an object instead of a string. This will allow OpenAI to correctly interpret the schema when generating the response."
}

generated by issue-reviewer

[pelikhan]

Ha yes, this is a hack to speed up parsing of the script to extract the metadata to avoid loading and executing the script.

I will bring back running the script as a bailout scenario.

[pelikhan]

For now you have to inline it

[pelikhan]

Actually since it parsed it correctly, we should try to test the properties are a string (unresolved variables) and remove them. We don't need the schema at this point. I'll get a fix later.

[waltoss]

Yes I inlined it like this

import { dialog } from "./zod-magnolia.mts";
import { zodResponseFormat } from "openai/helpers/zod";

const schema = zodResponseFormat(dialog, "dialog").json_schema
  .schema as unknown as JSONSchemaObject;

const result = await runPrompt(
  (_) => {
    _.defSchema("DIALOG_SCHEMA", schema, { format: "json" });
    _.$`Generate a dialog for Magnolia CMS respecting DIALOG_SCHEMA. This dialog allows to add an article`;
  },
  {
    model: "openai:gpt-4o",
    responseType: "json_schema",
    responseSchema: schema,
  }
);

By the way, what do you think about a small helper around zod -> schema ?

[pelikhan]

Yes that would be great. Could be added to the already existing schema validation helpers.