Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce experimental FileOutput interface for models that output File and Path types #305

Merged
merged 5 commits into from
Sep 12, 2024

Conversation

aron
Copy link
Contributor

@aron aron commented Sep 11, 2024

This PR is a proposal to add a new FileOutput type to the client SDK to abstract away file outputs from Replicate models.

It can be enabled by passing the useFileOutput flag to the Replicate constructor.

const replicate = new Replicate({ useFileOutput: true });

When enabled any URLs or data-uris will be converted into a FileOutput type. This is essentially a ReadableStream that has two additional methods url() to return the underlying URL and blob() which will return a Blob() instance with the file data loaded into memory.

The intention here is to make it easier to work with file outputs and allows us to optimize the delivery of file assets to the client in future iterations.

Usage is as follows:

const [output] = await replicate.run("black-forest-labs/flux-schnell", { 
	  input: {
	    prompt: "astronaut riding a rocket like a horse"
	  }
});

For most basic cases you'll want to utilize either the url() or blob() methods depending on whether you want to directly consume the file or pass it on.

To access the file URL:

console.log(output.url()) // "https://delivery.replicate.com/..."

To consume the file directly:

// Assign an image directly to an image tag.
img.src = URL.createObjectURL(await output.blob());

Or for very large files they can be streamed:

for await (const chunk of output) {
  console.log(chunk); // UInt8Array
}

Or passed on to methods that support ReadableStream.

fs.writeFile("my-image.png", output);

An effort has been made to maintain backwards compatibility by implementing a toString() method which should cause the FileObject to act like a URL string.

const data = await fetch(output).then(r => r.blob()); // Blob

This takes a url and creates a `ReadableStream` that has two additional
methods `url()` and `blob()` for easily working with remote URLs or
base64 encoded data.
This is currently behind the `useFileOutput` flag provided to the
Replicate constructor. This allows us to test the feature before rolling
it out more widely.

When enabled any URLs or data-uris will be converted into a FileOutput
type. This is essentially a `ReadableStream` that has two additional
methods `url()` to return the underlying URL and `blob()` which will
return a `Blob()` object with the file data loaded into memory.

The intention here is to make it easier to work with file outputs and
allows us to optimize the delivery of file assets to the client in
future iterations.
@aron aron changed the title file output Introduce experimental FileOutput interface for models that output File and Path types Sep 11, 2024
Copy link
Contributor

@mattt mattt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this. Really nice work, @aron. No comments and changes other than to add some doc comments. Quite happy with this approach.

@aron
Copy link
Contributor Author

aron commented Sep 12, 2024

@mattt thoughts on transforming prediction.output from all API responses when this is enabled, rather than just the output of replicate.run()? I think that would be better for consistency.

@aron aron merged commit 445f2ab into main Sep 12, 2024
19 checks passed
@aron aron deleted the file-output branch September 12, 2024 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants