Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Imagen Documentation #8776

Merged
merged 15 commits into from
Feb 20, 2025
Merged

Imagen Documentation #8776

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
54cf6d5
Revert "remove public docs"
dlarocque Feb 7, 2025
95c7093
update docs
dlarocque Feb 7, 2025
225e8b0
update docs
dlarocque Feb 7, 2025
3da6b4b
format
dlarocque Feb 7, 2025
3277ceb
minor fixes
dlarocque Feb 10, 2025
a9c70b8
format
dlarocque Feb 10, 2025
1320f62
mark all APIs beta instead of public
dlarocque Feb 12, 2025
50a1f5c
review fixes
dlarocque Feb 12, 2025
431ea66
format
dlarocque Feb 12, 2025
504a1e2
Merge branch 'dl/vertex-imagen' into dl/imagen-documentation
dlarocque Feb 13, 2025
ade589d
Replace (BETA) with (Public Preview) in docs
dlarocque Feb 13, 2025
718c7e2
fix links
dlarocque Feb 13, 2025
e05b8d3
another pass
dlarocque Feb 14, 2025
44ab79d
format
dlarocque Feb 14, 2025
60bcae9
review fixes
dlarocque Feb 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 14 additions & 50 deletions common/api-review/vertexai.api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ export class GenerativeModel extends VertexAIModel {
// @public
export function getGenerativeModel(vertexAI: VertexAI, modelParams: ModelParams, requestOptions?: RequestOptions): GenerativeModel;

// @public (undocumented)
// @beta
export function getImagenModel(vertexAI: VertexAI, modelParams: ImagenModelParams, requestOptions?: RequestOptions): ImagenModel;

// @public
Expand Down Expand Up @@ -430,122 +430,87 @@ export enum HarmSeverity {
HARM_SEVERITY_NEGLIGIBLE = "HARM_SEVERITY_NEGLIGIBLE"
}

// @public (undocumented)
// @beta
export enum ImagenAspectRatio {
// (undocumented)
LANDSCAPE_16x9 = "16:9",
// (undocumented)
LANDSCAPE_3x4 = "3:4",
// (undocumented)
PORTRAIT_4x3 = "4:3",
// (undocumented)
PORTRAIT_9x16 = "9:16",
// (undocumented)
SQUARE = "1:1"
}

// @public (undocumented)
// @public
export interface ImagenGCSImage {
// (undocumented)
gcsURI: string;
// (undocumented)
mimeType: string;
}

// @public (undocumented)
// @beta
export interface ImagenGenerationConfig {
// (undocumented)
addWatermark?: boolean;
// (undocumented)
aspectRatio?: ImagenAspectRatio;
// (undocumented)
imageFormat?: ImagenImageFormat;
// (undocumented)
negativePrompt?: string;
// (undocumented)
numberOfImages?: number;
}

// @public (undocumented)
// @beta
export interface ImagenGenerationResponse<T extends ImagenInlineImage | ImagenGCSImage> {
// (undocumented)
filteredReason?: string;
// (undocumented)
images: T[];
}

// @public
// @beta
export class ImagenImageFormat {
// (undocumented)
compressionQuality?: number;
// (undocumented)
static jpeg(compressionQuality?: number): ImagenImageFormat;
// (undocumented)
mimeType: string;
// (undocumented)
static png(): ImagenImageFormat;
}

// @public
// @beta
export interface ImagenInlineImage {
// (undocumented)
bytesBase64Encoded: string;
// (undocumented)
mimeType: string;
}

// @public (undocumented)
// @beta
export class ImagenModel extends VertexAIModel {
constructor(vertexAI: VertexAI, modelParams: ImagenModelParams, requestOptions?: RequestOptions | undefined);
// (undocumented)
generateImages(prompt: string): Promise<ImagenGenerationResponse<ImagenInlineImage>>;
// (undocumented)
// @internal
generateImagesGCS(prompt: string, gcsURI: string): Promise<ImagenGenerationResponse<ImagenGCSImage>>;
// (undocumented)
generationConfig?: ImagenGenerationConfig;
// (undocumented)
requestOptions?: RequestOptions | undefined;
// (undocumented)
safetySettings?: ImagenSafetySettings;
}

// @public (undocumented)
// @beta
export interface ImagenModelParams {
// (undocumented)
generationConfig?: ImagenGenerationConfig;
// (undocumented)
model: string;
// (undocumented)
safetySettings?: ImagenSafetySettings;
}

// @public (undocumented)
// @beta
export enum ImagenPersonFilterLevel {
// (undocumented)
ALLOW_ADULT = "allow_adult",
// (undocumented)
ALLOW_ALL = "allow_all",
// (undocumented)
BLOCK_ALL = "dont_allow"
}

// @public (undocumented)
// @beta
export enum ImagenSafetyFilterLevel {
// (undocumented)
BLOCK_LOW_AND_ABOVE = "block_low_and_above",
// (undocumented)
BLOCK_MEDIUM_AND_ABOVE = "block_medium_and_above",
// (undocumented)
BLOCK_NONE = "block_none",
// (undocumented)
BLOCK_ONLY_HIGH = "block_only_high"
}

// @public (undocumented)
// @beta
export interface ImagenSafetySettings {
// (undocumented)
personFilterLevel?: ImagenPersonFilterLevel;
// (undocumented)
safetyFilterLevel?: ImagenSafetyFilterLevel;
}

Expand Down Expand Up @@ -838,13 +803,12 @@ export const enum VertexAIErrorCode {
RESPONSE_ERROR = "response-error"
}

// @public (undocumented)
// @public
export abstract class VertexAIModel {
// @internal
protected constructor(vertexAI: VertexAI, modelName: string);
// @internal (undocumented)
protected _apiSettings: ApiSettings;
// (undocumented)
readonly model: string;
static normalizeModelName(modelName: string): string;
}
Expand Down
18 changes: 16 additions & 2 deletions docs-devsite/vertexai.imagengcsimage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ https://github.com/firebase/firebase-js-sdk
{% endcomment %}

# ImagenGCSImage interface
An image generated by Imagen, stored in a Cloud Storage for Firebase bucket.

This feature is not available yet.

<b>Signature:</b>

```typescript
Expand All @@ -20,19 +24,29 @@ export interface ImagenGCSImage

| Property | Type | Description |
| --- | --- | --- |
| [gcsURI](./vertexai.imagengcsimage.md#imagengcsimagegcsuri) | string | |
| [mimeType](./vertexai.imagengcsimage.md#imagengcsimagemimetype) | string | |
| [gcsURI](./vertexai.imagengcsimage.md#imagengcsimagegcsuri) | string | The URI of the file stored in a Cloud Storage for Firebase bucket. |
| [mimeType](./vertexai.imagengcsimage.md#imagengcsimagemimetype) | string | The MIME type of the image; either <code>&quot;image/png&quot;</code> or <code>&quot;image/jpeg&quot;</code>.<!-- -->To request a different format, set the <code>imageFormat</code> property in your <code>[ImagenGenerationConfig](./vertexai.imagengenerationconfig.md#imagengenerationconfig_interface)</code>. |

## ImagenGCSImage.gcsURI

The URI of the file stored in a Cloud Storage for Firebase bucket.

<b>Signature:</b>

```typescript
gcsURI: string;
```

### Example

`"gs://bucket-name/path/sample_0.jpg"`<!-- -->.

## ImagenGCSImage.mimeType

The MIME type of the image; either `"image/png"` or `"image/jpeg"`<!-- -->.

To request a different format, set the `imageFormat` property in your <code>[ImagenGenerationConfig](./vertexai.imagengenerationconfig.md#imagengenerationconfig_interface)</code>.

<b>Signature:</b>

```typescript
Expand Down
54 changes: 49 additions & 5 deletions docs-devsite/vertexai.imagengenerationconfig.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@ https://github.com/firebase/firebase-js-sdk
{% endcomment %}

# ImagenGenerationConfig interface
> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Configuration options for generating images with Imagen.

See the [documentation](http://firebase.google.com/docs/vertex-ai/generate-images-imagen) for more details.

<b>Signature:</b>

```typescript
Expand All @@ -20,14 +27,23 @@ export interface ImagenGenerationConfig

| Property | Type | Description |
| --- | --- | --- |
| [addWatermark](./vertexai.imagengenerationconfig.md#imagengenerationconfigaddwatermark) | boolean | |
| [aspectRatio](./vertexai.imagengenerationconfig.md#imagengenerationconfigaspectratio) | [ImagenAspectRatio](./vertexai.md#imagenaspectratio) | |
| [imageFormat](./vertexai.imagengenerationconfig.md#imagengenerationconfigimageformat) | [ImagenImageFormat](./vertexai.imagenimageformat.md#imagenimageformat_class) | |
| [negativePrompt](./vertexai.imagengenerationconfig.md#imagengenerationconfignegativeprompt) | string | |
| [numberOfImages](./vertexai.imagengenerationconfig.md#imagengenerationconfignumberofimages) | number | |
| [addWatermark](./vertexai.imagengenerationconfig.md#imagengenerationconfigaddwatermark) | boolean | <b><i>(Public Preview)</i></b> Whether to add an invisible watermark to generated images.<!-- -->If set to <code>true</code>, an invisible SynthID watermark is embedded in generated images to indicate that they are AI generated. If set to <code>false</code>, watermarking will be disabled.<!-- -->For Imagen 3 models, the default value is <code>true</code>; see the <a href="http://firebase.google.com/docs/vertex-ai/model-parameters#imagen"><code>addWatermark</code></a> documentation for more details. |
| [aspectRatio](./vertexai.imagengenerationconfig.md#imagengenerationconfigaspectratio) | [ImagenAspectRatio](./vertexai.md#imagenaspectratio) | <b><i>(Public Preview)</i></b> The aspect ratio of the generated images. The default value is square 1:1. Supported aspect ratios depend on the Imagen model, see <code>[ImagenAspectRatio](./vertexai.md#imagenaspectratio)</code> for more details. |
| [imageFormat](./vertexai.imagengenerationconfig.md#imagengenerationconfigimageformat) | [ImagenImageFormat](./vertexai.imagenimageformat.md#imagenimageformat_class) | <b><i>(Public Preview)</i></b> The image format of the generated images. The default is PNG.<!-- -->See <code>[ImagenImageFormat](./vertexai.imagenimageformat.md#imagenimageformat_class)</code> for more details. |
| [negativePrompt](./vertexai.imagengenerationconfig.md#imagengenerationconfignegativeprompt) | string | <b><i>(Public Preview)</i></b> A description of what should be omitted from the generated images.<!-- -->Support for negative prompts depends on the Imagen model.<!-- -->See the [documentation](http://firebase.google.com/docs/vertex-ai/model-parameters#imagen) for more details. |
| [numberOfImages](./vertexai.imagengenerationconfig.md#imagengenerationconfignumberofimages) | number | <b><i>(Public Preview)</i></b> The number of images to generate. The default value is 1.<!-- -->The number of sample images that may be generated in each request depends on the model (typically up to 4); see the <a href="http://firebase.google.com/docs/vertex-ai/model-parameters#imagen"><code>sampleCount</code></a> documentation for more details. |

## ImagenGenerationConfig.addWatermark

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

Whether to add an invisible watermark to generated images.

If set to `true`<!-- -->, an invisible SynthID watermark is embedded in generated images to indicate that they are AI generated. If set to `false`<!-- -->, watermarking will be disabled.

For Imagen 3 models, the default value is `true`<!-- -->; see the <a href="http://firebase.google.com/docs/vertex-ai/model-parameters#imagen"><code>addWatermark</code></a> documentation for more details.

<b>Signature:</b>

```typescript
Expand All @@ -36,6 +52,11 @@ addWatermark?: boolean;

## ImagenGenerationConfig.aspectRatio

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

The aspect ratio of the generated images. The default value is square 1:1. Supported aspect ratios depend on the Imagen model, see <code>[ImagenAspectRatio](./vertexai.md#imagenaspectratio)</code> for more details.

<b>Signature:</b>

```typescript
Expand All @@ -44,6 +65,13 @@ aspectRatio?: ImagenAspectRatio;

## ImagenGenerationConfig.imageFormat

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

The image format of the generated images. The default is PNG.

See <code>[ImagenImageFormat](./vertexai.imagenimageformat.md#imagenimageformat_class)</code> for more details.

<b>Signature:</b>

```typescript
Expand All @@ -52,6 +80,15 @@ imageFormat?: ImagenImageFormat;

## ImagenGenerationConfig.negativePrompt

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

A description of what should be omitted from the generated images.

Support for negative prompts depends on the Imagen model.

See the [documentation](http://firebase.google.com/docs/vertex-ai/model-parameters#imagen) for more details.

<b>Signature:</b>

```typescript
Expand All @@ -60,6 +97,13 @@ negativePrompt?: string;

## ImagenGenerationConfig.numberOfImages

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

The number of images to generate. The default value is 1.

The number of sample images that may be generated in each request depends on the model (typically up to 4); see the <a href="http://firebase.google.com/docs/vertex-ai/model-parameters#imagen"><code>sampleCount</code></a> documentation for more details.

<b>Signature:</b>

```typescript
Expand Down
25 changes: 23 additions & 2 deletions docs-devsite/vertexai.imagengenerationresponse.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ https://github.com/firebase/firebase-js-sdk
{% endcomment %}

# ImagenGenerationResponse interface
> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

The response from a request to generate images with Imagen.

<b>Signature:</b>

```typescript
Expand All @@ -20,11 +25,20 @@ export interface ImagenGenerationResponse<T extends ImagenInlineImage | ImagenGC

| Property | Type | Description |
| --- | --- | --- |
| [filteredReason](./vertexai.imagengenerationresponse.md#imagengenerationresponsefilteredreason) | string | |
| [images](./vertexai.imagengenerationresponse.md#imagengenerationresponseimages) | T\[\] | |
| [filteredReason](./vertexai.imagengenerationresponse.md#imagengenerationresponsefilteredreason) | string | <b><i>(Public Preview)</i></b> The reason that images were filtered out. This property will only be defined if one or more images were filtered.<!-- -->Images may be filtered out due to the <code>[ImagenSafetyFilterLevel](./vertexai.md#imagensafetyfilterlevel)</code>, <code>[ImagenPersonFilterLevel](./vertexai.md#imagenpersonfilterlevel)</code>, or filtering included in the model. The filter levels may be adjusted in your <code>[ImagenSafetySettings](./vertexai.imagensafetysettings.md#imagensafetysettings_interface)</code>.<!-- -->See the [Responsible AI and usage guidelines for Imagen](https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen) for more details. |
| [images](./vertexai.imagengenerationresponse.md#imagengenerationresponseimages) | T\[\] | <b><i>(Public Preview)</i></b> The images generated by Imagen.<!-- -->The number of images generated may be fewer than the number requested if one or more were filtered out; see <code>filteredReason</code>. |

## ImagenGenerationResponse.filteredReason

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

The reason that images were filtered out. This property will only be defined if one or more images were filtered.

Images may be filtered out due to the <code>[ImagenSafetyFilterLevel](./vertexai.md#imagensafetyfilterlevel)</code>, <code>[ImagenPersonFilterLevel](./vertexai.md#imagenpersonfilterlevel)</code>, or filtering included in the model. The filter levels may be adjusted in your <code>[ImagenSafetySettings](./vertexai.imagensafetysettings.md#imagensafetysettings_interface)</code>.

See the [Responsible AI and usage guidelines for Imagen](https://cloud.google.com/vertex-ai/generative-ai/docs/image/responsible-ai-imagen) for more details.

<b>Signature:</b>

```typescript
Expand All @@ -33,6 +47,13 @@ filteredReason?: string;

## ImagenGenerationResponse.images

> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
>

The images generated by Imagen.

The number of images generated may be fewer than the number requested if one or more were filtered out; see `filteredReason`<!-- -->.

<b>Signature:</b>

```typescript
Expand Down
Loading
Loading