Merge pull request #2825 from weaviate/academy/ts-multimodal

Academy/ts multimodal

developers/academy/js/_snippets/intro_next_steps_js.mdx

@@ -0,0 +1,32 @@
+Congratulations! You have completed this introductory course on Weaviate.
+
+Now that you have completed this course, you may be interested in exploring our documentation or the Academy for more advanced courses.
+
+Some of our more popular resources include:
+
+### Documentation
+
+- How-to guides
+    - [How-to: manage data](/developers/weaviate/manage-data/index.md): Code examples for *create*, *update* and *delete* operations.
+    - [How-to: search](/developers/weaviate/search/index.md): Code examples for all types of search operations.
+    - [How-to: configure Weaviate](/developers/weaviate/configuration/index.md): Guides for configuring Weaviate, such as [PQ](/developers/weaviate/configuration/compression/pq-compression.md) and [BQ](/developers/weaviate/configuration/compression/bq-compression.md) compression, [backups](/developers/weaviate/configuration/backups.md) and [replication](/developers/weaviate/configuration/replication.md).
+- [Concepts guides](/developers/weaviate/concepts/index.md): Guides for understanding Weaviate's architecture and concepts.
+- [API reference](/developers/weaviate/api/index.md): Detailed information about Weaviate's APIs.
+
+### Academy
+
+- [Working with Text](/developers/academy/js/starter_text_data): Learn how to use work with text data in Weaviate.
+- [Which search is right for me?](/developers/academy/js/standalone/which_search): Learn about the different types of searches in Weaviate and when to use them.
+
+
+import CTASocials from './cta_socials.mdx';
+
+<CTASocials />
+
+See you soon! 👋
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>

developers/academy/js/starter_multimodal_data/101_setup_weaviate/10_client.mdx

@@ -0,0 +1,37 @@
+---
+title: Weaviate Typescript client
+description: Client Setup for Multimodal Data in Weaviate
+---
+
+## <i class="fa-solid fa-code"></i> Installation
+
+The latest Weaviate TypeScript client library can be installed using npm. The client library is tested on Node v18 and later. Install it using the following command:
+
+```bash
+npm install weaviate-client
+```
+
+The latest major version is `v3` (e.g. `3.x.x`). You can check the version like so:
+
+```bash
+npm view weaviate-client version
+```
+
+## <i class="fa-solid fa-code"></i> Basic usage
+
+You can import the Weaviate client library like so:
+
+```typescript
+import weaviate, { generateUuid5, ApiKey } from "weaviate-client"
+```
+
+The client provides sets of helper functions (e.g. `generateUuid5, ApiKey`) to make it easier to interact with Weaviate.
+
+Next, we'll show you how create a Weaviate instance and connect to it.
+
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>

developers/academy/js/starter_multimodal_data/101_setup_weaviate/20_create_instance/10_create_wcs.mdx

@@ -0,0 +1,67 @@
+---
+title: "Option 1: A cloud WCD instance"
+description: "Create a Weaviate instance on WCS for scalable, cloud-based data projects."
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+import FilteredTextBlock from '@site/src/components/Documentation/FilteredTextBlock';
+import TSCode from '!!raw-loader!../../_snippets/101_connect.mts';
+
+Here, you will create a Weaviate Cloud (WCD) instance. WCD is a fully managed Weaviate instance that runs in the cloud. It's a great way to get started with Weaviate, as it requires no installation or maintenance.
+
+### <i class="fa-solid fa-chalkboard"></i> Log in to the WCD Console
+
+Go to the [WCD Console](https://console.weaviate.cloud/) and log in with your credentials. If you don't have an account yet, you can sign up by clicking on the <kbd>Register here</kbd> link from the login screen.
+
+### <i class="fa-solid fa-chalkboard"></i> Create a Weaviate instance
+
+From the console, go to the Dashboard and click on the <kbd>Create cluster</kbd> button. From the following screen:
+
+- Select the "Free sandbox" tab
+- Provide a cluster name
+- Set "Enable authentication" to "Yes"
+
+Click on the <kbd>Create</kbd> button to create your Weaviate instance. The process will take a few minutes.
+
+### <i class="fa-solid fa-chalkboard"></i> Retrieve your Weaviate instance details
+
+Once the instance is created, you will be able see its details by clicking on the <kbd>Details</kbd> button. Find the cluster URL and the API key.
+
+You will need these details to connect to your Weaviate instance.
+
+### <i class="fa-solid fa-code"></i> Connect to your WCD instance
+
+To connect to the Weaviate Cloud (WCD) instance, you need to use the cluster URL and the API key. You can find these details in the WCD Console.
+
+Use the `connectToWeaviateCloud()` function to connect to your WCD instance.
+
+<FilteredTextBlock
+  text={TSCode}
+  startMarker="// WCDInstantiation"
+  endMarker="// END WCDInstantiation"
+  language="ts"
+/>
+
+#### Provide inference API keys
+
+Some Weaviate modules can use inference APIs for vectorizing data or large language model integration. You can provide the API keys for these services to Weaviate at instantiation.
+
+This course uses VoyageAI, so you can provide the VoyageAI API key to Weaviate through `headers: {"X-VoyageAI-Api-Key": <YOUR_KEY>}` as shown below:
+
+<FilteredTextBlock
+  text={TSCode}
+  startMarker="// WCDAPIKeyInstantiation"
+  endMarker="// END WCDAPIKeyInstantiation"
+  language="ts"
+/>
+
+:::note What next?
+If you have completed this, you can skip the next page [Option 2: A local Weaviate instance](./20_create_docker.mdx) and continue with [Communicate with Weaviate](../30_communicate.mdx).
+:::
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>

developers/academy/js/starter_multimodal_data/101_setup_weaviate/20_create_instance/20_create_docker.mdx

@@ -0,0 +1,93 @@
+---
+title: "Option 2: A local Docker instance"
+description: "Set up Weaviate with Docker for a quick and customizable local deployment."
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+import FilteredTextBlock from '@site/src/components/Documentation/FilteredTextBlock';
+import TSCode from '!!raw-loader!../../_snippets/101_connect.mts';
+
+:::note Have you already created a Weaviate instance?
+If you have [created a cloud instance](./10_create_wcs.mdx) of Weaviate, you can skip this page and continue with [Communicate with Weaviate](../30_communicate.mdx).
+:::
+
+Here, you will create a Weaviate instance using Docker.
+
+### <i class="fa-solid fa-chalkboard"></i> Download and run the docker-compose file
+
+Install Docker on your machine. We recommend following the [official Docker installation guide](https://docs.docker.com/get-docker/).
+
+Create a new directory and navigate to it in your terminal. Then, create a new file called `docker-compose.yml` and add the following content:
+
+```yaml
+---
+version: '3.4'
+services:
+  weaviate_anon:
+    command:
+    - --host
+    - 0.0.0.0
+    - --port
+    - '8080'
+    - --scheme
+    - http
+    image: cr.weaviate.io/semitechnologies/weaviate:||site.weaviate_version||
+    ports:
+    - 8080:8080
+    - 50051:50051
+    restart: on-failure:0
+    environment:
+      COHERE_APIKEY: $COHERE_APIKEY
+      VOYAGEAI_APIKEY: $VOYAGEAI_APIKEY
+      QUERY_DEFAULTS_LIMIT: 25
+      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
+      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
+      DEFAULT_VECTORIZER_MODULE: 'none'
+      ENABLE_MODULES: 'text2vec-cohere,multi2vec-voyageai,generative-cohere'
+      BACKUP_FILESYSTEM_PATH: '/var/lib/weaviate/backups'
+      CLUSTER_HOSTNAME: 'node1'
+...
+```
+
+### <i class="fa-solid fa-chalkboard"></i> Create a Weaviate instance
+
+Run the following command to start Weaviate:
+
+```bash
+docker compose up -d
+```
+
+### <i class="fa-solid fa-chalkboard"></i> Your Weaviate instance details
+
+Once the instance is created, you can access it at `http://localhost:8080`.
+
+### <i class="fa-solid fa-code"></i> Connect to your Weaviate instance
+
+To connect to the Weaviate instance, use the `connectToLocal()` function.
+
+<FilteredTextBlock
+  text={TSCode}
+  startMarker="// DockerInstantiation"
+  endMarker="// END DockerInstantiation"
+  language="ts"
+/>
+
+#### Provide inference API keys
+
+Some Weaviate modules can use inference APIs for vectorizing data or large language model integration. You can provide the API keys for these services to Weaviate at instantiation.
+
+This course uses VoyageAI, so you can provide the VoyageAI API key to Weaviate through `headers: {"X-VoyageAI-Api-Key": <YOUR_KEY>}` as shown below:
+
+<FilteredTextBlock
+  text={TSCode}
+  startMarker="// DockerAPIKeyInstantiation"
+  endMarker="// END DockerAPIKeyInstantiation"
+  language="ts"
+/>
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>

developers/academy/js/starter_multimodal_data/101_setup_weaviate/20_create_instance/index.mdx

@@ -0,0 +1,19 @@
+---
+title: Create a Weaviate instance
+description: "Create a new Weaviate instance tailored for text data applications."
+---
+
+For this unit, you can choose to create a Weaviate Cloud (WCD) instance or a local Docker instance.
+
+- [Create a Weaviate Cloud (WCD) instance](./10_create_wcs.mdx)
+    - If you want a managed service and don't want to worry about installation and maintenance.
+- [Create a local Docker instance](./20_create_docker.mdx)
+    - If you want to run Weaviate on your local machine, or want to have full control over the installation and maintenance.
+
+Either option is fine for this course. If you're not sure which to choose, we recommend starting with a WCD instance.
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>

developers/academy/js/starter_multimodal_data/101_setup_weaviate/30_communicate.mdx

@@ -0,0 +1,64 @@
+---
+title: Communicate with Weaviate
+description: Communication Setup for Multimodal Data
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+import FilteredTextBlock from '@site/src/components/Documentation/FilteredTextBlock';
+import TSCode from '!!raw-loader!../_snippets/101_connect.mts';
+
+Here, we'll perform basic operations to communicate with Weaviate using the TypeScript client library.
+
+### <i class="fa-solid fa-code"></i> Check Weaviate status
+
+You can check whether the Weaviate instance is up using the `isLive` function.
+
+<FilteredTextBlock
+  text={TSCode}
+  startMarker="// PollLiveness"
+  endMarker="// END PollLiveness"
+  language="ts"
+/>
+
+### <i class="fa-solid fa-code"></i> Retrieve server meta information
+
+You can retrieve meta information about the Weaviate instance using the `getMeta` function.
+
+<FilteredTextBlock
+  text={TSCode}
+  startMarker="// GetMeta"
+  endMarker="// END GetMeta"
+  language="ts"
+/>
+
+This will print the server meta information to the console. The output will look similar to the following:
+
+<details>
+  <summary>Example <code>getMeta()</code> output</summary>
+
+<FilteredTextBlock
+  text={TSCode}
+  startMarker="// OutputGetMeta"
+  endMarker="// END OutputGetMeta"
+  language="ts"
+/>
+</details>
+
+### <i class="fa-solid fa-code"></i> Close the connection
+
+After you have finished using the Weaviate client, you should close the connection. This frees up resources and ensures that the connection is properly closed.
+
+We suggest using a `try`-`finally` block as a best practice. For brevity, we will not include the `try`-`finally` blocks in the remaining code snippets.
+
+<FilteredTextBlock
+  text={TSCode}
+  startMarker="// TryFinallyCloseDemo"
+  endMarker="// END TryFinallyCloseDemo"
+  language="ts"
+/>
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>

developers/academy/js/starter_multimodal_data/101_setup_weaviate/index.mdx

@@ -0,0 +1,26 @@
+---
+title: Set up Weaviate
+description: Weaviate Setup for Multimodal Data
+---
+<!-- Like a subject number (e.g. CS101) -->
+
+<!-- ## Overview -->
+
+<!-- Provide context for this course, in addition to the concrete learning goals and outcomes. Why would someone want to do this unit? -->
+
+<!-- :::warning TODO
+Intro video here
+::: -->
+
+## <i class="fa-solid fa-bullseye-arrow"></i> Learning objectives
+
+import LearningGoals from '/src/components/Academy/learningGoals.jsx';
+
+<!-- Replace unitName with name from `unitData.js` - will pull in learning goals and outcomes -->
+<LearningGoals unitName="docker_mm_basics_ts"/>
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>

developers/academy/js/starter_multimodal_data/102_mm_collections/10_preparation.mdx

@@ -0,0 +1,36 @@
+---
+title: Preparation
+---
+
+In this section you are going to populate your Weaviate instance with a movie dataset, using the multi-modal, Cohere Embed 3 models to embed the text and image data.
+
+### <i class="fa-solid fa-chalkboard"></i> Weaviate instance
+
+Make sure to have your Weaviate instance set up. You should have [created an instance](../101_setup_weaviate/20_create_instance/index.mdx) and be able to connect to it.
+
+### <i class="fa-solid fa-code"></i> Source data
+
+We are going to use a movie dataset sourced from [TMDB](https://www.themoviedb.org/). The dataset can be found in this [GitHub repository](https://raw.githubusercontent.com/weaviate-tutorials/edu-datasets/main/movies_data_1990_2024.json), and it contains bibliographic information on ~700 movies released between 1990 and 2024.
+
+As a multimodal project, we'll also use [corresponding posters for each movie](https://raw.githubusercontent.com/weaviate-tutorials/edu-datasets/main/movies_data_1990_2024_posters.zip), which are available in the same repository.
+
+<details>
+  <summary>See sample text data</summary>
+
+|    | backdrop_path                    | genre_ids       |   id | original_language   | original_title              | overview                                                                                                                                                                                                                                                                                                                                                                                                                        |   popularity | poster_path                      | release_date   | title                       | video   |   vote_average |   vote_count |
+|---:|:---------------------------------|:----------------|-----:|:--------------------|:----------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------:|:---------------------------------|:---------------|:----------------------------|:--------|---------------:|-------------:|
+|  0 | /3Nn5BOM1EVw1IYrv6MsbOS6N1Ol.jpg | [14, 18, 10749] |  162 | en                  | Edward Scissorhands         | A small suburban town receives a visit from a castaway unfinished science experiment named Edward.                                                                                                                                                                                                                                                                                                                              |       45.694 | /1RFIbuW9Z3eN9Oxw2KaQG5DfLmD.jpg | 1990-12-07     | Edward Scissorhands         | False   |            7.7 |        12305 |
+|  1 | /sw7mordbZxgITU877yTpZCud90M.jpg | [18, 80]        |  769 | en                  | GoodFellas                  | The true story of Henry Hill, a half-Irish, half-Sicilian Brooklyn kid who is adopted by neighbourhood gangsters at an early age and climbs the ranks of a Mafia family under the guidance of Jimmy Conway.                                                                                                                                                                                                                     |       57.228 | /aKuFiU82s5ISJpGZp7YkIr3kCUd.jpg | 1990-09-12     | GoodFellas                  | False   |            8.5 |        12106 |
+|  2 | /6uLhSLXzB1ooJ3522ydrBZ2Hh0W.jpg | [35, 10751]     |  771 | en                  | Home Alone                  | Eight-year-old Kevin McCallister makes the most of the situation after his family unwittingly leaves him behind when they go on Christmas vacation. But when a pair of bungling burglars set their sights on Kevin's house, the plucky kid stands ready to defend his territory. By planting booby traps galore, adorably mischievous Kevin stands his ground as his frantic mother attempts to race home before Christmas Day. |        3.538 | /onTSipZ8R3bliBdKfPtsDuHTdlL.jpg | 1990-11-16     | Home Alone                  | False   |            7.4 |        10599 |
+|  3 | /vKp3NvqBkcjHkCHSGi6EbcP7g4J.jpg | [12, 35, 878]   |  196 | en                  | Back to the Future Part III | The final installment of the Back to the Future trilogy finds Marty digging the trusty DeLorean out of a mineshaft and looking for Doc in the Wild West of 1885. But when their time machine breaks down, the travelers are stranded in a land of spurs. More problems arise when Doc falls for pretty schoolteacher Clara Clayton, and Marty tangles with Buford Tannen.                                                       |       28.896 | /crzoVQnMzIrRfHtQw0tLBirNfVg.jpg | 1990-05-25     | Back to the Future Part III | False   |            7.5 |         9918 |
+|  4 | /3tuWpnCTe14zZZPt6sI1W9ByOXx.jpg | [35, 10749]     |  114 | en                  | Pretty Woman                | When a millionaire wheeler-dealer enters a business contract with a Hollywood hooker Vivian Ward, he loses his heart in the bargain.                                                                                                                                                                                                                                                                                            |       97.953 | /hVHUfT801LQATGd26VPzhorIYza.jpg | 1990-03-23     | Pretty Woman                | False   |            7.5 |         7671 |
+
+</details>
+
+Next, you will create a corresponding object collection and import the data.
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>