Skip to content

Commit

Permalink
Add Metropolis NIM Workflows as submodule (#237)
Browse files Browse the repository at this point in the history
* add vision worfklow assets

* add metropolis workflows submodule to track main

* update top level readme

* fix formatting
  • Loading branch information
ssmmoo1 authored Nov 17, 2024
1 parent a3f956a commit 2b0f6d7
Show file tree
Hide file tree
Showing 8 changed files with 70 additions and 0 deletions.
4 changes: 4 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[submodule "vision_workflows/metropolis-nim-workflows"]
path = vision_workflows/metropolis-nim-workflows
url = https://github.com/NVIDIA/metropolis-nim-workflows
branch = main
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ This repository is a starting point for developers looking to integrate with the
* [Knowledge Graph RAG](#knowledge-graph-rag)
* [Agentic Workflows with Llama 3.1](#agentic-workflows-with-llama-31)
* [RAG with Local NIM Deployment and LangChain](#rag-with-local-nim-deployment-and-langchain)
* [Vision NIM Workflows](#vision-nim-workflows)
* [Try it Now!](#try-it-now)
* [RAG](#rag)
* [RAG Notebooks](#rag-notebooks)
Expand Down Expand Up @@ -50,6 +51,23 @@ This example implements a GPU-accelerated pipeline for creating and querying kno

For more information, refer to the [Generative AI Example releases](https://github.com/NVIDIA/GenerativeAIExamples/releases/).

### Vision NIM Workflows
A collection of Jupyter notebooks, sample code and reference applications built with Vision NIMs.

To pull the vision NIM workflows, clone this repository recursively:
```
git clone https://github.com/nvidia/GenerativeAIExamples --recurse-submodules
```

The workflows will then be located at [GenerativeAIExamples/vision_workflows](vision_workflows/README.md)

Follow the links below to learn more:
- [Learn how to use VLMs to automatically monitor a video stream for custom events.](nim_workflows/vlm_alerts/README.md)
- [Learn how to search images with natural language using NV-CLIP.](nim_workflows/nvclip_multimodal_search/README.md)
- [Learn how to combine VLMs, LLMs and CV models to build a robust text extraction pipeline.](nim_workflows/vision_text_extraction/README.md)
- [Learn how to use embeddings with NVDINOv2 and a Milvus VectorDB to build a few shot classification model.](nim_workflows/nvdinov2_few_shot/README.md)


## Try it Now!

Experience NVIDIA RAG Pipelines with just a few steps!
Expand Down
47 changes: 47 additions & 0 deletions vision_workflows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Vision Workflows with NVIDIA NIM

This submodule includes NIM based workflows for vision applications. To get access to this submodule, do one of the following:

1) Clone nvidia/GenerativeAIExamples recursively
```
git clone https://github.com/nvidia/GenerativeAIExamples --recurse-submodules
```

2) If you have already cloned the repository without the ```--recurse-submodules``` flag then you can manually pull the submodule
```
git submodule update --init
```

3) You can also clone the vision workflow repository directly
```
git clone https://github.com/NVIDIA/metropolis-nim-workflows.git
```

Once the submodule is pulled or you have cloned the repository directly you can explore the available vision NIM workflows.

## Available Workflows

### [VLM Alert System](https://github.com/NVIDIA/metropolis-nim-workflows/blob/main/nim_workflows/vlm_alerts/README.md)
<p float="left">
<img src="readme_assets/vlm_alert.gif" alt="" width="800"/>
</p>

### [Structured Text Extraction](https://github.com/NVIDIA/metropolis-nim-workflows/tree/main/nim_workflows/vision_text_extraction)
<p float="left">
<img src="readme_assets/text_extraction.gif" alt="" width="800"/>
</p>

### [NVCLIP Multimodal Search](https://github.com/NVIDIA/metropolis-nim-workflows/tree/main/nim_workflows/nvclip_multimodal_search)
<p float="left">
<img src="readme_assets/nvclip_multimodal.gif" alt="" width="800"/>
</p>

### [NV-DINOv2 Few Shot Classification](https://github.com/NVIDIA/metropolis-nim-workflows/tree/main/nim_workflows/nvdinov2_few_shot)
<p float="left">
<img src="readme_assets/few_shot.gif" alt="" width="800"/>
</p>


To learn more, view [this page](https://github.com/NVIDIA/metropolis-nim-workflows)


1 change: 1 addition & 0 deletions vision_workflows/metropolis-nim-workflows
Binary file added vision_workflows/readme_assets/few_shot.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added vision_workflows/readme_assets/vlm_alert.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 2b0f6d7

Please sign in to comment.