diff --git a/_blog.yml b/_blog.yml index 2f91d87772..7bcfa34532 100644 --- a/_blog.yml +++ b/_blog.yml @@ -77,16 +77,16 @@ thumbnail: ./assets/10_tf-serving/thumbnail.png date: January 26, 2021 -- local: ray-rag - title: "Retrieval Augmented Generation with Huggingface Transformers and Ray" - thumbnail: ./assets/12_ray_rag/ray_arch_updated.png - author: amogkam - guest: true - date: February 1, 2021 - - local: pytorch-xla title: "Hugging Face on PyTorch / XLA TPUs" thumbnail: ./assets/13_pytorch_xla/pytorch_xla_thumbnail.png author: jysohn23 guest: true date: February 9, 2021 + +# - local: ray-rag +# title: "Retrieval Augmented Generation with Huggingface Transformers and Ray" +# thumbnail: ./assets/12_ray_rag/ray_arch_updated.png +# author: amogkam +# guest: true +# date: February 10, 2021 \ No newline at end of file diff --git a/porting-fsmt.md b/porting-fsmt.md index 22a3764dee..e7e784eb7c 100644 --- a/porting-fsmt.md +++ b/porting-fsmt.md @@ -18,7 +18,7 @@ thumbnail: /blog/assets/07_porting_fsmt/thumbnail.png
stas Stas Bekman - guest + guest
diff --git a/ray-rag.md b/ray-rag.md index c3c32bbf8f..0e9c7d20ff 100644 --- a/ray-rag.md +++ b/ray-rag.md @@ -1,6 +1,29 @@ -## Retrieval Augmented Generation with Huggingface Transformers and Ray +--- +title: "Retrieval Augmented Generation with Huggingface Transformers and Ray" +thumbnail: /blog/assets/12_ray_rag/ray_arch_updated.png +--- -_[Huggingface Transformers](https://huggingface.co/) recently added the [Retrieval Augmented Generation (RAG)](https://twitter.com/huggingface/status/1310597560906780680) model, a new NLP architecture that leverages external documents (like Wikipedia) to augment its knowledge and achieve state of the art results on knowledge-intensive tasks. In this blog post, we introduce the integration of [Ray](https://docs.ray.io/en/master/), a library for building scalable applications, into the RAG contextual document retrieval mechanism. This speeds up retrieval calls by 2x and improves the scalability of RAG distributed [fine-tuning](https://github.com/huggingface/transformers/tree/master/examples/research_projects/rag)._ +# Retrieval Augmented Generation with Huggingface Transformers and Ray + +
+ Published Feb 10, 2021. + + Update on GitHub + +
+ +
+ + +
+ ray-project + Ray Project (Anyscale) + guest +
+ +
+ +[Huggingface Transformers](https://huggingface.co/) recently added the [Retrieval Augmented Generation (RAG)](https://twitter.com/huggingface/status/1310597560906780680) model, a new NLP architecture that leverages external documents (like Wikipedia) to augment its knowledge and achieve state of the art results on knowledge-intensive tasks. In this blog post, we introduce the integration of [Ray](https://docs.ray.io/en/master/), a library for building scalable applications, into the RAG contextual document retrieval mechanism. This speeds up retrieval calls by 2x and improves the scalability of RAG distributed [fine-tuning](https://github.com/huggingface/transformers/tree/master/examples/research_projects/rag). @@ -101,7 +124,7 @@ _A performance comparison of different retrieval implementations. For each docum To try it out, first install the necessary requirements -``` +```bash pip install ray pip install transformers pip install -r transformers/examples/research_projects/rag/requirements.txt @@ -111,7 +134,7 @@ pip install -r transformers/examples/research_projects/rag/requirements.txt Then, you can specify your data paths and other configurations and run [finetune-rag-ray.sh](https://github.com/huggingface/transformers/blob/master/examples/research_projects/rag/finetune_rag_ray.sh)! -``` +```bash # Sample script to finetune RAG using Ray for distributed retrieval. # Add parent directory to python path to access lightning_base.py diff --git a/ray-tune.md b/ray-tune.md index 7aaf312202..021f8fb7d6 100644 --- a/ray-tune.md +++ b/ray-tune.md @@ -20,7 +20,7 @@ thumbnail: /blog/assets/06_ray_tune/ray-hf.jpg
ray-project Ray Project (Anyscale) - guest + guest
diff --git a/zero-deepspeed-fairscale.md b/zero-deepspeed-fairscale.md index 4255251812..615809b56c 100644 --- a/zero-deepspeed-fairscale.md +++ b/zero-deepspeed-fairscale.md @@ -18,7 +18,7 @@ thumbnail: /blog/assets/11_zero_deepspeed_fairscale/zero-partitioning.png
stas Stas Bekman - guest + guest