From 0b3b4a6417bd5e65cf19b54994bc1a4ecc35edf5 Mon Sep 17 00:00:00 2001 From: Enrico Ros Date: Tue, 9 Jul 2024 06:49:27 -0700 Subject: [PATCH] Reverse proxy instructions. Fixes #587 --- docs/README.md | 1 + docs/config-local-ollama.md | 3 +- docs/deploy-docker.md | 11 +++++++ docs/deploy-k8s.md | 3 ++ docs/deploy-reverse-proxy.md | 58 ++++++++++++++++++++++++++++++++++++ docs/installation.md | 2 ++ 6 files changed, 77 insertions(+), 1 deletion(-) create mode 100644 docs/deploy-reverse-proxy.md diff --git a/docs/README.md b/docs/README.md index d9ae4903d..6b442a071 100644 --- a/docs/README.md +++ b/docs/README.md @@ -46,6 +46,7 @@ Step-by-step deployment and system configuration instructions. - **[Source code alterations guide](customizations.md)**: source code primer and alterations guidelines - **[Basic Authentication](deploy-authentication.md)**: Optional, adds a username and password wall - **[Database Setup](deploy-database.md)**: Optional, enables "Chat Link Sharing" + - **[Reverse Proxy](deploy-reverse-proxy.md)**: Optional, enables custom domain and SSL - **[Environment Variables](environment-variables.md)**: 📌 Pre-configures models and services ## Support and Community diff --git a/docs/config-local-ollama.md b/docs/config-local-ollama.md index 79baca069..d0920d542 100644 --- a/docs/config-local-ollama.md +++ b/docs/config-local-ollama.md @@ -81,7 +81,8 @@ Then, edit the nginx configuration file `/etc/nginx/sites-enabled/default` and a proxy_buffering off; proxy_cache off; - # Longer timeouts + # Longer timeouts (1hr) + keepalive_timeout 3600; proxy_read_timeout 3600; proxy_connect_timeout 3600; proxy_send_timeout 3600; diff --git a/docs/deploy-docker.md b/docs/deploy-docker.md index ae502a0fc..8314ee817 100644 --- a/docs/deploy-docker.md +++ b/docs/deploy-docker.md @@ -59,6 +59,17 @@ To make local services running on your host machine accessible to a Docker conta
+### Reverse Proxy Configuration + +A reverse proxy is a server that sits in front of big-AGI's container and can forwards web +requests to it. Often used to run multiple web applications, expose them to the internet, +increase security. + +If you're deploying big-AGI behind a reverse proxy, you may want to see +our [Reverse Proxy Deployment Guide](deploy-reverse-proxy.md) for more information. + +
+ ### More Information The [`Dockerfile`](../Dockerfile) describes how to create a Docker image. It establishes a Node.js environment, diff --git a/docs/deploy-k8s.md b/docs/deploy-k8s.md index c298799d2..410256fd3 100644 --- a/docs/deploy-k8s.md +++ b/docs/deploy-k8s.md @@ -79,4 +79,7 @@ To update big-AGI to the latest version: This will trigger a rolling update of the deployment with the latest image. +**Note**: If you're deploying big-AGI behind a reverse proxy, you may need to configure +your proxy to support streaming. See our [Reverse Proxy Deployment Guide](deploy-reverse-proxy.md) for more information. + Note: For production use, consider setting up an Ingress Controller or Load Balancer instead of using port-forward. \ No newline at end of file diff --git a/docs/deploy-reverse-proxy.md b/docs/deploy-reverse-proxy.md new file mode 100644 index 000000000..08dcad0a9 --- /dev/null +++ b/docs/deploy-reverse-proxy.md @@ -0,0 +1,58 @@ +# Advanced: Deploying big-AGI behind a Reverse Proxy + +Note: if you don't have a reverse proxy set up, you can skip this guide. + +If you're deploying big-AGI behind a reverse proxy, you may want to configure your proxy to support streaming output. +This guide provides instructions on how to configure your reverse proxy to support streaming output from big-AGI. + +This is for advanced deployments, and you should have a basic understanding of how reverse proxies work. + +## Nginx Configuration + +If you're using Nginx as your reverse proxy, add the following configuration to your server block: + +```nginx +server { + listen 80; + server_name your-domain.com; + + location / { + # ...your specific proxy_pass configuration, example below... + proxy_pass http://localhost:3000; # Assuming big-AGI is running on port 3000 + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection 'upgrade'; + proxy_set_header Host $host; + proxy_cache_bypass $http_upgrade; + # ... + + # Important: Disable buffering for the streaming responses (SSE) + chunked_transfer_encoding on; # Turn on chunked transfer encoding + proxy_buffering off; # Turn off proxy buffering + proxy_cache off; # Turn off caching + tcp_nodelay on; # Turn on TCP NODELAY option, disable delay ACK algorithm + tcp_nopush on; # Turn on TCP NOPUSH option, disable Nagle algorithm + + # Important: Longer timeouts (5 min) + keepalive_timeout 300; + proxy_connect_timeout 300; + proxy_read_timeout 300; + proxy_send_timeout 300; + } +} +``` + +This configuration disables caching and buffering, enables chunked transfer encoding, and adjusts TCP settings to optimize for streaming content. + +## Troubleshooting + +If you're experiencing issues with streaming not working, especially when deploying behind a reverse proxy, +ensure that your proxy is configured to support streaming output as described above. + +## Additional Resources + +- For Docker deployments, see our [Docker Deployment Guide](deploy-docker.md) +- For Kubernetes deployments, see our [Kubernetes Deployment Guide](deploy-k8s.md) +- For general installation instructions, see our [Installation Guide](installation.md) + +If you continue to experience issues, please reach out to our [community support channels](README.md#support-and-community). diff --git a/docs/installation.md b/docs/installation.md index 9b687e6f2..a0528f7c1 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -99,6 +99,8 @@ or follow the steps below for a quick start. ``` Access your big-AGI instance at `http://localhost:3000`. +If you deploy big-AGI behind a reverse proxy, you may want to check out the [Reverse Proxy Configuration Guide](deploy-reverse-proxy.md). + ### Kubernetes Deployment Deploy big-AGI on a Kubernetes cluster for enhanced scalability and management. Follow these steps for a Kubernetes deployment: