Merge pull request #190 from cert-ee/162-change-django-wsgi-for-asgi

cert-ee-raidar · web-flow · commit fb7b7b3e705c · 2024-12-17T12:53:56.000+02:00
162 change django wsgi for asgi
diff --git a/docs/src/configuring/web-ui.md b/docs/src/configuring/web-ui.md
@@ -2,47 +2,125 @@
 
 Cuckoo3 frontend is built with [Django](https://www.djangoproject.com/){:target=_blank}.   
 Default settings can be overwritten in `~/.cuckoocwd/web/web_local_settings.py`.  
-You can use the frontend in two ways - commandline or [Nginx](https://nginx.org/en/){:target=_blank} and [uWSGI](https://uwsgi-docs.readthedocs.io/en/latest/){:target=_blank}.
+You can use the frontend in two ways - commandline or [Nginx](https://nginx.org/en/){:target=_blank} and [WSGI](https://uwsgi-docs.readthedocs.io/en/latest/){:target=_blank} / [ASGI](https://asgi.readthedocs.io/en/latest/){:target=_blank}.  
 
-## Commandline 
+!!! note "Recommendation"
+    We recommend ASGI from Cuckoo version 0.10.0 and on
 
-!!! warning "Unverified"
+## Generating static content
+Before serving Cuckoo web server, you have to generate the static content.
 
-    This is from the old documentation and needs verification.  
-    It may contain errors, bugs or outdated information.
+**Steps**
 
-This is a development server.  
-You can start Cuckoo frontend from the commandline with the following command:
+1. Set `STATIC_ROOT` variable in `~/.cuckoocwd/web/web_local_settings.py`. Cuckoo user must have access to that path.
 
-```bash
-cuckoo web --host <listen ip> --port <listen port>
-```
+2. Build documentation for Cuckoo. You need to be in `cuckoo3` directory and have Python virtual environment active.
 
-## Serving the web UI with Nginx And uWSGI 
+        cd docs
+        python3.10 -m pip install -r requirements.txt
+        mkdocs build
+        cp -R site ../web/cuckoo/web/static/docs
+
+3. Run Django `collectstatic` command. 
+
+        cuckoo web djangocommand collectstatic
+
+## Serving the web UI with Daphne(ASGI) and Nginx
 
 !!! note "Requirements"
 
     Please make sure that you have:
 
-    - installed all dependencies for [serving API and web](../installing/dependencies.md#serving-api-and-web){:target=_blank}  
+    - installed all dependencies for [serving API and web - ASGI](../installing/dependencies.md#asgi){:target=_blank}  
 
-If you want to serve Cuckoo in an environment such as development, testing, staging or production, you need to use uWSGI and Nginx.  
-uWSGI is used as an application server and Nginx as a webserver.
+Daphne is used as an ASGI application server and Nginx as a webserver.
 
 **Steps**
 
-1. Set `STATIC_ROOT` variable in `~/.cuckoocwd/web/web_local_settings.py`. Cuckoo user must have access to that path.
+1. Create a systemd service for Daphne
+
+        sudo cat <<EOF > /etc/systemd/system/cuckoo-asgi.service
+        [Unit]
+        Description=Daphne ASGI Server
+        After=network.target
+
+        [Service]
+        User=cuckoo
+        Group=cuckoo
+        WorkingDirectory=/home/cuckoo/cuckoo3/web/cuckoo/web
+        ExecStart=/home/cuckoo/cuckoo3/venv/bin/daphne -p 9090 cuckoo.web.web.asgi:application
+        Environment=CUCKOO_APP=web
+        Environment=CUCKOO_CWD=/home/cuckoo/.cuckoocwd
+        Environment=CUCKOO_LOGLEVEL=DEBUG
+        Restart=always
+
+        [Install]
+        WantedBy=multi-user.target
+        EOF
+
+2. Enable and start ASGI service
+
+        sudo systemctl enable cuckoo-asgi.service && \
+        sudo systemctl start cuckoo-asgi.service
+
+3. Create a new configuration for Nginx
+
+        sudo cat <<EOF > /etc/nginx/sites-available/cuckoo-web.conf
+        upstream _asgi_server {
+            server 127.0.0.1:9090;
+        }
+
+        server {
+            listen 8080;
+
+            # Directly serve the static files for Cuckoo web. Copy
+            # (and update these after Cuckoo updates) these by running:
+            # 'cuckoo web djangocommand collectstatic'. The path after alias should
+            # be the same path as STATIC_ROOT. These files can be cached. Be sure
+            # to clear the cache after any updates.
+            location /static {
+                alias /opt/cuckoo3/static;
+            }
+
+            # Pass any non-static requests to the Cuckoo web wsgi application run
+            # by uwsgi. It is not recommended to cache paths here, this can cause
+            # the UI to no longer reflect the correct state of analyses and tasks.
+            location / {
+                proxy_set_header Host $host;  # Ensures Host header is preserved
+                proxy_set_header X-Real-IP $remote_addr;
+                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+                proxy_set_header X-Forwarded-Proto $scheme;
+                client_max_body_size 1G;
+                proxy_redirect off;
+                proxy_pass http://_asgi_server;
+            }
+        }
+        EOF
+
+4. Create symlink to enable Nginx configuration.
 
-2. Build documentation for Cuckoo. You need to be in `cuckoo3` directory and have Python virtual environment active.
+        sudo ln -s /etc/nginx/sites-available/cuckoo-web.conf /etc/nginx/sites-enabled/cuckoo-web.conf
 
-        cd docs
-        python3.10 -m pip install -r requirements.txt
-        mkdocs build
-        cp -R site ../web/cuckoo/web/static/docs
+5. Delete Nginx default enabled configuration.
 
-3. Run Django `collectstatic` command. 
+        sudo rm /etc/nginx/sites-enabled/default
 
-        cuckoo web djangocommand collectstatic
+6. Reload the new Nginx configuration.
+
+        sudo systemctl reload nginx
+
+
+## Serving the web UI with uWSGI(WSGI) and Nginx
+
+!!! note "Requirements"
+
+    Please make sure that you have:
+
+    - installed all dependencies for [serving API and web - WSGI](../installing/dependencies.md#wsgi){:target=_blank}  
+
+uWSGI is used as WSGI application server and Nginx as a webserver.
+
+**Steps**
 
 4. Generate uWSGI configuration.
 
@@ -76,3 +154,17 @@ uWSGI is used as an application server and Nginx as a webserver.
 
         sudo systemctl reload nginx
 
+
+## Commandline 
+
+!!! warning "Unverified"
+
+    This is from the old documentation and needs verification.  
+    It may contain errors, bugs or outdated information.
+
+This is a development server.  
+You can start Cuckoo frontend from the commandline with the following command:
+
+```bash
+cuckoo web --host <listen ip> --port <listen port>
+```
diff --git a/docs/src/installing/dependencies.md b/docs/src/installing/dependencies.md
@@ -133,6 +133,7 @@ VMCloak uses QEMU to create and manage virtual machines.
 |`qemu-system-x86`|full system emulation binaries to emulate the following x86 hardware: x86_64|
 
 ### Serving API and web
+#### WSGI
 ```bash 
 sudo apt-get install -y uwsgi uwsgi-plugin-python3 nginx
 ```
@@ -143,3 +144,16 @@ sudo apt-get install -y uwsgi uwsgi-plugin-python3 nginx
 |`uwsgi-plugin-python3`|Python plugin for uWSGI|
 |`nginx`|Webserver|
 
+#### ASGI
+Install Python dependencies inside Cuckoo virtual environment
+```bash 
+python3.10 -m pip install daphne
+sudo apt-get install -y nginx
+```
+
+| Package |  Description |
+|---|---|
+|`daphne`|Django ASGI server|
+|`nginx`|Webserver|
+
+
diff --git a/web/cuckoo/web/web/asgi.py b/web/cuckoo/web/web/asgi.py
@@ -9,8 +9,13 @@
 
 import os
 
-from django.core.asgi import get_asgi_application
+if os.environ.get("CUCKOO_APP", "").lower() == "web":
+    from cuckoo.web.web.startup import init_and_get_asgi
 
-os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'web.settings')
+    application = init_and_get_asgi()
 
-application = get_asgi_application()
+else:
+    from django.core.asgi import get_asgi_application
+
+    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "web.settings")
+    application = get_asgi_application()
diff --git a/web/cuckoo/web/web/startup.py b/web/cuckoo/web/web/startup.py
@@ -9,8 +9,11 @@
 from cuckoo.common import shutdown
 from cuckoo.common.log import exit_error, name_to_level
 from cuckoo.common.startup import (
-    init_global_logging, load_configuration, init_database,
-    init_elasticsearch, StartupError
+    init_global_logging,
+    load_configuration,
+    init_database,
+    init_elasticsearch,
+    StartupError,
 )
 from cuckoo.common.storage import cuckoocwd, Paths, CWD_ENVVAR, CWDError
 from cuckoo.common.submit import settings_maker
@@ -31,38 +34,32 @@ def set_path_settings():
 def _init_remote_storage():
     api = APIClient(
         cfg("web.yaml", "remote_storage", "api_url", subpkg="web"),
-        cfg("web.yaml", "remote_storage", "api_key", subpkg="web")
+        cfg("web.yaml", "remote_storage", "api_key", subpkg="web"),
     )
     retriever.set_api_client(api)
 
 
 def _init_elasticsearch_web():
     hosts = cfg("web.yaml", "elasticsearch", "hosts", subpkg="web")
-    indices = cfg(
-        "web.yaml", "elasticsearch", "indices", "names", subpkg="web"
+    indices = cfg("web.yaml", "elasticsearch", "indices", "names", subpkg="web")
+    max_window = cfg("web.yaml", "elasticsearch", "max_result_window", subpkg="web")
+    user = cfg("web.yaml", "elasticsearch", "user", subpkg="web")
+    password = cfg("web.yaml", "elasticsearch", "password", subpkg="web")
+    ca_certs = cfg("web.yaml", "elasticsearch", "ca_certs", subpkg="web")
+
+    init_elasticsearch(
+        hosts,
+        indices,
+        max_result_window=max_window,
+        create_missing_indices=False,
+        user=user,
+        password=password,
+        ca_certs=ca_certs,
     )
-    max_window = cfg(
-        "web.yaml", "elasticsearch", "max_result_window", subpkg="web"
-    )
-    user = cfg(
-        "web.yaml", "elasticsearch", "user", subpkg="web"
-    )
-    password = cfg(
-        "web.yaml", "elasticsearch", "password", subpkg="web"
-    )
-    ca_certs = cfg(
-        "web.yaml", "elasticsearch", "ca_certs", subpkg="web"
-    )
-
-    init_elasticsearch(hosts, indices, max_result_window=max_window,
-                       create_missing_indices=False,
-                       user=user, password=password, ca_certs=ca_certs)
 
 
 def _init_statistics_web():
-    charts = cfg(
-        "web.yaml", "elasticsearch", "statistics", "charts", subpkg="web"
-    )
+    charts = cfg("web.yaml", "elasticsearch", "statistics", "charts", subpkg="web")
 
     try:
         for chart in charts:
@@ -103,12 +100,8 @@ def init_web(cuckoo_cwd, loglevel, logfile=""):
         if cfg("web.yaml", "remote_storage", "enabled", subpkg="web"):
             _init_remote_storage()
 
-        search = cfg(
-            "web.yaml", "elasticsearch", "web_search", "enabled", subpkg="web"
-        )
-        stats = cfg(
-            "web.yaml", "elasticsearch", "statistics", "enabled", subpkg="web"
-        )
+        search = cfg("web.yaml", "elasticsearch", "web_search", "enabled", subpkg="web")
+        stats = cfg("web.yaml", "elasticsearch", "statistics", "enabled", subpkg="web")
         if search or stats:
             _init_elasticsearch_web()
 
@@ -136,9 +129,55 @@ def djangocommands(*args):
     execute_from_command_line(("cuckoo",) + args)
 
 
+def init_and_get_asgi():
+    """Initializes and returns ASGI server application
+
+    Args:
+        None
+
+    Returns:
+        ASGI 3 callable
+
+    Raises:
+        ValueError: When log level is invalid
+        custom error: When Cuckoo CWD is not set
+    """
+
+    import logging
+    from cuckoo.common.log import disable_console_colors
+
+    levelname = os.environ.get("CUCKOO_LOGLEVEL")
+    if not levelname:
+        loglevel = logging.DEBUG
+    else:
+        try:
+            loglevel = name_to_level(levelname)
+        except ValueError as e:
+            exit_error(f"Invalid log level name. {e}")
+
+    cwd_path = os.environ.get(CWD_ENVVAR)
+    if not cwd_path:
+        exit_error(
+            f"Cannot start. Environment variable '{CWD_ENVVAR}' must "
+            f"contain path to Cuckoo CWD."
+        )
+
+    # Disable console colors, because log files for services such as uWSGI
+    # capture console logs. Color formatting characters can make the log file
+    # unreadable.
+    disable_console_colors()
+    init_web(cwd_path, loglevel)
+
+    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "cuckoo.web.web.settings")
+    from django.core.asgi import get_asgi_application
+
+    return get_asgi_application()
+
+
 def init_and_get_wsgi():
     import logging
     from cuckoo.common.log import disable_console_colors
+
     levelname = os.environ.get("CUCKOO_LOGLEVEL")
     if not levelname:
         loglevel = logging.DEBUG
@@ -161,6 +200,7 @@ def init_and_get_wsgi():
     disable_console_colors()
     init_web(cwd_path, loglevel)
 
-    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'cuckoo.web.web.settings')
+    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "cuckoo.web.web.settings")
     from django.core.wsgi import get_wsgi_application
+
     return get_wsgi_application()
