adding support for async callbacks and page layouts #3089

BSd3v · 2024-11-21T19:08:41Z

This PR is attempting to allow for layouts and callbacks to be performed in a way that will allow for async functions natively.

…ayouts reference this in order to determine whether or not they should load as async functions.

…ing at the function to determine

…r testing and not relying on a specific string to be present as a comment

…e as `async`. Added more defined error messages when libraries are missing or async was used without async turned on.

-- failing lint on purpose for test

…for missing `callback_ctx`

ndrezn · 2025-04-10T15:54:54Z

@BSd3v is there a feature ticket for this change anywhere?

BSd3v · 2025-04-10T15:57:41Z

Not that I can find. I've seen it come up a couple times on the forums.

T4rk1n

Looks good, there is quite a bit of code duplication but I guess there is not really a choice with this implementation, most of the code of the has been extracted well into functions 👍

Just need to merge with dev and resolve the conflict and should be good.

T4rk1n · 2025-06-10T18:51:54Z

.circleci/config.yml

+  test-312-async: &test
+    working_directory: ~/dash
+    docker:
+      - image: cimg/python:3.12.1-browsers
+        auth:
+          username: dashautomation
+          password: $DASH_PAT_DOCKERHUB
+        environment:
+          PERCY_ENABLE: 1
+          PERCY_PARALLEL_TOTAL: -1
+          PUPPETEER_SKIP_CHROMIUM_DOWNLOAD: True
+          PYVERSION: python312
+          REDIS_URL: redis://localhost:6379
+      - image: cimg/redis:6.2.6
+        auth:
+          username: dashautomation
+          password: $DASH_PAT_DOCKERHUB
+    parallelism: 3
+    steps:
+      - checkout:
+          path: ~/dash
+      - run:
+          name: Add chrome keys & update.
+          command: |
+            wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
+            sudo apt-get update
+      - run: echo $PYVERSION > ver.txt
+      - run: cat requirements/*.txt > requirements-all.txt
+      - restore_cache:
+          key: dep-{{ checksum ".circleci/config.yml" }}-{{ checksum "ver.txt" }}-{{ checksum "requirements-all.txt" }}
+      - browser-tools/install-browser-tools:
+          chrome-version: 120.0.6099.71
+          install-firefox: false
+          install-geckodriver: false
+      - attach_workspace:
+          at: ~/dash
+      - run:
+          name: ️️🏗️ Install package
+          command: |
+            . venv/bin/activate
+            npm ci
+            pip install dash-package/dash-package.tar.gz[async,ci,dev,testing,celery,diskcache] --progress-bar off
+            pip list | grep dash
+      - run:
+          name: 🧪 Run Integration Tests
+          command: |
+            . venv/bin/activate
+            npm run citest.integration
+      - store_artifacts:
+          path: test-reports
+      - store_test_results:
+          path: test-reports
+      - store_artifacts:
+          path: /tmp/dash_artifacts
+


Can we move the new tests to a github action, maybe bundled with the background tests?

GHA werent implemented when this was originally created, I can look into adjusting the yaml for that.

T4rk1n · 2025-06-10T18:55:08Z

dash/background_callback/managers/celery_manager.py

+        if asyncio.iscoroutinefunction(fn):
+            func = partial(ctx.run, async_run)
+            asyncio.run(func())
+        else:
+            ctx.run(run)


Are we guaranteed to be out of the asyncio loop? In which case, the run might fail. Otherwise it might be better to only have the async_run definition so there is less code repetition.

T4rk1n · 2025-06-10T18:56:11Z

dash/background_callback/managers/diskcache_manager.py

+        if asyncio.iscoroutinefunction(fn):
+            func = partial(ctx.run, async_run)
+            asyncio.run(func())
+        else:
+            ctx.run(run)


Same comment as per celery_manager, but for this one I am pretty sure there is no loop running.

T4rk1n · 2025-06-10T18:57:52Z

dash/testing/browser.py

+        try:
+            import asgiref  # pylint: disable=unused-import, import-outside-toplevel # noqa: F401, C0415
+
+            name += "_async"
+        except ImportError:
+            pass


We don't really need the percy integration if we run the tests separately in a new github action.=

T4rk1n · 2025-06-10T19:01:39Z

package.json

@@ -31,7 +31,7 @@
    "private::test.unit-dash": "pytest tests/unit",
    "private::test.unit-renderer": "cd dash/dash-renderer && npm run test",
    "private::test.unit-generation": "cd @plotly/dash-generator-test-component-typescript && npm ci && npm test",
-    "private::test.integration-dash": "TESTFILES=$(circleci tests glob \"tests/integration/**/test_*.py\" | circleci tests split --split-by=timings) && pytest --headless --nopercyfinalize --junitxml=test-reports/junit_intg.xml ${TESTFILES}",
+    "private::test.integration-dash": "TESTFILES=$(circleci tests glob \"tests/integration/**/test_*.py\" | circleci tests split --split-by=timings) && pytest --headless --nopercyfinalize --junitxml=test-reports/junit_intg.xml ${TESTFILES} && python rerun_failed_tests.py",


We have a package pytest-rerunfailures, we used to use when the tests were very flaky. We could re-enable it for the time being.

T4rk1n · 2025-06-10T19:03:09Z

dash/_callback.py

+        func_kwargs,
+        app,
+        original_packages,
+        False,


If this false is hardcoded it should be removed from where it is used.
I feel like there is a bit too much in this tuple, it may be worth it to define a NamedTuple and add -> DispatchContext to the function signature.

dash/_callback.py

T4rk1n · 2025-06-10T19:11:14Z

dash/_callback.py

-                output_value = []
-                flat_output_values = []
+            _prepare_response(
+                output_value,
+                output_spec,
+                multi,
+                response,
+                callback_ctx,
+                app,
+                original_packages,
+                background,
+                has_update,
+                has_output,
+                output,
+                callback_id,
+                allow_dynamic_callbacks,


Too much arguments, maybe with the NamedTuple we can pass these arguments as one?

# Conflicts: # dash/_callback.py # dash/dash.py

Co-authored-by: Philippe Duval <[email protected]>

…support-async-callbacks

adding support for async callbacks and page layouts

e1002d5

BSd3v requested review from alexcjohnson, T4rk1n, ndrezn, gvwilson and emilykl as code owners November 21, 2024 19:08

BSd3v added 14 commits November 21, 2024 14:59

adding new use_async attribute to Dash and having callbacks and l…

e24e094

…ayouts reference this in order to determine whether or not they should load as async functions.

fixing issue with indentations of layouts

5d492dc

removing needing to classify the function as async, instead I am look…

69ee069

…ing at the function to determine

fixing for lint

950888c

adjustments for the test for the debugger, making test more robust fo…

916efdc

…r testing and not relying on a specific string to be present as a comment

fixing lint issues

0dd778e

Adjustments for use_async and determining whether the app can be us…

2d9a930

…e as `async`. Added more defined error messages when libraries are missing or async was used without async turned on.

disable lint for unused import on asgiref

b2f9cd6

adjustments for formatting

f1ac667

Merge remote-tracking branch 'dash/dev' into support-async-callbacks

378099f

adding additional ignore for flake8

52de22e

more adjustments for lint

4189ed6

adding dash[async]

0ddf254

Merge branch 'plotly:dev' into support-async-callbacks

76a9a56

gvwilson assigned BSd3v and T4rk1n Nov 26, 2024

gvwilson added feature something new community community contribution labels Nov 26, 2024

BSd3v added 6 commits December 3, 2024 21:52

attempt no 1 for refactoring dash for dispatch

337ec26

-- failing lint on purpose for test

fixing for lint

a730b4b

fixing no outputs

538515d

attempt no 1 for refactoring callbacks

4f14a1a

fixing for multi outputs

96df44e

attempt no 1 refactoring background callbacks for async functions

91400a3

BSd3v added 9 commits March 25, 2025 12:51

updating missing long -> background adjustments and fixing issue …

fc6c6bc

…for missing `callback_ctx`

fixing for lint

503f7c0

adding tests for async background callbacks

c26671e

updating to bypass test if not async

20dcd67

fixing for lint

bb3d881

update for celery app process

c6940c7

Merge branch 'dev' into support-async-callbacks

b7a8c4a

Merge branch 'dev' into support-async-callbacks

51c1689

Merge branch 'dev' into support-async-callbacks

eb96c5d

T4rk1n reviewed Jun 10, 2025

View reviewed changes

BSd3v and others added 18 commits June 11, 2025 09:39

Merge remote-tracking branch 'remote/dev' into support-async-callbacks

832ab4d

# Conflicts: # dash/_callback.py # dash/dash.py

fixing tyop

d0b2dd3

fixing for issues with callback structure

e88bc3c

Update dash/_callback.py

a1bd54f

Co-authored-by: Philippe Duval <[email protected]>

adjusting issue with try

7f35ecd

Merge branch 'support-async-callbacks' of github.com:BSd3v/dash into …

28b0693

…support-async-callbacks

removing test_async from the utils to keep from confusing pytest

443f785

fixing for lint

07bcd38

updates for new background_callbacks test path

e82f561

adjustment for GHA to run the async test

43fe153

adding step for creating the test components

6fd556d

altering path for the get background callback manager

abbfcd2

verifying redis is still running

13a33be

issue with celery

e3a1496

adjusting order

e8bcb22

attempt to fix failing test

43d10c4

moving async install

4b3c011

adding test components

777a12b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

adding support for async callbacks and page layouts #3089

adding support for async callbacks and page layouts #3089

BSd3v commented Nov 21, 2024

Uh oh!

ndrezn commented Apr 10, 2025

Uh oh!

BSd3v commented Apr 10, 2025

Uh oh!

T4rk1n left a comment

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

BSd3v Jun 11, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

Uh oh!

Uh oh!

adding support for async callbacks and page layouts #3089

Are you sure you want to change the base?

adding support for async callbacks and page layouts #3089

Conversation

BSd3v commented Nov 21, 2024

Uh oh!

ndrezn commented Apr 10, 2025

Uh oh!

BSd3v commented Apr 10, 2025

Uh oh!

T4rk1n left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!