adding support for async callbacks and page layouts #3089

BSd3v · 2024-11-21T19:08:41Z

This PR is attempting to allow for layouts and callbacks to be performed in a way that will allow for async functions natively.

…ayouts reference this in order to determine whether or not they should load as async functions.

…ing at the function to determine

…r testing and not relying on a specific string to be present as a comment

…e as `async`. Added more defined error messages when libraries are missing or async was used without async turned on.

-- failing lint on purpose for test

…ests vs 2

…ing the clearing input values to make sure that we were only triggering callbacks when we desired.

…eck for async and wrapping a view.

…being cleared allows the failed integrations to pass

…t name to note if the image is `async` vs default

# Conflicts: # .circleci/config.yml # dash/_callback.py # dash/background_callback/managers/celery_manager.py # dash/dash.py

…for missing `callback_ctx`

ndrezn · 2025-04-10T15:54:54Z

@BSd3v is there a feature ticket for this change anywhere?

BSd3v · 2025-04-10T15:57:41Z

Not that I can find. I've seen it come up a couple times on the forums.

T4rk1n

Looks good, there is quite a bit of code duplication but I guess there is not really a choice with this implementation, most of the code of the has been extracted well into functions 👍

Just need to merge with dev and resolve the conflict and should be good.

T4rk1n · 2025-06-10T18:51:54Z

.circleci/config.yml

+  test-312-async: &test
+    working_directory: ~/dash
+    docker:
+      - image: cimg/python:3.12.1-browsers
+        auth:
+          username: dashautomation
+          password: $DASH_PAT_DOCKERHUB
+        environment:
+          PERCY_ENABLE: 1
+          PERCY_PARALLEL_TOTAL: -1
+          PUPPETEER_SKIP_CHROMIUM_DOWNLOAD: True
+          PYVERSION: python312
+          REDIS_URL: redis://localhost:6379
+      - image: cimg/redis:6.2.6
+        auth:
+          username: dashautomation
+          password: $DASH_PAT_DOCKERHUB
+    parallelism: 3
+    steps:
+      - checkout:
+          path: ~/dash
+      - run:
+          name: Add chrome keys & update.
+          command: |
+            wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
+            sudo apt-get update
+      - run: echo $PYVERSION > ver.txt
+      - run: cat requirements/*.txt > requirements-all.txt
+      - restore_cache:
+          key: dep-{{ checksum ".circleci/config.yml" }}-{{ checksum "ver.txt" }}-{{ checksum "requirements-all.txt" }}
+      - browser-tools/install-browser-tools:
+          chrome-version: 120.0.6099.71
+          install-firefox: false
+          install-geckodriver: false
+      - attach_workspace:
+          at: ~/dash
+      - run:
+          name: ️️🏗️ Install package
+          command: |
+            . venv/bin/activate
+            npm ci
+            pip install dash-package/dash-package.tar.gz[async,ci,dev,testing,celery,diskcache] --progress-bar off
+            pip list | grep dash
+      - run:
+          name: 🧪 Run Integration Tests
+          command: |
+            . venv/bin/activate
+            npm run citest.integration
+      - store_artifacts:
+          path: test-reports
+      - store_test_results:
+          path: test-reports
+      - store_artifacts:
+          path: /tmp/dash_artifacts
+


Can we move the new tests to a github action, maybe bundled with the background tests?

T4rk1n · 2025-06-10T18:55:08Z

dash/background_callback/managers/celery_manager.py

+        if asyncio.iscoroutinefunction(fn):
+            func = partial(ctx.run, async_run)
+            asyncio.run(func())
+        else:
+            ctx.run(run)


Are we guaranteed to be out of the asyncio loop? In which case, the run might fail. Otherwise it might be better to only have the async_run definition so there is less code repetition.

T4rk1n · 2025-06-10T18:56:11Z

dash/background_callback/managers/diskcache_manager.py

+        if asyncio.iscoroutinefunction(fn):
+            func = partial(ctx.run, async_run)
+            asyncio.run(func())
+        else:
+            ctx.run(run)


Same comment as per celery_manager, but for this one I am pretty sure there is no loop running.

T4rk1n · 2025-06-10T18:57:52Z

dash/testing/browser.py

+        try:
+            import asgiref  # pylint: disable=unused-import, import-outside-toplevel # noqa: F401, C0415
+
+            name += "_async"
+        except ImportError:
+            pass


We don't really need the percy integration if we run the tests separately in a new github action.=

T4rk1n · 2025-06-10T19:01:39Z

package.json

@@ -31,7 +31,7 @@
    "private::test.unit-dash": "pytest tests/unit",
    "private::test.unit-renderer": "cd dash/dash-renderer && npm run test",
    "private::test.unit-generation": "cd @plotly/dash-generator-test-component-typescript && npm ci && npm test",
-    "private::test.integration-dash": "TESTFILES=$(circleci tests glob \"tests/integration/**/test_*.py\" | circleci tests split --split-by=timings) && pytest --headless --nopercyfinalize --junitxml=test-reports/junit_intg.xml ${TESTFILES}",
+    "private::test.integration-dash": "TESTFILES=$(circleci tests glob \"tests/integration/**/test_*.py\" | circleci tests split --split-by=timings) && pytest --headless --nopercyfinalize --junitxml=test-reports/junit_intg.xml ${TESTFILES} && python rerun_failed_tests.py",


We have a package pytest-rerunfailures, we used to use when the tests were very flaky. We could re-enable it for the time being.

T4rk1n · 2025-06-10T19:03:09Z

dash/_callback.py

+        func_kwargs,
+        app,
+        original_packages,
+        False,


If this false is hardcoded it should be removed from where it is used.
I feel like there is a bit too much in this tuple, it may be worth it to define a NamedTuple and add -> DispatchContext to the function signature.

T4rk1n · 2025-06-10T19:07:53Z

dash/_callback.py

+    )
+
+
+def _get_callback_manager(kwargs, background):


We can add the typing here:

Suggested change

def _get_callback_manager(kwargs, background):

def _get_callback_manager(kwargs: dict, background: dict) -> Union[BaseBackgroundCallbackManager, None]:

T4rk1n · 2025-06-10T19:11:14Z

dash/_callback.py

-                output_value = []
-                flat_output_values = []
+            _prepare_response(
+                output_value,
+                output_spec,
+                multi,
+                response,
+                callback_ctx,
+                app,
+                original_packages,
+                background,
+                has_update,
+                has_output,
+                output,
+                callback_id,
+                allow_dynamic_callbacks,


Too much arguments, maybe with the NamedTuple we can pass these arguments as one?

adding support for async callbacks and page layouts

e1002d5

BSd3v requested review from alexcjohnson, T4rk1n, ndrezn, gvwilson and emilykl as code owners November 21, 2024 19:08

BSd3v added 14 commits November 21, 2024 14:59

adding new use_async attribute to Dash and having callbacks and l…

e24e094

…ayouts reference this in order to determine whether or not they should load as async functions.

fixing issue with indentations of layouts

5d492dc

removing needing to classify the function as async, instead I am look…

69ee069

…ing at the function to determine

fixing for lint

950888c

adjustments for the test for the debugger, making test more robust fo…

916efdc

…r testing and not relying on a specific string to be present as a comment

fixing lint issues

0dd778e

Adjustments for use_async and determining whether the app can be us…

2d9a930

…e as `async`. Added more defined error messages when libraries are missing or async was used without async turned on.

disable lint for unused import on asgiref

b2f9cd6

adjustments for formatting

f1ac667

Merge remote-tracking branch 'dash/dev' into support-async-callbacks

378099f

adding additional ignore for flake8

52de22e

more adjustments for lint

4189ed6

adding dash[async]

0ddf254

Merge branch 'plotly:dev' into support-async-callbacks

76a9a56

gvwilson assigned BSd3v and T4rk1n Nov 26, 2024

gvwilson added feature something new community community contribution labels Nov 26, 2024

BSd3v added 6 commits December 3, 2024 21:52

attempt no 1 for refactoring dash for dispatch

337ec26

-- failing lint on purpose for test

fixing for lint

a730b4b

fixing no outputs

538515d

attempt no 1 for refactoring callbacks

4f14a1a

fixing for multi outputs

96df44e

attempt no 1 refactoring background callbacks for async functions

91400a3

BSd3v and others added 25 commits December 5, 2024 16:41

Merge remote-tracking branch 'dash/dev' into support-async-callbacks

1856f5c

fixing for lint and progress outputs

671cb2b

lint adjustments

f19fe23

adding async tests

17c824d

bypassing test_rdrh003_refresh_jwt as this fails with 3 failed requ…

fdfd058

…ests vs 2

removing __init__ from async directory

fb691c3

adjusting jwt test to adjust value in the MultiProcessing and remov…

2473546

…ing the clearing input values to make sure that we were only triggering callbacks when we desired.

removing flaky for lint

b2e8884

adjustments for formatting

df29ee6

simplifying jwt test by using before_request removing needs to ch…

5342573

…eck for async and wrapping a view.

attempting to allow for failed tests to be rerun to see if the cache …

c79debf

…being cleared allows the failed integrations to pass

moving retry to integration-dash

37d649c

adjusting browser for percy snapshot to append async to the snapsho…

eaec04f

…t name to note if the image is `async` vs default

fixing browser for lint

68320f4

Merge branch 'dev' into support-async-callbacks

5b201a1

Merge remote-tracking branch 'remote/dev' into support-async-callbacks

1aea686

# Conflicts: # .circleci/config.yml # dash/_callback.py # dash/background_callback/managers/celery_manager.py # dash/dash.py

updating missing long -> background adjustments and fixing issue …

fc6c6bc

…for missing `callback_ctx`

fixing for lint

503f7c0

adding tests for async background callbacks

c26671e

updating to bypass test if not async

20dcd67

fixing for lint

bb3d881

update for celery app process

c6940c7

Merge branch 'dev' into support-async-callbacks

b7a8c4a

Merge branch 'dev' into support-async-callbacks

51c1689

Merge branch 'dev' into support-async-callbacks

eb96c5d

T4rk1n reviewed Jun 10, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

adding support for async callbacks and page layouts #3089

adding support for async callbacks and page layouts #3089

Uh oh!

BSd3v commented Nov 21, 2024

Uh oh!

ndrezn commented Apr 10, 2025

Uh oh!

BSd3v commented Apr 10, 2025

Uh oh!

T4rk1n left a comment

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

T4rk1n Jun 10, 2025

Uh oh!

Uh oh!

	def _get_callback_manager(kwargs, background):
	def _get_callback_manager(kwargs: dict, background: dict) -> Union[BaseBackgroundCallbackManager, None]:

Uh oh!

adding support for async callbacks and page layouts #3089

Are you sure you want to change the base?

adding support for async callbacks and page layouts #3089

Uh oh!

Conversation

BSd3v commented Nov 21, 2024

Uh oh!

ndrezn commented Apr 10, 2025

Uh oh!

BSd3v commented Apr 10, 2025

Uh oh!

T4rk1n left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!