`ipykernel` 7.0.0 cannot send display data from threads

[jorgensd]

Minimal python code to put in notebook:

import logging

import ipyparallel as ipp


def hello_mpi():
    # We define all imports inside the function as they have to be launched on the remote engines
    from mpi4py import MPI

    print(f"Hello from rank {MPI.COMM_WORLD.rank}/{MPI.COMM_WORLD.size - 1}")


with ipp.Cluster(engines="mpi", n=3, log_level=logging.ERROR) as cluster:
    # We send the query to run the function `hello_mpi` on all engines
    query = cluster[:].apply_async(hello_mpi)
    # We wait for all engines to finish
    query.wait()
    # We check that all engines exited successfully
    assert query.successful(), query.error
    # We print the output from each engine
    print("".join(query.stdout))

Using conda to install the packages:

name: test_ipp
channels:
  - conda-forge
dependencies:
  - mpi4py
  - ipyparallel
  - python=3.12
  - ipykernel>=7.0.0
  - jupyter-book
  - ipywidgets
  - jupytext

results in

ERROR:tornado.general:Uncaught exception in ZMQStream callback
Traceback (most recent call last):
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/tqdm/notebook.py", line 262, in update
    return super().update(n=n)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/tqdm/std.py", line 1242, in update
    self.refresh(lock_args=self.lock_args)
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/tqdm/std.py", line 1347, in refresh
    self.display()
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/tqdm/notebook.py", line 157, in display
    pbar.value = self.n
    ^^^^^^^^^^
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/traitlets/traitlets.py", line 716, in __set__
    self.set(obj, value)
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/traitlets/traitlets.py", line 706, in set
    obj._notify_trait(self.name, old_value, new_value)
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/traitlets/traitlets.py", line 1513, in _notify_trait
    self.notify_change(
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/ipywidgets/widgets/widget.py", line 700, in notify_change
    self.send_state(key=name)
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/ipywidgets/widgets/widget.py", line 586, in send_state
    self._send(msg, buffers=buffers)
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/ipywidgets/widgets/widget.py", line 825, in _send
    self.comm.send(data=msg, buffers=buffers)
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/comm/base_comm.py", line 144, in send
    self.publish_msg(
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/ipykernel/comm/comm.py", line 42, in publish_msg
    parent=self.kernel.get_parent(),
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dokken/src/mambaforge/envs/test_ipp/lib/python3.12/site-packages/ipykernel/kernelbase.py", line 797, in get_parent
    return self._shell_parent.get()
           ^^^^^^^^^^^^^^^^^^^^^^^^
LookupError: <ContextVar name='shell_parent' at 0x7dd1814c94e0>

Only happens in notebooks, and if ipywidgets is installed, which might indicate that the problem lies elsewhere.

First observed in: https://github.com/jorgensd/adios4dolfinx/actions/runs/18489789453/job/52692118274
Work if one bounds ipykernel, i.e.

  - ipykernel<7.0.0

[minrk]

This is a bug in ipykernel, I'll look into it

[minrk]

simpler repro:

from threading import Thread
from IPython.display import display

def foo():
    display("x")

t = Thread(target=foo)
t.start()
t.join()

[minrk]

Fixed by #1451

[steveepreston]

when fix will arrive in release?
Does it solve huggingface/xet-core#526?