Very Strange Triangulation of Segmentation, Cannot Globally Update Marker Size or Face Color

[jeffquinn-msk]

I tried out napari-spatialdata on a Xenium project and I noticed a few usability issues. I wonder if these are specific to me (maybe my workstation doesn't have sufficient resources? I have an M1 macbook with 16GB memory). Overall the program was very laggy/non-responsive so that makes me suspect a resource limitation.

The first thing is that the triangulation of the segmentation shapes seems kind of wonky. This is not at all what the nuclear segments look like in the Xenium browser. Is there some configuration parameter I can pass to get a higher poly rendering of the segments?

After that there were a few usability issues, like not being able to change the size of the transcript dots globally, however adjusting the opacity globally seemed to work fine.

cant_change_dot_size.mp4

It seems I need to actually select the dots to change their size, but when I try and use cmd-A to select them all the software crashed (hung indefinitely).

select_all_transcripts_crashes.mp4

Maybe these issues are not relevant to this software project directly but I'm just curious if something is horribly wrong with my installation or if this is expected. If I'm viewing a full Xenium slide should I maybe downsample to just one quadrant of the image maybe?

[LucaMarconato]

Hi @jeffquinn-msk thanks for reporting this. I'll answer point-by-point.

The first thing is that the triangulation of the segmentation shapes seems kind of wonky. This is not at all what the nuclear segments look like in the Xenium browser. Is there some configuration parameter I can pass to get a higher poly rendering of the segments?

@jeffquinn-msk to double check if this is a problem of napari and not of the Xenium dataset you are using, please plot single polygons using spatialdata-plot or the shapely plotting APIs. Xenium Explorer reads the polygons from the cells.zarr.zip file not from the .parquet files, so it could be that they are different.

If the above is not the case, @melonora do you know why the downsampling happens? I have never noticed this. We discussed adding this option in napari-spatialdata to mitigate the poor performance with shapes but I don't recall someone adding it in the end, so maybe this is happening at the napari side?

After that there were a few usability issues, like not being able to change the size of the transcript dots globally, however adjusting the opacity globally seemed to work fine.

This is the napari design, I also found it a bit confusing in the beginning but then I got used. If you could open a issue on napari on this I will also jump in the conversation please.

It seems I need to actually select the dots to change their size, but when I try and use cmd-A to select them all the software crashed (hung indefinitely).

For me this works, but selecting lot of points takes some time (napari limitation), I have a machine with higher specs so in my case it's still acceptable. I would consider reporting also this to napari please.

Maybe these issues are not relevant to this software project directly but I'm just curious if something is horribly wrong with my installation or if this is expected. If I'm viewing a full Xenium slide should I maybe downsample to just one quadrant of the image maybe?

For me even if points are not a problem, I can't use napari for rendering the full set of polygons if I first don't crop the data or remove vertices. This has been reported to napari before but unfortunately they currently don't have the bandwidth to work on this. We are exploring internally some ideas to mitigate this as part of the napari-spatialdata plugin, such as using multiscale vector representations or rasterizing the vector data.

Also, specific to Xenium data, we have now an updated version of the xenium() reader that reads the multiscale raster masks for the cells and nuclei. Maybe this would be useful for your use case (the table needs to be mapped to the labels before being able to plot the annotations). scverse/spatialdata-io#112. Please let me know if the performance is good for you in this case.

[melonora]

Hi @jeffquinn-msk,

A few of the issues you are having are seemingly related to parts of napari-core. We have a class called EventedSet that could be slowing down things when doing very large scale manipulations. Currentlly work is in progress to fix that.

Regarding the wonky segmentations. I wonder if this is due to the epsilon parameter of the rdp algorithm. In the napari menubar could you please go to File -> preferences and then the Experimental tab. Here you see the Shapes polygon lasso RDP epsilon parameter. Set this to 0 and please check if it resolves the issue. If not, I would have happy to jump in a call to try get a better understanding of your issue so we can resolve it.

[jeffquinn-msk]

Regarding the wonky segmentations. I wonder if this is due to the epsilon parameter of the rdp algorithm. In the napari menubar could you please go to File -> preferences and then the Experimental tab. Here you see the Shapes polygon lasso RDP epsilon parameter. Set this to 0 and please check if it resolves the issue. If not, I would have happy to jump in a call to try get a better understanding of your issue so we can resolve it.

I gave this a try but unfortunately it didn't seem to have any effect on the triangulation.

For me even if points are not a problem, I can't use napari for rendering the full set of polygons if I first don't crop the data or remove vertices.

In light of this comment I cropped my sdata with:

cropped_sdata = sdata.query.bounding_box(
  axes=["x", "y"],
  min_coordinate=[25401/1.7, 9560/1.7],
  max_coordinate=[26365/1.7, 10647/1.7],
  target_coordinate_system="global",
)
Interactive(cropped_sdata)

And now it looks great! Shapes look exactly as I expect (apart from that one shape being clearly drawn wrong..):

Note I had to write some manual code to iterate through all my shapes and remove any duplicate vertices or collinearities due to this issue (vispy/vispy#2247): https://github.com/tansey-lab/nuc2seg/blob/9fecc02d6d5236cca866b3c3a998ab4f80cd3d7e/src/nuc2seg/segment.py#L282

So im not sure why there would be any issues drawing them..

[jeffquinn-msk]

Also btw, spatialdata-plot produces the correct shapes

[jeffquinn-msk]

The issue with many points in a single layer being slow has already been filed here: napari/napari#6275

And apparently a fix was merged just last week in upstream project: pyapp-kit/psygnal#275, so maybe when dependencies are bumped to latest version it will be better

[melonora]

Yes this was indeed related to 'EventedSet' which uses that library