Skip to content

n5 library version bumps #290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Apr 21, 2025
Merged

n5 library version bumps #290

merged 5 commits into from
Apr 21, 2025

Conversation

cmhulbert
Copy link
Contributor

bumps some recently released N5 library versions

  • n5
  • n5-aws-s3
  • n5-google-cloud
  • n5-ij
  • n5-zarr
  • n5-universe

@bogovicj bogovicj mentioned this pull request Apr 15, 2025
Closed
@ctrueden
Copy link
Member

Thanks @cmhulbert! From the CI log, it looks like only one n5-related integration failure: multiview-reconstruction.

Here is the relevant log 
[INFO] -------------------------------------------------------------
Error:  COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/fiji/spimdata/pointspreadfunctions/PointSpreadFunction.java:[32,53] cannot find symbol
  symbol:   class StorageFormat
  location: class org.janelia.saalfeldlab.n5.universe.N5Factory
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/fiji/plugin/resave/Resave_N5Api.java:[43,53] cannot find symbol
  symbol:   class StorageFormat
  location: class org.janelia.saalfeldlab.n5.universe.N5Factory
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/n5api/N5ApiTools.java:[46,53] cannot find symbol
  symbol:   class StorageFormat
  location: class org.janelia.saalfeldlab.n5.universe.N5Factory
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/n5api/N5ApiTools.java:[90,91] cannot find symbol
  symbol:   class StorageFormat
  location: class net.preibisch.mvrecon.process.n5api.N5ApiTools
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/n5api/N5ApiTools.java:[100,91] cannot find symbol
  symbol:   class StorageFormat
  location: class net.preibisch.mvrecon.process.n5api.N5ApiTools
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/n5api/N5ApiTools.java:[109,85] cannot find symbol
  symbol:   class StorageFormat
  location: class net.preibisch.mvrecon.process.n5api.N5ApiTools
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/n5api/N5ApiTools.java:[118,85] cannot find symbol
  symbol:   class StorageFormat
  location: class net.preibisch.mvrecon.process.n5api.N5ApiTools
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/n5api/N5ApiTools.java:[132,91] cannot find symbol
  symbol:   class StorageFormat
  location: class net.preibisch.mvrecon.process.n5api.N5ApiTools
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/n5api/N5ApiTools.java:[137,89] cannot find symbol
  symbol:   class StorageFormat
  location: class net.preibisch.mvrecon.process.n5api.N5ApiTools
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/n5api/N5ApiTools.java:[157,100] cannot find symbol
  symbol:   class StorageFormat
  location: class net.preibisch.mvrecon.process.n5api.N5ApiTools
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/fiji/spimdata/interestpoints/InterestPointsN5.java:[39,53] cannot find symbol
  symbol:   class StorageFormat
  location: class org.janelia.saalfeldlab.n5.universe.N5Factory
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/export/ExportN5Api.java:[44,53] cannot find symbol
  symbol:   class StorageFormat
  location: class org.janelia.saalfeldlab.n5.universe.N5Factory
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/n5api/SpimData2Tools.java:[45,53] cannot find symbol
  symbol:   class StorageFormat
  location: class org.janelia.saalfeldlab.n5.universe.N5Factory
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/export/ExportN5Api.java:[111,9] cannot find symbol
  symbol:   class StorageFormat
  location: class net.preibisch.mvrecon.process.export.ExportN5Api
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/n5api/SpimData2Tools.java:[91,31] cannot find symbol
  symbol:   class StorageFormat
  location: class net.preibisch.mvrecon.process.n5api.SpimData2Tools
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/net/preibisch/mvrecon/process/n5api/SpimData2Tools.java:[161,31] cannot find symbol
  symbol:   class StorageFormat
  location: class net.preibisch.mvrecon.process.n5api.SpimData2Tools
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/util/URITools.java:[51,53] cannot find symbol
  symbol:   class StorageFormat
  location: class org.janelia.saalfeldlab.n5.universe.N5Factory
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/util/URITools.java:[187,59] cannot find symbol
  symbol:   class StorageFormat
  location: class util.URITools
Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/net.preibisch/multiview-reconstruction/src/main/java/util/URITools.java:[223,59] cannot find symbol
  symbol:   class StorageFormat
  location: class util.URITools
[INFO] 19 errors

I checked against the current multiview-reconstruction codebase, and these new versions of n5 are working with it, so I guess the problems are already fixed. Therefore, I pushed an additional commit to this PR bringing the version of multiview-reconstruction (and multiview-simulation and BigStitcher) up to date. If the melting pot errors are gone afterward, I'll merge (even though the melting pot as a whole is not fully passing right now—the other errors are not n5's fault).

@ctrueden
Copy link
Member

Unfortunately, it failed again. The compile errors are different this time. It boils down to two problems:

net/preibisch/mvrecon/fiji/spimdata/imgloaders/AllenOMEZarrLoader.java:[32,18] cannot find symbol
  symbol:   class N5Properties
  location: package bdv.img.n5

and

src/main/java/net/preibisch/mvrecon/process/fusion/blk/Blending.java:[29,26] cannot find symbol
  symbol:   class BlockInterval
  location: package net.imglib2.blocks

The net.imglib2.blocks.BlockInterval class was added in imglib2 v7.1.4, violating SemVer.

The bdv.img.n5.N5Properties class was added in bigdataviewer-core v10.6.3, also violating SemVer.

I pushed another fix updating these versions accordingly here in pom-scijava.

@ctrueden
Copy link
Member

Still problems.

multiview-reconstruction:

... is not abstract and does not override abstract method copy(long[],java.lang.Object,int[]) in net.imglib2.algorithm.blocks.BlockSupplier

imglib2-algorithm: some failures, but the version here was out of date (0.16.0), so pushed another commit updating it (0.18.0).

labkit-ui:

Error:  Failures: 
Error:    HDF5SaverTest.testPartitionedWriting:73
Error:    HDF5SaverTest.testPartitionedWriting2:86
Error:    HDF5SaverTest.testSave2d:62
Error:    HDF5SaverTest.testSave4d:53

I feel like these labkit-ui errors came up before with N5, but my memory is fuzzy.

@bogovicj
Copy link
Contributor

Thanks @ctrueden !

At first glance that labkit error seems to not be N5, at least not directly. HDF5Saver seems to use bdv methods for saving, I think, eventually ending up at WritgeSequenceToHdf5.

Can't dig deeper right now I'm afraid.

@ctrueden
Copy link
Member

OK, now bigdataviewer-core is too new for the latest imglib2-algorithm:

Error:  .../net.imglib2/imglib2-algorithm/src/test/java/net/imglib2/algorithm/blocks/downsample/DownsampleBdvPlayground.java:[36,18] cannot find symbol
  symbol:   class DownsampleBlock
  location: package bdv.export

but bdv.export.DownsampleBlock was removed in bigdataviewer-core 10.6.2... violating SemVer... 😮‍💨

@tpietzsch I can replicate this failure by cloning imglib2-algorithm and then running mvn -Dbigdataviewer-core.version=10.6.4 clean test. So we either need to put back that class, or else change imglib2-algorithm not to use it anymore.

I cannot downgrade bigdataviewer-core from 10.6.3 back to 10.6.1 because bdv.img.n5.N5Properties is needed by multiview-reconstruction.

@tpietzsch @StephanPreibisch @axtimwalde How shall we proceed to reconcile these things?

To head off problems like this, I'd like to stop allowing direct uploads to the core Fiji update site, in favor of everything going through pom-scijava... so that we don't unleash these sorts of tangles on Fiji users without being aware of them first.

@tpietzsch
Copy link
Member

@ctrueden I removed the DownsampleBlock references and released imglib2-algorithm-0.18.1. I updated the imglib2-algorithm.version in pom-scijava accordingly.

Sorry for the mess... 🫤

@ctrueden
Copy link
Member

ctrueden commented Apr 21, 2025 •
edited
Loading

bigdataviewer-core has one error now:

Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/sc.fiji/bigdataviewer-core/src/main/java/bdv/export/ExportScalePyramid.java:[317,63] method copy in interface net.imglib2.algorithm.blocks.BlockSupplier<T> cannot be applied to given types;
Error:    required: net.imglib2.Interval,java.lang.Object
Error:    found: long[],capture#1 of ?,int[]
Error:    reason: actual and formal argument lists differ in length

and labkit-ui still has the four test failures:

Error:  Failures: 
Error:    HDF5SaverTest.testPartitionedWriting:73
Error:    HDF5SaverTest.testPartitionedWriting2:86
Error:    HDF5SaverTest.testSave2d:62
Error:    HDF5SaverTest.testSave4d:53

I'll try to dig more tomorrow, if nobody else does first.

@tpietzsch
Copy link
Member

bigdataviewer-core has one error now:

Error:  /home/runner/work/pom-scijava/pom-scijava/target/mega-melt/melting-pot/sc.fiji/bigdataviewer-core/src/main/java/bdv/export/ExportScalePyramid.java:[317,63] method copy in interface net.imglib2.algorithm.blocks.BlockSupplier<T> cannot be applied to given types;
Error:    required: net.imglib2.Interval,java.lang.Object
Error:    found: long[],capture#1 of ?,int[]
Error:    reason: actual and formal argument lists differ in length

This should be already fixed in bigdataviewer-core-10.6.4. I update the version in pom.xml

and labkit-ui still has the four test failures:

Error:  Failures: 
Error:    HDF5SaverTest.testPartitionedWriting:73
Error:    HDF5SaverTest.testPartitionedWriting2:86
Error:    HDF5SaverTest.testSave2d:62
Error:    HDF5SaverTest.testSave4d:53

This should be fixed with imglib2-7.1.5, which I just released. I update the version in pom.xml

@imagesc-bot
Copy link

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/fiji-friends-weekly-dev-update-thread/103718/87

ctrueden and others added 5 commits April 21, 2025 16:04
@ctrueden
Update scripting-clojure: 0.1.6 -> 1.0.0
1917850 
Which implicitly updates clojure: 1.8.0 -> 1.12.0,
which addresses CVE-2017-20189.
@cmhulbert @ctrueden
n5 library version bumps
fda7549
@cmhulbert @ctrueden
bump n5-ij
8852403
@cmhulbert @ctrueden
bump n5-hdf5
11c98b2
@ctrueden
Update BigStitcher and multiview-reconstruction
3f9d1c9 
* BigStitcher: 2.0.1 -> 2.5.0
* multiview-reconstruction: 4.1.1 -> 5.4.1
@ctrueden
Copy link
Member

Awesome, all the relevant tests are passing. Thank you all!

@ctrueden ctrueden merged commit 4fad676 into scijava:master Apr 21, 2025
1 check failed
@tpietzsch
Copy link
Member

🙏 thank you @ctrueden !

@cmhulbert
Copy link
Contributor Author

Thanks @ctrueden

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@cmhulbert @ctrueden @bogovicj @tpietzsch @imagesc-bot