Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not write unused VDS files in the first place #87

Open
d-perl opened this issue Nov 30, 2022 · 3 comments
Open

Do not write unused VDS files in the first place #87

d-perl opened this issue Nov 30, 2022 · 3 comments

Comments

@d-perl
Copy link
Contributor

d-perl commented Nov 30, 2022

When, in artemis, we have a large grid, there are more than two datasets linked in the nexus files, and for the second nexus file, the first dataset may not be used. In that case,

https://github.com/DiamondLightSource/python-dlstbx/blob/main/src/dlstbx/swmr/h5check.py

fails when running

plist = dataset.id.get_create_plist()
...
vspace = plist.get_virtual_vspace(0)
frames = vspace.get_regular_hyperslab()[3][0]

because that dataset has no corresponding slab of the vds.

I would like it if completely unused datasets are not linked in the nexus file.

@d-perl
Copy link
Contributor Author

d-perl commented Dec 6, 2022

Turns out the issue is not just that it is linked but that it is actually used as a source for the VDS as well

d-perl added a commit to d-perl/nexgen that referenced this issue Dec 7, 2022
d-perl added a commit to d-perl/nexgen that referenced this issue Dec 7, 2022
noemifrisina pushed a commit that referenced this issue Dec 7, 2022
* #87 Don't use unused datasets as VDS source
and function to clean unused links

* #87 fix tests

* update changelog
@DominicOram
Copy link
Collaborator

DominicOram commented Dec 8, 2022

#92 fixes this issue and is good as an urgent fix required for the beamline but I think we could do more:

  • This will strip any files out at the start of the datasets but not at the end (we currently do this inside Artemis itself but makes more sense to do in nexgen)
  • Adding the files in then cleaning them up later is a bit messy, we can probably tidy it to never add them
  • More tests around this would probably be good

@DominicOram
Copy link
Collaborator

This is still not fixed properly, and should be looked into

@DominicOram DominicOram changed the title Unused linked dataset causes issues with dlstbx Do not write unused VDS files in the first place May 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants