We need to deal with compatibility between this kit and CBFlib

[yayahjb]

This kit has a link to a 2018 CBFlib 0.9.6 kit. The 2021 CBFlib 0.9.7 kit i]with hdf5 1.12 support and many fewer warnings is nearly done. Keeping an obsolete version going is in nobody's interests. How do we resolve this both for the near term and the long term.

[graeme-winter]

@yayahjb the intention behind this was to contrbute towards conda install dials - we can conda install cctbx however this does not include the dependencies necessary for dxtbx. @ndevenish identified that within dials we only really depend on the pycbf bindings rather than directly on CBFlib -> made this release kit for conda based on the most recent release of CBFlib.

One of the key requirements of conda is that packages are made from releases rather than the current state of a branch - it is also the case that these should be as close as possible in name, intent etc. to the source of the code as possible hence naming this pycbf and versioning accordingly - it is my understanding that this is entirely within the spirit of both conda and the licensing used for CBFlib.

I would anticipate an update to this package following an updated release of CBFlib however if there are hard dependencies on HDF5 1.12 libraries this may prove to be more complex.

I hope this helps to clarify the situation. While this version may be obsolete it does allow conda install pycbf in a way which was previously impossible, so it is evidently in some people's interests.

[ndevenish]

Graeme is correct - a key requirement of conda-forge is that it build off of a direct release, rather than whatever the main branch commit is at the time. 0.9.6 is at time of writing this the current "official" version - so the only one that we can build from.

We'll of course be happy to update to 0.9.7 when it's available.

Meanwhile, python3 has never been officially supported by pycbf - there was last year some debian effort to get pycbf working (see dials/cbflib#18), but they ran into the same SWIG_PYTHON_STRICT_BYTE_CHAR issues that we did - dials/cbflib#19 and #2 (both the same changes) should fix this cleanly, but implicitly cause a minor API break if people have routed around the problem - like we did in cctbx/dxtbx@44b6d72. To maintain initial compatibility with 0.9.6, we decided not to put this in, though did put in #4 for some improvement - functions that take char * can now accept either bytes or str.

Regarding HDF5 - the pycbf subset without it seemed a good target for initial distribution - it's 100% certainty that if anyone is using it, they've compiled it themselves, and we've also never compiled the HDF5 parts in dials or cctbx - so starting with a working minimum seem reasonably safe. Since this is explicitly not intended for linking to in a compiled sense - there's no risk of anyone using it for that, and if/when a full conda-forge cbflib package is available we can just switch to linking to that directly, keeping the python/non-python parts separate. (the cbf_H5* functions have never been exposed in pycbf, but I don't know if the higher level library interacts with HDF5 directly).

As previously offered, we are happy to discuss the possibility of assisting with maintenance and porting.

[yayahjb]

Dear Graeme, Thanks for the clarification. I hope to have finished the cleanup of the 0.9.7 release of CBFlib, which is currently a branch in the repository by the end of June. The release will include a pycbf kit, making your current structure become circular if you just update the release version in your kit, and the handling of HDF5 is made tricky by the increasing adoption of 1.12 as a default system version, so CBFlib has to be adjusted to work both ways -- with a system hdf5 1.12 or with a downloaded 1.12. I would very much like to drop 1.8 completely and deprecate 1.10. I urge that all interested parties meet and try to work out a sensible plan to go forward, preferably without loops requiring manual intervention too frequently. At the very least, I could use help in making sure I pick up all features of your pycbf in the CBFlib pycbf without breaking python 2 support. Ideally what is in CBFlib should work transparently both in "obsolete" python 2.17 and "current" python 3.9. Regards, Herbert

…

On Sat, May 29, 2021 at 5:49 AM Graeme Winter ***@***.***> wrote: @yayahjb <https://github.com/yayahjb> the intention behind this was to contrbute towards conda install dials - we can conda install cctbx however this *does not* include the dependencies necessary for dxtbx. @ndevenish <https://github.com/ndevenish> identified that within dials we only really depend on the pycbf bindings rather than directly on CBFlib -> made this release kit for conda based on the most recent release of CBFlib. One of the key requirements of conda is that packages are made from *releases* rather than the current state of a branch - it is also the case that these should be as close as possible in name, intent etc. to the source of the code as possible hence naming this pycbf and versioning accordingly - it is my understanding that this is entirely within the spirit of both conda and the licensing used for CBFlib. I would anticipate an update to this package following an updated release of CBFlib however if there are hard dependencies on HDF5 1.12 libraries this may prove to be more complex. I hope this helps to clarify the situation. While this version may be obsolete it does allow conda install pycbf in a way which was previously impossible, so it is evidently in some people's interests. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABB6EAI6R4QXNDS5N3JWAXDTQC2BXANCNFSM45VEY2VA> .