-
Notifications
You must be signed in to change notification settings - Fork 388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add DL4GAMAlps dataset #2508
base: main
Are you sure you want to change the base?
Add DL4GAMAlps dataset #2508
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for the contribution :) I hope this first round of review is able to resolve the import xarray
error that is preventing the unit tests from running. Once they are working, will take another look for anything else!
@microsoft-github-policy-service agree |
I hope it will be somehow useful. Happy to contribute and learn new things on the way :) |
I'm completely fine with adding xarray and netcdf4 as optional dependencies. We've been talking about adding xarray support for a long time. We're still not sure if it will be directly through xarray or something like rioxarray, but that doesn't matter for this PR. |
Can you resolve the merge conflicts so the tests run? |
Can you resolve the merge conflicts? These requirements files unfortunately get updated a lot, causing merge issues. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code looks fantastic, just a few minor formatting comments. Thanks for taking the time to contribute your dataset to TorchGeo, let's hope it makes it easier for people to actually use and do cool science with!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also need to update requirements/min-reqs.old
with the minimum supported versions so we can test those.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added the (current) minimum versions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like the mininum
test is failing now, need an earlier version of xarray. Could you please explain briefly how this works or otherwise help me with it? Should I create an environment and then install only what's in requirements/min-reqs.old and then look for an xarray version that passes my dataset's tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, you can do that locally, or you can try different versions in CI (a bit slower). I usually do some kind of binary search to find the minimum working version. It's kinda painful, but we don't usually have to do it too often, so I haven't found a better solution.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm also happy to help debug this in CI for you if you get stuck.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Found a much older version. Should be fine now :)
This PR adds a a Multi-modal Dataset for Glacier Mapping (Segmentation) in the European Alps.
The dataset consists of Sentinel-2 images from 2015 (mainly), 2016 and 2017, and binary segmentation masks for glaciers, based on an inventory built by glaciology experts (Paul et al. 2020).
Given that glacier ice is not always visible in the images, due to seasonal snow, shadow/cloud cover and, most importantly, debris cover, the dataset also includes additional features that can help in the segmentation task.
Here's a sample extracted with the plot function implemented in the dataset:
A preprint is available here that describes in detail how the dataset was constructed.
For a shorter description check also: https://huggingface.co/datasets/dcodrut/dl4gam_alps.