Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GreenIT coding guidelines tutorial #9

Open
wants to merge 34 commits into
base: main
Choose a base branch
from
Open

Conversation

tromain
Copy link
Collaborator

@tromain tromain commented Feb 6, 2025

This tutorial's goal is to show good practices to develop an image processing chain in python, how to optimize the code, the size of the data, and monitor your application to find the bottlenecks and speed up your processing

@tromain tromain changed the title Add GreenIT coding guidelines tutorial GreenIT coding guidelines tutorial Feb 6, 2025
@ytanguy
Copy link
Collaborator

ytanguy commented Feb 12, 2025

Some comments

  • cell "work directories" : set a data_dir for inputs and a separate output_dir. Users should set the first as their favorite tile in datalake, and the second directory as a temp directory in their scratch for instance
  • the two cells about OTB should also declare the output_file as f"{output_dir}/img_ndvi_otb_py.tif" (for instance)
  • there are a lot of warnings about "RuntimeWarning: invalid value encountered in divide" : I think we can disable these warnings
  • cell "simple LZW compression" and "create a cog" : 'raster' is not defined => you should add a cell to define a path to a PHR image.
  • this cell could also measure elapsed time with %%time
  • be careful because this cell only writes 1 band : if the input image has multiple bands, you have to loop on the different bands (I did not find an other way to do it..)

@ytanguy
Copy link
Collaborator

ytanguy commented Feb 12, 2025

In compute RIOXarray cells, you don't define phr_product_lzw / _cog...

@tromain
Copy link
Collaborator Author

tromain commented Feb 13, 2025

In compute RIOXarray cells, you don't define phr_product_lzw / _cog...

They are defined in the work directories as "input_greenit/phr_product_lzw/cog.tif" , I generated them by hand using the code in the cell "compression", but maybe I could modify the code in this cell to take phr_product.tif as input and produce phr_product_lzw and cog instead of generating them outside the notebook

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants