Testing pyrs in the data viewer

[norlandrhagen]

No description provided.

[andersy005]

@norlandrhagen, do you have admin priviledges in the leap-stc organization? if so, you will need to install the leap-catalog app in this repo: https://github.com/apps/leap-stc-catalog/installations/56826523

otherwise, we will need @jbusecke's help

[norlandrhagen]

@andersy005 I just added this repo to the list.

[andersy005]

everything seems to be working fine except the data viewer :(.

@katamartin, do you happen to have some ideas on how to circumvent CORS issue (in data-viewer) when trying to visualize pyramids hosted on a server we don't control?

for e.g. https://ncview-js.staging.carbonplan.org/?dataset=https://nyu1.osn.mghpcc.org/leap-pangeo-pipeline/CHIRPS/CHIRPS_pyramid_5_lvls.zarr doesn;t work because of CORS issues.

P.S.

the raw dataset works because we are proxying everything through the zarr-proxy: https://ncview-js.staging.carbonplan.org/?dataset=https://nyu1.osn.mghpcc.org/leap-pangeo-pipeline/chirps_feedstock/chirps-global-daily.zarr

[katamartin]

Hmmm good question! We have already set up a serverless function to handle CORS issues for the initial .zmetadata request (the only request we need to make directly for zarr-proxy-ed utilities).

I think we should be able to extend that pattern and pass a proxied URL @carbonplan/maps to handle pyramid tile requests. Looking our Vercel billing, I think the cost of handling those requests for a few pyramids should be negligible. So I think the main issue would be the 4.5 MB request body limit, which means that this pattern would break for pyramids that contain large tiles (which they hopefully don't for visualization performance in general). Do you know if the size limit would affect this pyramid?

Taking a step back, it would be ideal to give guidance that CORS be enabled for visualization-ready pyramids. Since the pyramid optimization is especially helpful to browser environments.

[norlandrhagen]

Taking a step back, it would be ideal to give guidance that CORS be enabled for visualization-ready pyramids. Since the pyramid optimization is especially helpful to browser environments.

To add a bit, I am storing the pyramid on LEAP's OSN bucket, which I have no idea how CORS would works for that.

[andersy005]

Taking a step back, it would be ideal to give guidance that CORS be enabled for visualization-ready pyramids. Since the pyramid optimization is especially helpful to browser environments.

➕ ....

@jbusecke, does OSN expose a web console to the admins? If so, it is worth investigating to see if we can enable CORS similar to how AWS exposes this feature: https://docs.aws.amazon.com/AmazonS3/latest/userguide/enabling-cors-examples.html