-
Notifications
You must be signed in to change notification settings - Fork 3
Initial Experiment for Mozambique LULC #207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
yawenzzzz
wants to merge
47
commits into
master
Choose a base branch
from
yawenz/20250917_mozambique
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
47 commits
Select commit
Hold shift + click to select a range
8cb4415
add mozambique runs
yawenzzzz fbb9267
change to descending
yawenzzzz de56e2e
linting
gabrieltseng 78fd6fb
tmp
gabrieltseng 2920e99
add mozambique es run configs
gabrieltseng b9ee3b1
class to crop_type_classification
gabrieltseng dd939f4
Merge branch 'master' into yawenz/20250917_mozambique
gabrieltseng 6bc59d1
new beaker image means this is no longer necessary
gabrieltseng 2090fd8
create label raster for mozambique
gabrieltseng 5fa632b
use segmentation for finetuning
gabrieltseng 86532c1
actually its not in the newest config yet
gabrieltseng 4710c0c
Add more metrics
gabrieltseng ed24d07
:facepalm:
gabrieltseng 4d56262
lets ignore this for now
gabrieltseng 4228443
Update label in yaml
gabrieltseng 4f55680
incremental fixes
gabrieltseng 6849d40
tmp
gabrieltseng e1cfa71
Have to crop the labels too
gabrieltseng e2b3829
its just label
gabrieltseng 474578e
cargo culting
gabrieltseng 7e147fe
update es run model config to reflect new changes
gabrieltseng 96d566c
fix typo
gabrieltseng ef8a1b6
I don't think these keys are necessary in the es run config?
gabrieltseng 3527240
Add comment about necessary upload
gabrieltseng d4e74d7
We need to encode zeros as invalid
gabrieltseng f648484
We are missing a comma
gabrieltseng fa48b31
zero is invalid with the new rasters
gabrieltseng 9954dde
we have +1 classes
gabrieltseng a5db79b
oops
gabrieltseng ebc7a05
update es run yamls for mozambique
gabrieltseng 939776b
BIGGER
gabrieltseng f726b84
the new images need an extra /
gabrieltseng 10e92b3
SMALLER
gabrieltseng 8cc6cc8
smaller still
gabrieltseng 41fac26
Merge branch 'master' into yawenz/20250917_mozambique
gabrieltseng b7ccffb
Finetune s2 only
gabrieltseng ea7c1be
Reduce grid size
gabrieltseng e8a68b1
Remove sentinel1 from yaml
gabrieltseng ae09f9e
the beaker image has been updated
gabrieltseng 0692d8c
Update with main
gabrieltseng a693174
move out of crop folder
gabrieltseng 55300d5
update path to segmentation pooling decoder
gabrieltseng ee68b73
helios -> olmoearth
gabrieltseng 1450205
Add notes, add Gaza geometry
gabrieltseng f0bce22
predict 16x16 at inference time
gabrieltseng b39bada
fix task name
gabrieltseng 9e90c37
Reduce batch size
gabrieltseng File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,250 @@ | ||
| model: | ||
| class_path: rslearn.train.lightning_module.RslearnLightningModule | ||
| init_args: | ||
| model: | ||
| class_path: rslearn.models.multitask.MultiTaskModel | ||
| init_args: | ||
| encoder: | ||
| - class_path: rslearn.models.olmoearth_pretrain.model.OlmoEarth | ||
| init_args: | ||
| checkpoint_path: /weka/dfive-default/helios/checkpoints/henryh/base_v6.1_add_chm_cdl_worldcereal/step500000 | ||
| selector: ["encoder"] | ||
| forward_kwargs: | ||
| patch_size: 1 | ||
| decoders: | ||
| crop_type_classification: | ||
| - class_path: rslp.nandi.train.SegmentationPoolingDecoder | ||
| init_args: | ||
| in_channels: 768 | ||
| out_channels: 8 | ||
| - class_path: rslearn.train.tasks.segmentation.SegmentationHead | ||
| lr: 0.0001 | ||
| scheduler: | ||
| class_path: rslearn.train.scheduler.PlateauScheduler | ||
| init_args: | ||
| factor: 0.2 | ||
| patience: 2 | ||
| min_lr: 0 | ||
| cooldown: 10 | ||
| data: | ||
| class_path: rslearn.train.data_module.RslearnDataModule | ||
| init_args: | ||
| path: /weka/dfive-default/rslearn-eai/datasets/crop/mozambique_lulc | ||
| inputs: | ||
| sentinel2_l2a: | ||
| data_type: "raster" | ||
| layers: ["sentinel2"] | ||
| bands: ["B02", "B03", "B04", "B08", "B05", "B06", "B07", "B8A", "B11", "B12", "B01", "B09"] | ||
| passthrough: true | ||
| dtype: FLOAT32 | ||
| load_all_item_groups: true | ||
| load_all_layers: true | ||
| sentinel1: | ||
| data_type: "raster" | ||
| layers: ["sentinel1_descending"] | ||
| bands: ["vv", "vh"] | ||
| passthrough: true | ||
| dtype: FLOAT32 | ||
| load_all_item_groups: true | ||
| load_all_layers: true | ||
| label: | ||
| data_type: "raster" | ||
| layers: ["label_raster"] | ||
| bands: ["label"] | ||
| is_target: true | ||
| dtype: INT32 | ||
| task: | ||
| class_path: rslearn.train.tasks.multi_task.MultiTask | ||
| init_args: | ||
| tasks: | ||
| crop_type_classification: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationTask | ||
| init_args: | ||
| num_classes: 8 | ||
| zero_is_invalid: true | ||
| metric_kwargs: | ||
| average: "micro" | ||
| other_metrics: | ||
| water_precision: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassPrecision | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 1 | ||
| water_recall: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassRecall | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 1 | ||
| bareground_precision: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassPrecision | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 2 | ||
| bareground_recall: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassRecall | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 2 | ||
| rangeland_precision: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassPrecision | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 3 | ||
| rangeland_recall: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassRecall | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 3 | ||
| floodedvegetation_precision: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassPrecision | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 4 | ||
| floodedvegetation_recall: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassRecall | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 4 | ||
| trees_precision: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassPrecision | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 5 | ||
| trees_recall: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassRecall | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 5 | ||
| cropland_precision: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassPrecision | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 6 | ||
| cropland_recall: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassRecall | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 6 | ||
| buildings_precision: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassPrecision | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 7 | ||
| buildings_recall: | ||
| class_path: rslearn.train.tasks.segmentation.SegmentationMetric | ||
| init_args: | ||
| metric: | ||
| class_path: torchmetrics.classification.MulticlassRecall | ||
| init_args: | ||
| num_classes: 8 | ||
| average: null | ||
| class_idx: 7 | ||
| input_mapping: | ||
| crop_type_classification: | ||
| label: "targets" | ||
| batch_size: 32 | ||
| num_workers: 32 | ||
| default_config: | ||
| transforms: | ||
| - class_path: rslearn.models.olmoearth_pretrain.norm.OlmoEarthNormalize | ||
| init_args: | ||
| band_names: | ||
| sentinel2_l2a: ["B02", "B03", "B04", "B08", "B05", "B06", "B07", "B8A", "B11", "B12", "B01", "B09"] | ||
| sentinel1: ["vv", "vh"] | ||
| - class_path: rslearn.train.transforms.pad.Pad | ||
| init_args: | ||
| size: 4 | ||
| mode: "center" | ||
| image_selectors: ["sentinel2_l2a", "sentinel1", "target/crop_type_classification/classes", "target/crop_type_classification/valid"] | ||
| train_config: | ||
| groups: ["gaza"] | ||
| tags: | ||
| split: "train" | ||
| val_config: | ||
| groups: ["gaza"] | ||
| tags: | ||
| split: "test" | ||
| test_config: | ||
| groups: ["gaza"] | ||
| tags: | ||
| split: "test" | ||
| trainer: | ||
| max_epochs: 100 | ||
| callbacks: | ||
| - class_path: lightning.pytorch.callbacks.LearningRateMonitor | ||
| init_args: | ||
| logging_interval: "epoch" | ||
| - class_path: lightning.pytorch.callbacks.ModelCheckpoint | ||
| init_args: | ||
| save_top_k: 1 | ||
| save_last: true | ||
| monitor: val_loss | ||
| mode: min | ||
| - class_path: rslearn.train.callbacks.freeze_unfreeze.FreezeUnfreeze | ||
| init_args: | ||
| module_selector: ["model", "encoder", 0] | ||
| unfreeze_at_epoch: 20 | ||
| unfreeze_lr_factor: 10 | ||
| # - class_path: rslearn.train.prediction_writer.RslearnWriter | ||
| # init_args: | ||
| # path: /weka/dfive-default/rslearn-eai/datasets/crop/mozambique_lulc | ||
| # output_layer: prediction_v0 | ||
| # selector: ["crop_type_classification"] | ||
| # merger: | ||
| # class_path: rslearn.train.prediction_writer.RasterMerger | ||
| # init_args: | ||
| # padding: 2 | ||
| rslp_project: 2025_09_18_mozambique_lulc | ||
| rslp_experiment: mozambique_lulc_helios_base_S1_S2_ts_ws4_ps1_gaza | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It makes sense that we need to pad (or crop) both the inputs and the targets - the naming convention for the targets here is confusing to me.