Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for CANDELS catalog and multiresolution #30

Open
wants to merge 150 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
150 commits
Select commit Hold shift + click to select a range
954ab42
first (buggued) version of candels
Aug 11, 2020
c95b783
add candels to the possible problems
Aug 11, 2020
a519d35
new release
Oct 23, 2020
bc84c2e
working RNG
Oct 29, 2020
f1a21d2
Added multiresolution support
Feb 18, 2021
d4d2e24
Correct typo
Feb 18, 2021
647304a
Added candels to problems
Feb 18, 2021
d404c27
Correct typo
Feb 18, 2021
c5c9ec7
Change parameters for testing
Feb 19, 2021
640b057
Correct variable names
Feb 19, 2021
685667d
changed tmp_dir to data_dir
Feb 19, 2021
d43e955
Correct PSF size
Feb 19, 2021
3cf0f0a
Correct PSF size
Feb 19, 2021
3e564c2
Correct image size
Feb 19, 2021
8851553
Missing import
Feb 19, 2021
9df2cc2
Fix issue with sigmas
Feb 19, 2021
c10e2c2
Hopefully fixing shape problem
Feb 19, 2021
91de7ac
Really fixing size issue
Feb 19, 2021
dc7ed7a
Integer error
Feb 19, 2021
6025147
Typo
Feb 19, 2021
0ecd0a4
Typo
Feb 19, 2021
3ca91fe
Typo
Feb 19, 2021
68b8de9
Typo
Feb 19, 2021
9851215
Fix filter drawing
Feb 19, 2021
0bc2415
Troubleshooting galaxy number
Feb 20, 2021
63f4380
Put a higher number of galaxies per shard
Feb 20, 2021
cd5a12d
Fix the inconsistant size
Feb 20, 2021
9a047ce
Get the right base resolution ?
Feb 22, 2021
7241b4b
Small changes
Feb 22, 2021
ab94a46
CHanged parameters and added some prints
Feb 22, 2021
0807b86
Get the right resolution
Feb 22, 2021
ddab0d2
Fix typo
Feb 22, 2021
09af55d
Fix typo
Feb 22, 2021
83fea88
Fix rescaling problem
Feb 22, 2021
60dbc55
FIx typo
Feb 22, 2021
1b16371
Fix resolutions and correct noise level and new blending definition
Feb 22, 2021
d1d694e
Correct noise
Feb 23, 2021
70e0527
Add missing noise
Feb 23, 2021
faaead3
Commented possibly problematic lines
Feb 23, 2021
34fee53
Handled multi channel PSF convolution and changed resolutions to 0.06…
Feb 24, 2021
5f070aa
Added power spectrum computation to CANDELS (probably not good)
Feb 24, 2021
7ab7e0f
Ensure power spectrum is properly exported
Feb 24, 2021
152f593
Fix dimension problem
Feb 24, 2021
5126f95
Fix dimension issues
Feb 25, 2021
4a3f246
Fix power spectrum error
Feb 25, 2021
2a73024
Fix typo
Feb 25, 2021
6e836b6
Add missing transposition for power spectrum
Feb 25, 2021
a1dd1e8
Made loss multiband
Feb 26, 2021
e60cf53
Fix issue with shards
Feb 28, 2021
ba46fb6
Clean up to try avoiding the error
Mar 1, 2021
3a46464
Fix exporter
Mar 1, 2021
e27cbfe
Fix exporter
Mar 1, 2021
b51366b
Fix exporter
Mar 1, 2021
2af4a42
Fix hub pixel scale error
Mar 1, 2021
8ef6cbd
Made use of task id
Mar 3, 2021
aebc0bc
Fixed fields and changed params
Mar 4, 2021
f1a6e71
Changing the main filter
Mar 4, 2021
4dceda8
Fix empty catalog
Mar 4, 2021
4874891
CHeck task id
Mar 4, 2021
58f7bc5
Fix typo
Mar 4, 2021
4dff64c
Modify code for task_id
Mar 4, 2021
08ff2c9
Add multiprocessing to candels
Mar 4, 2021
d9a8fd9
Fix indexes
Mar 4, 2021
691524e
Fix problem with field num
Mar 4, 2021
c9733e3
Fix weird summary error
Mar 5, 2021
71e44bc
Add all fields and correct typo
Mar 7, 2021
718c198
Change shard number
Mar 8, 2021
52ee908
Change task_id thing
Mar 8, 2021
0089b61
Fix task_id
Mar 8, 2021
967a052
typo
Mar 8, 2021
c1ddaaa
force int
Mar 8, 2021
2a19c36
Corrected mistake
Mar 8, 2021
bda47e5
Change noise treatment
Mar 9, 2021
99e9b08
Fix wrong shape with single band
Mar 10, 2021
5d4844c
Fix wrong shape with single band
Mar 10, 2021
930d2e6
Fix wrong shape with single band
Mar 10, 2021
2decc43
Fix export error
Mar 10, 2021
4dca50c
Fix export error
Mar 10, 2021
a9335d3
New attrs2img
Mar 10, 2021
067589c
Added multiband cosmos
Mar 11, 2021
2831ea1
Fix typo
Mar 11, 2021
1e5c7dd
Fix typo
Mar 11, 2021
f0be20f
Fix typo
Mar 11, 2021
4081c81
Test fix error
Mar 11, 2021
d798e87
Test fix error
Mar 11, 2021
63731f2
Test fix error
Mar 11, 2021
34483ff
Test fix error
Mar 11, 2021
0207770
Test fix error
Mar 11, 2021
6103013
Test fix error
Mar 11, 2021
321394d
Added some prints
Mar 12, 2021
b3fcc7c
Changed print
Mar 14, 2021
44b43d2
Squeezed to see if it fixes error
Mar 15, 2021
4a06be6
Fixed some parameters
Mar 15, 2021
3e74712
Typo
Mar 15, 2021
428f2a7
Fix the damn type bug
Mar 16, 2021
3e54665
Data augmentation
Mar 22, 2021
e289595
Removed useless things in candels and added random rotation to cosmos
Mar 23, 2021
81b3a5f
Merge remote-tracking branch 'upstream/master'
Mar 24, 2021
f76c5a0
Correct psf shape
Mar 24, 2021
ca8aac7
Try to solve issue with psf
Mar 24, 2021
914670a
Try to solve issue with psf
Mar 24, 2021
d9a7b63
Try to solve issue with psf
Mar 24, 2021
56a0b46
Try to solve issue with psf
Mar 24, 2021
3c5c10c
Try to solve issue with psf
Mar 24, 2021
963a5d1
Update
Mar 30, 2021
a923779
Updated PSF for candels
thuiop May 3, 2021
d6e836e
missing coma
thuiop May 3, 2021
6954bc5
Fix error
thuiop May 3, 2021
5949104
Fix psf
thuiop May 3, 2021
f7bae01
Fix psf
thuiop May 3, 2021
8cb2139
Import galsim
thuiop May 3, 2021
0b67d35
Fix typo
thuiop May 3, 2021
89cf2d4
Fix psf
thuiop May 3, 2021
94ca1b0
Fix psf
thuiop May 3, 2021
a98ed51
Added magnitude cut and fixed attributes
thuiop May 5, 2021
93419cb
Fix psf scale
thuiop May 5, 2021
cc903f7
Remove images missing a galaxy (hopefully)
thuiop May 6, 2021
54c5369
Correction
thuiop May 6, 2021
ec2e1fa
Removed rotation candels
thuiop May 7, 2021
36e3e4b
Fix sep
thuiop May 7, 2021
6531333
Changed noise power spectrum
thuiop May 11, 2021
14ae526
Fix Nan error
thuiop May 11, 2021
b442855
Added new images for tensorboard
thuiop May 12, 2021
b27ecfa
All channels to image summary
thuiop May 13, 2021
ff48c72
Normalize
thuiop May 14, 2021
54b2591
Rescale to correct flux
thuiop May 17, 2021
8452ed4
Added images for flow
thuiop Jun 1, 2021
e458d00
Added images for flow
thuiop Jun 1, 2021
d649f7e
Added images for flow
thuiop Jun 1, 2021
e622bb5
Changed normalization and excluded some placeholder values in catalog
thuiop Jun 3, 2021
a7e79e4
Changed a few things
thuiop Jun 9, 2021
ebc6ab6
Added prints
thuiop Jun 9, 2021
6d4912e
Removing the problematic galaxies (?)
thuiop Jun 10, 2021
91393c7
Added redshift as an attribute
thuiop Jun 17, 2021
ba15115
Normalization
thuiop Jun 17, 2021
85f2309
Added problem with blue band
thuiop Jun 22, 2021
1c69746
Normalize COSMOS
thuiop Jun 22, 2021
0310f0c
Correct mistake
thuiop Jun 24, 2021
3d63331
Added possibility to change conditionning parameters for flow
thuiop Jun 29, 2021
0f8bc4f
Fixed attribute param
thuiop Jun 29, 2021
1606ae2
Fixed attribute param
thuiop Jun 29, 2021
160e6ac
Fixed changing the flow attributes and added random flux for cosmos m…
thuiop Jul 14, 2021
ce4e760
Rename class and fix detecting central galaxy
thuiop Jul 20, 2021
da7c3e5
Change hardcoded params to relative
thuiop Jul 20, 2021
aa32411
Fixed error and minor adjustments
thuiop Jul 20, 2021
8d0b578
Correct typo
thuiop Jul 20, 2021
ee84d8c
Remove useless file
thuiop Jul 26, 2021
ff19206
Cleanup
thuiop Jul 26, 2021
a878dd0
Cleaning
thuiop Jul 27, 2021
de73237
Remove useless changes
thuiop Jul 27, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions galaxy2galaxy/bin/g2g-exporter
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ from tensor2tensor.utils import decoding
from tensor2tensor.utils import t2t_model
from tensor2tensor.utils import trainer_lib
from tensor2tensor.utils import usr_dir
from tensor2tensor.utils import registry

from galaxy2galaxy import models
from galaxy2galaxy import problems
Expand Down Expand Up @@ -101,7 +102,12 @@ def main(_):

estimator = create_estimator(run_config, hparams)
hparams.img_len = problem.get_hparams().img_len
hparams.attributes = problem.get_hparams().attributes
try:
if len(hparams.attributes) == 0:
hparams.attributes = hparams.problem.get_hparams().attributes
except:
pass


# Use tf hub to export any module that has been registered
exporter = hub.LatestModuleExporter("tf_hub",
Expand All @@ -115,4 +121,4 @@ def main(_):

if __name__ == "__main__":
tf.logging.set_verbosity(tf.logging.INFO)
tf.app.run()
tf.app.run()
1 change: 1 addition & 0 deletions galaxy2galaxy/data_generators/all_problems.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
try:
import galsim
MODULES += ["galaxy2galaxy.data_generators.cosmos"]
MODULES += ["galaxy2galaxy.data_generators.candels"]
except:
print("Could not import GalSim, excluding some data generators")

Expand Down
Loading