Skip to content

Message decoding abc zoom in #54

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 64 commits into
base: main
Choose a base branch
from
Open

Message decoding abc zoom in #54

wants to merge 64 commits into from

Conversation

sunildkumar
Copy link
Member

No description provided.

ROIM1998 added 30 commits March 21, 2025 08:12
@ROIM1998
fully random mapping image generation
b62227f
@ROIM1998
Merge branch 'main' into message_decoding_zoom_in
d02fa88
@ROIM1998
revert the in-place change to the generate_decoder_image function
ccd138c
@ROIM1998
outline of the message_decoding_words zoom-in version dataset generation
1f66be3
@ROIM1998
find-grained control of difficulty level in decoder image generation
98a0192
@ROIM1998
tweak the hyperparameters for generating the decoder image
2e9b297
@ROIM1998
add num_small_positives and coordinates+font_size for all texts in th…
4beda67 
…e dataset
@ROIM1998
generating decoder images before adding underscore
8e02081
@ROIM1998
make sure not sampling larger than population
d3c165a
@ROIM1998
apply the same logic to the 26th character mapping
58c8748
@ROIM1998
make sure the bottom text is still on the image
7f826b6
@ROIM1998
add util function for zooming in
9e909fb
@ROIM1998
fix the zoomed-in image's resolution
084dcb3
@ROIM1998
move zoom_in to tools/
24b0e83
@ROIM1998
separate out the base class ImageHashTableTool for reusing
a13431f
@ROIM1998
update the zoom_in tool to use the image-based hash table to get coor…
4811f1b 
…dinates
@ROIM1998
add r1 dataset generation for the zoom-in message decoding task
1f09320
@ROIM1998
allow image return from tools being called
8e3d3e9
@ROIM1998
add env for message_decoding_zoom_in task
e8bb05e
@ROIM1998
reuse _get_step_count to name the tool-respond images
93715be
@ROIM1998
add training script for message_decoding_zoom_in
090febe
@ROIM1998
fix the bbox example to be JSON format
60a8b3c
@ROIM1998
fix the default dataset name to use the r1 version
5b6c10d
@ROIM1998
convert normalized bbox coordinates to absolute values
f34621e
@ROIM1998
fix the correctness reward to use the last assistant's response for m…
644e7b9 
…essage_decoding_zoom_in
@ROIM1998
fix the error where coordinates is not properly fetched from the hash…
aac65bc 
… table
@ROIM1998
enable gradient checkpointing and increase the limitation of num-imag…
5ef31c1 
…es-per-prompt
@ROIM1998
better docstrings and error logs for the zoom-in tool
e3e45f2
@ROIM1998
typo fix for limit_image_per_prompt
09325f5
@ROIM1998
resize the image if its size is not equal to that of the hash matrix
a741856
ROIM1998 added 30 commits March 31, 2025 09:58
@ROIM1998
do not truncate the completion_ids after the state is marked as complete
0585720
@ROIM1998
get correct tool response image index
8ccffd9
@ROIM1998
reduce the image resolution to 300
5df86f1
@ROIM1998
resolve conflicts and merge from main
4cd609c
@ROIM1998
add tool execution reward
d719c7d
@ROIM1998
resize the image if needed for hashing
39617ea
@ROIM1998
zoom-in according to the longer side to reduce memory consumption
3625f99
@ROIM1998
make sure the zoomed-in image follows the requirement of qwen2.5-vl's…
f9fe6f6 
… resizing strategy
@ROIM1998
set maximum aspect ratio for qwen2.5-vl smart_resize compatibility
08a3545
@ROIM1998
handle the downcast issue to underflow qwen_resize_factor
d61b364
@ROIM1998
use 'in' rather than 'startswith' to check tool execution success
b60ee6f
@ROIM1998
hyperparam to control the number of steps/images per conversation
0cc2f53
@ROIM1998
make sure the last assistant's meesage is <answer> but not <tool> to …
963dae4 
…receive a reward
@ROIM1998
add char- and answer-level intermediate rewards following the previou…
57eedbd 
…s message decoding runs
@ROIM1998
add <chars> field to be parsed
57e4d34
@ROIM1998
bug fixed
c6edd54
@ROIM1998
bring back the simple ABC message decoding task
76aa9e6
@ROIM1998
log full coordinates in the dataset
d962426
@ROIM1998
apply randomized position for the ABC mappings
35dc53f
@ROIM1998
apply the most recent path logic for images
17c1fa6
@ROIM1998
bug fixes
e8169b8
@ROIM1998
add r1-styled training/validation dataset with ABC task curation
cc84afe
@ROIM1998
log decoded message (ground truth)
4cd4144
@ROIM1998
Merge branch 'message_decoding_zoom_in' into message_decoding_abc_zoo…
a44519c 
…m_in
@ROIM1998
add zoom-in environment for the ABC message deocoding task (no chars …
cd8228b 
…reward)
@ROIM1998
use space-separated decoded message
8630252
@ROIM1998
refactor the intermediate correctness reward for the ABC task
8bc8897
@ROIM1998
add the training script for the message-decoding ABC task
8b7ff41
@ROIM1998
add zero3 config
19dc9d4
@ROIM1998
Merge branch 'message_decoding_zoom_in' into message_decoding_abc_zoo…
4f9a4f5 
…m_in
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sunildkumar @ROIM1998