About code errors

[jiewjwang]

The code in your latest update has many errors, and the video generated by the script “python -m sample.local_generate_obj --model_path ./save/behave_enc_512/model000020000.pt --num_samples 10 --num_repetitions 1 --motion_length 10 --multi_backbone_split 4 --guidance” is completely blank, unlike the previous version which could generate interactive videos. I would really appreciate any help in identifying the potential problem.

[sunshineatnoon]

Same here, I was able to generate something like this after fixing many errors in the code, but I believe this is not the correct results

sample00_rep00.mp4

[jiewjwang]

After I fixed the error message, the generated animation was completely blank, and I'm not sure what went wrong. This code was very helpful for my research, and I'm especially looking forward to the author fixing it.

[ymingxie]

Hi, we recently reorganized the code, especially the dataset part, as certain parts violated the data policy. We are currently focused on an upcoming deadline and plan to address this issue after November 13 or November 20. Thank you for your patience!

[jiewjwang]

Thanks a lot for the update! Appreciate all the hard work you’re putting into maintaining the repo.
I’ll check back after mid-November.
Good luck with the upcoming deadline!

[sunshineatnoon]

Hi @ymingxie Thanks for the quick response. I think I fixed everything but would like to double check. Could you please let me know if these results look reasonable to you?

https://github.com/user-attachments/assets/30613b30-4430-4c6f-ad5f-a7cee5efc4da
https://github.com/user-attachments/assets/3963a1d7-2cdb-46c2-8954-e6e5123a1bf7
https://github.com/user-attachments/assets/6b7e8719-6d23-4307-a791-94b04295ab12
https://github.com/user-attachments/assets/faa46531-23a6-4169-a636-6ca17a2ff573
https://github.com/user-attachments/assets/cd705bf4-cd9b-4da8-bade-8d52e97a2674
https://github.com/user-attachments/assets/a2762ccd-af72-4578-8335-2329f85c8707
https://github.com/user-attachments/assets/cf9a49f7-97d5-4d39-9aba-ab69e1857a87
https://github.com/user-attachments/assets/e75d0f89-8d24-4ca1-942e-dde54d57c169
https://github.com/user-attachments/assets/5ee1826a-1be4-4fc5-84c7-4050850a7561
https://github.com/user-attachments/assets/dcf377bb-377f-4b1f-a211-9bea542084a5

[jiewjwang]

@sunshineatnoon Hi, your results look great! Could you please share what changes you made to get them?I really appreciate your work.
thanks in advance!

[sunshineatnoon]

@jiewjwang I roughly noted all the changes I made as follows for tracking purpose, this might not be all.

Basically if I found anything is missing or not working, I went back to older commit and compare the code.

mkdir data_loaders/behave
mkdir data_loaders/behave/scripts
cp ../HOI-Diff-repo/HOI-Diff-old/data_loaders/behave/scripts/motion_process.py data_loaders/behave/scripts/
mkdir data_loaders/behave/common
cp ../HOI-Diff-repo/HOI-Diff-old/data_loaders/behave/common/skeleton.py data_loaders/behave/common/
cp ../HOI-Diff-repo/HOI-Diff-old/data_loaders/behave/common/quaternion.py data_loaders/behave/common/
mkdir data_loaders/behave/utils
cp ../HOI-Diff-repo/HOI-Diff-old/data_loaders/behave/utils/paramUtil.py data_loaders/behave/utils/

# add simplified_mesh to visualize/vis_utils.py

cp -r ../HOI-Diff-repo/HOI-Diff-old/glove/ ./
model/afford_est.py -> line 35: from model.points_encoder import PointNet2Encoder

model/afford_est.py -> line 17 & 18
17: self.njoints = 15
18: self.nfeats = 6

sample/condition.py 
-> line 44: loss, grad, loss_list = self.gradients(x, t, self.afford_sample, y['obj_points'])
-> line 49: gradients(self, x, t, afford_sample, obj_points)
-> line 109: delete

sample/local_generate_obj.py
-> delete line 162, 171
-> line 188: all_obj_name.append(model_kwargs['y']['seq_name'])
-> delete line 189
-> line 212: 'num_samples': args.num_samples, 'num_repetitions': args.num_repetitions, 'obj_name': all_obj_name}
-> line 242: obj_name = all_obj_name[0][rep_i*args.batch_size + sample_i].split('_')[2]

sample/local_generate_obj.py 
-> line 81: #model_kwargs['y']['obj_normals'] = model_kwargs['y']['obj_normals'].to(dist_util.dev())
-> line 111: skip_timesteps=0
-> line 145: skip_timesteps=0
-> line 238: all_vertices = temp_simp.vertices
            center = np.mean(all_vertices, 0)
            all_vertices -= center

sample/condition.py
-> 105:  # center
        vertices = obj_points[i][:-2,:].float()
        center = torch.mean(vertices, 0)
        vertices -= center

utils/plot_script.py
-> line 5: fix ffmpeg path
 
python -m sample.local_generate_obj --model_path ./save/behave_enc_512/model000020000.pt --num_samples 10 --num_repetitions 1 --motion_length 10 --multi_backbone_split 4 --guidance

HOI-Diff-old refers to an older commit

[jiewjwang]

@sunshineatnoon Thank you so much for sharing your notes and the changes you made!
That’s really helpful — I’ll go through them and adjust my setup accordingly. Appreciate your time!

[ymingxie]

Hi @ymingxie Thanks for the quick response. I think I fixed everything but would like to double check. Could you please let me know if these results look reasonable to you?

https://github.com/user-attachments/assets/30613b30-4430-4c6f-ad5f-a7cee5efc4da https://github.com/user-attachments/assets/3963a1d7-2cdb-46c2-8954-e6e5123a1bf7 https://github.com/user-attachments/assets/6b7e8719-6d23-4307-a791-94b04295ab12 https://github.com/user-attachments/assets/faa46531-23a6-4169-a636-6ca17a2ff573 https://github.com/user-attachments/assets/cd705bf4-cd9b-4da8-bade-8d52e97a2674 https://github.com/user-attachments/assets/a2762ccd-af72-4578-8335-2329f85c8707 https://github.com/user-attachments/assets/cf9a49f7-97d5-4d39-9aba-ab69e1857a87 https://github.com/user-attachments/assets/e75d0f89-8d24-4ca1-942e-dde54d57c169 https://github.com/user-attachments/assets/5ee1826a-1be4-4fc5-84c7-4050850a7561 https://github.com/user-attachments/assets/dcf377bb-377f-4b1f-a211-9bea542084a5

Yes, these results look reasonable to me. Thanks for making the fix!

[jiewjwang]

@sunshineatnoon Hi,

I followed the tutorial you provided, but I noticed that the generated animation video has some jittering or shaking issues.
I'm not sure whether this problem comes from the data, the model, or something else.

Could you please confirm if you processed the data following the new tutorial as well?
Do you have any suggestions or ideas on what might cause the jittering?

I’m a bit lost at this point and would really appreciate any guidance.
Thanks in advance!

sample04_rep00.mp4

[sunshineatnoon]

@sunshineatnoon Hi,

I followed the tutorial you provided, but I noticed that the generated animation video has some jittering or shaking issues. I'm not sure whether this problem comes from the data, the model, or something else.

Could you please confirm if you processed the data following the new tutorial as well? Do you have any suggestions or ideas on what might cause the jittering?

I’m a bit lost at this point and would really appreciate any guidance. Thanks in advance!

sample04_rep00.mp4

Hi, I did follow the data pre-processing steps. Maybe you can compare the videos that I upload above and see if the results match. I see the authors think the results look reasonable.

[sunshineatnoon]

Hi @ymingxie Thanks for the quick response. I think I fixed everything but would like to double check. Could you please let me know if these results look reasonable to you?
https://github.com/user-attachments/assets/30613b30-4430-4c6f-ad5f-a7cee5efc4da https://github.com/user-attachments/assets/3963a1d7-2cdb-46c2-8954-e6e5123a1bf7 https://github.com/user-attachments/assets/6b7e8719-6d23-4307-a791-94b04295ab12 https://github.com/user-attachments/assets/faa46531-23a6-4169-a636-6ca17a2ff573 https://github.com/user-attachments/assets/cd705bf4-cd9b-4da8-bade-8d52e97a2674 https://github.com/user-attachments/assets/a2762ccd-af72-4578-8335-2329f85c8707 https://github.com/user-attachments/assets/cf9a49f7-97d5-4d39-9aba-ab69e1857a87 https://github.com/user-attachments/assets/e75d0f89-8d24-4ca1-942e-dde54d57c169 https://github.com/user-attachments/assets/5ee1826a-1be4-4fc5-84c7-4050850a7561 https://github.com/user-attachments/assets/dcf377bb-377f-4b1f-a211-9bea542084a5

Yes, these results look reasonable to me. Thanks for making the fix!

Thanks for the confirmation!

[jiewjwang]

@sunshineatnoon Hi,
I followed the tutorial you provided, but I noticed that the generated animation video has some jittering or shaking issues. I'm not sure whether this problem comes from the data, the model, or something else.
Could you please confirm if you processed the data following the new tutorial as well? Do you have any suggestions or ideas on what might cause the jittering?
I’m a bit lost at this point and would really appreciate any guidance. Thanks in advance!
sample04_rep00.mp4

Hi, I did follow the data pre-processing steps. Maybe you can compare the videos that I upload above and see if the results match. I see the authors think the results look reasonable.

@sunshineatnoon Thanks a lot for your help! I really appreciate your support. I may need to ask you for guidance again in the future if that’s okay. Looking forward to learning more from you.