The attack success rate on LIBERO-Object is only about 30-40%.

[939386360]

May I ask whether your training setup is as follows? The patch I trained using this setup did not perform well in attacks on LIBERO-Object, so I’d like to ask about the actual configuration you used during training.

However, the attack success rate reaches 100% on the other three LIBERO suites.

CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nproc_per_node=4 --master_port=29501 VLAAttacker/UADA_wrapper3_ddp.py \
    --maskidx 0 \
    --lr 1e-3 \
    --iter 2000 \
    --MSE_weights 5 \
    --accumulate 1 \                  
    --bs 8 \                            
    --warmup 20 \                        
    --tags XXX \                         
    --geometry True \                    
    --patch_size 3,50,50 \               
    --wandb_project your_wandb_project \ 
    --wandb_entity your_wandb_entity \   
    --innerLoop 50 \                     
    --dataset bridge_orig \              
    --resize_patch False \               
    --reverse_direction True          

[William-wAng618]

You can first try the single-GPU attack script, i.e. scripts/run_UADA.sh.

Also, I suggest increasing the learning rate to 2e-3 and checking whether the performance on LIBERO-Object improves.

The multi-GPU setup you posted looks generally reasonable, but based on our experience, these two changes are worth trying first when the patch works well on the other three LIBERO suites but not on LIBERO-Object.

[939386360]

Thank you very much for your reply! In addition, after some exploration, I found that the issue may be due to the fact that my sampling strategy for the custom LIBERO-10 dataset is different from yours. I randomly sample 40 frames from each trajectory for training, and I wonder whether this differs significantly from your strategy.

[William-wAng618]

I think for libero-10, each trajectory is >>40 frames. Sampling will miss lots of key frames.