about deformable conv visualize

[huxian0402]

Thanks for your amazing work. Have you ever tried to visualize the deformable convolution just like the results in the paper 'Deformable Convolutional Networks'? like this:

I made a try to visualize your deformable convolution code. But I get the following results:


Can you help me?

[Zardinality]

@huxianxian I am afraid I have no clue on this problem since I have already tested the forward and gradient pass comparing to the original implementation. But still, I am curious about two questions concerning this problem:

1. How did you visualize the offset? Using the script from original implementation? Or you write it yourself?
2. How did you get the offset? Using the code from the other repo of mine?

[huxian0402]

Thanks， Yes ，I visualize the offset with the script from original implementation ， and the offset is got from your other repo : https://github.com/Zardinality/TF_Deformable_Net . I have successfully run your code for TF_Deformable_Net on VOC dataset.I get the following results:
Mean AP = 0.6289

Results:
0.708
0.697
0.645
0.475
0.448
0.661
0.792
0.783
0.374
0.634
0.566
0.695
0.716
0.683
0.755
0.363
0.593
0.612
0.760
0.616
0.629

---

Then i try to make a visualize for deform conv with the trained model , I got the offsets with the following code:

cls_score, cls_prob, bbox_pred, rois , a_offset,b_offset,c_offset, = \
    sess.run([net.get_output('cls_score'), net.get_output('cls_prob'), net.get_output('bbox_pred'),net.get_output('rois'),
          net.get_output('res5a_branch2b_offset'),net.get_output('res5b_branch2b_offset'),net.get_output('res5c_branch2b_offset'),],feed_dict=feed_dict)

all_offsets=[a_offset,b_offset,c_offset]

The I visualize the offsets with the script from original implementation,

res5a_offset = all_offsets[0]
res5b_offset = all_offsets[1]
res5c_offset = all_offsets[2]

im = im[:, :, (2, 1, 0)]

show_dconv_offset(im , [res5c_offset, res5b_offset, res5a_offset])

the function show_dconv_offset(im , [res5c_offset, res5b_offset, res5a_offset]) is the script from original implementation. but I can't get the correct results.

[Zardinality]

show_dconv_offset by default uses a step of [2,2], is it relevant to your issue since in Resnet101_train.py we use stride=1?