Where do the Sync-C and Sync-D benchmarks come from?

[mulanlew]

Hi I would like to know where you got the Sync-C and Sync-D benchmarks from, thanks!

[xuyangcao]

Hi I would like to know where you got the Sync-C and Sync-D benchmarks from, thanks!

We follow pipelines in the Wav2lip: https://github.com/Rudrabha/Wav2Lip

[mulanlew]

Thanks!

[sampagon]

We follow pipelines in the Wav2lip: https://github.com/Rudrabha/Wav2Lip

I went ahead and setup the test bench from Wav2Lip but started with cherry-picking videos just to see what the sync-net scores the confidence and distance. It seems to be almost arbitrary... Using videos from your project page, Echomimics, HDTF, etc... all yield results with little to no pattern. For example, I tried 4-5 ground truth clips from HDTF that all yield a 0.x confidence scores which doesn't make sense considering they should have higher confidence than anything that is generated by an audio driven solution. What was your experience with this benchmark? @xuyangcao