Optical Flow Results Flickering — Is It Normal?

[xdobetter]

Hi, thank you for this awesome repo! 🙌
I'm new to the field of optical flow estimation, and I've been experimenting with your model on my own video data.

I used the demo.py script to run inference on my own video. I tried all four provided models: KITTI, Sintel, Spring, and FlyingThings. However, as shown in the attached video, the results don't look very good — the flow appears inconsistent or unstable across frames.

I'd like to ask:

• Am I using the models correctly?
• Are these results expected for this type of data?
• Or should I expect better performance based on what was shown in the paper?

Since I'm new to optical flow estimation, I'd really appreciate any clarification!

IcYcwbypv_s_12.mp4

flow_kitti_result.mp4

flow_sintel_result.mp4

flow_spring_result.mp4

flow_things_result.mp4

[littlespray]

Hi,

Thank you for your question. Such bad cases are possible. Currently, optical flow models commonly suffer from insufficient generalization in weak-texture regions like skies and walls. You can directly compare the results by using the integrated RAFT model in torchvision. If similar jittering occurs, it might be an issue with the current model. Based on experience, checkpoints from Spring or Things often exhibit better generalization performance, so you may want to try these two checkpoints first. Additionally, you can try adjusting the iters in StreamFlow to a value between 15 and 30.