Difference between "i2v_subject" and "subject_consistency" dimensions

[AshkanTaghipour]

No description provided.

[ziqihuangg]

Hi, "i2v_subject" under "I2V" refers to subject consistency between the input image and the generated videos, whereas "subject_consistency" under "Video Quality" refers to subject consistency among the generated video frames.

[AshkanTaghipour]

thank you very much,
the "i2v_subject" has not reported for the base I2V models in leaderboard correct?

[ziqihuangg]

The preliminary results have been posted on our leaderboard under "VBench-I2V", under the "Video-Image Subject Consistency" dimension. We want to clarify that these results can be replicated using our current open-source VBench-I2V code. However, we are considering optimizations to the evaluation pipeline to align it even more closely with human perception. Therefore, there may be a slight update to the leaderboard results later this month.

[ziqihuangg]

[AshkanTaghipour]

Thank you very much for your response.

[AshkanTaghipour]

Could you please clarify the temporal flickering dimension for the I2V suite? I was unable to reproduce the leaderboard results for SVD; I obtained a number of 0.9476 for SVD-XT. Additionally, I could not find any usage of 'temporal_flickering' in the VBench repository at I2V.
is that as effective evaluation dimension as for T2V or other dimensions mentioned in the I2V would be enough for I2V evaluation?