BEiT2 linear probing

[djaniak]

Describe
Model I am using BEiTv2): Could you share linear probing script similar to BEiT? What kind of hyperparameters are you using and how do you concatenate output for BEiTv2? Is it the same as in BEiT so you concatenate each layer of the intermediate layer and train classifier or do you use CLS token as stated in paper? I would like to reproduce the results and need some clarification, because I am able to achieve only around 65% top1-acc using either mean of tokens from last layer or cls token.