How to solve the problem of null appearing in the evaluation results? Thank you very much

[AGTSAAA]

Hi, I noticed the there are many preference null in annotations.json. Is this norm?

[EtashGuha]

I also see the have the same error :(. I don't think this is normal but I do not know what would have caused this.

[yujiaw98]

I'm facing the same error now, this may lead to the "n_total" being less than 805 in the final output but I don't know how to fix this.

[YannDubs]

null means that there was an error while annotating. By default it will try 5 times and otherwise it will just give up and save the null to say that it wasn't able to annotate.

One thing to keep in mind is that those null will also be cached by default meaning that if you try again annotating again it won't actually retry. You can avoid that by using --is_store_missing_annotations=False