Spacy french NER transformer based model fr_dep_news_trf not working

[zmy1116]

Hello, we want to use spacy to do NER extraction for french texts. The transformer based model fr_dep_news_trf seems to be broken. The list of entities is always empty.

How to reproduce the behaviour

We create a minimum example to reproduce the issue with google colab
https://colab.research.google.com/drive/1mngC0EBDOP3SAngeTeNRdK2d3EP2Mc-v?authuser=0#scrollTo=eXeJRQvflErl

import spacy
doc = nlp("Bonjour, Emmanuel. Bonjour, monsieur. Donc voilà, je fais plein de choses. Biologie, c'est du pire veau, museau, lentilles, c'est voilà. Donc la pièce est bouchée au sep, c'est pareil. Je fais une sauce au sep avec la crème. Ah, ça doit être pas mal aussi. C'est pas mal aussi.  Alors on va prendre un petit pot de quoi ? On a le Beaujolais, on a le Saint-Joseph, le Trois-Hermitages. Ah non, je suis une fille du Beaujolais, moi. Merci. Alors attends, je pousse.")

for w in doc.ents:
  print(w.text,w.label_)

the model doesn't detect anything.

Your Environment

It's the default colab environment

Info about spaCy

• spaCy version: 3.6.1
• Platform: Linux-6.1.58+-x86_64-with-glibc2.35
• Python version: 3.10.12
• Pipelines: fr_dep_news_trf (3.6.1), fr_core_news_lg (3.6.0), en_core_web_sm (3.6.0)

[svlandeg]

Hi! You're using fr_dep_news_trf, which doesn't have an NER component. In the documentation you can find which components it does have:

Components: transformer, morphologizer, parser, attribute_ruler, lemmatizer.

So it's not broken - the NER feature simply isn't available in this model. You'll have to use fr_core_news_lg instead.

[ameni-ayedi]

Is there another way around using fr_dep_news_trf for NER like adding a NER component or does translating your own dataset to english and using en_core_web_trf is a suitable/only approach ?

[zmy1116]

Hello Sofie,

Thank you for the quick response! It makes sense now.

I have a separate related question.. (I can open a separate issue if you see fit).

We actually started with the fr_core_news_lg model, and we find many incorrect result (Not just incorrect, but also a bit weird).
For example:

nlp = spacy.load('fr_core_news_lg')
doc = nlp("Alors que quand c'est cuiné comme ça, c'est comme de la viande. Exactement, tout à fait.")
for w in doc.ents:
  print(w.text,w.label_)

The model outputs cuiné as a PER.

I don't know how the NER model is trained. But I find this error is really weird:

• there wouldn't be any training data with cuiné as a PER
• If we just consider the grammar syntax or just surrounding context, there would never exist a case where in c'est xxx comme ca xxx being a PER

I understand that it's normal that model make mistakes.. but I see multiple errors like this that I can't understand how it would happen (For an extreme case, with fr_core_news_sm, Merci is considered as an entity ). And for english and german models we don't see obvious errors like this.

Is french NER somehow more difficult? Should we perform some sort of pre-processing/post-processing to improve the accuracy?

Thank you

[svlandeg]

Hi! It's fine to keep in this thread, on the discussion forum we're less strict about one-topic-per-thread.

That does sound a bit odd, yea. There's no difference between the machinery used for training NER on the different languages - the only difference would be in the training data. WikiNER is used for both German and French, but I wouldn't rule out the fact that one language can be of different quality than another language within the same dataset (depending on annotators and/or dataset size). So to get to the bottom of why these weird cases are happening, it might be worth looking at the French WikiNER corpus... (sorry, not exactly a satisfying answer, I realise!)

[zmy1116]

Thanks for the clarification.

I guess for now I will try to do post processing and eliminate false positives based on grammar and syntax... time to review french .....

[svlandeg]

By the way - one more comment. The reason why the German and French trf models don't currently have an NER in their pipeline, is because they use different datasets for NER & syntax, and we didn't want to have two separate transformers in the pipeline...

This might be one reason why English performs better, because its transformer layer is being trained on both tasks in a sort of transfer learning setting.