why cats_macro_auc value is 0.0 and cats_auc_per_type item also is None

[wuye251]

results1 = Scorer.score_cats(example_with_predicted, attr="cats", labels=labels, multi_label=True, threshold=0.8)

spacy version with 3.6 python version 3.10.

output:

{
     "token_acc": 1,
     "token_p": 1,
     "token_r": 1,
     "token_f": 1,
     "cats_score": 0,
     "cats_score_desc": "macro AUC",
     "cats_micro_p": 1,
     "cats_micro_r": 0.966804979253112,
     "cats_micro_f": 0.9831223628691983,
     "cats_macro_p": 0.75,
     "cats_macro_r": 0.6312645687645687,
     "cats_macro_f": 0.6733189484966435,
     "cats_macro_auc": 0,
     "cats_f_per_type": {
          "抱怨": {
               "p": 1,
               "r": 0.5384615384615384,
               "f": 0.7000000000000001
          },
          "其他": {
               "p": 1,
               "r": 0.9935897435897436,
               "f": 0.9967845659163987
          },
          "表扬": {
               "p": 1,
               "r": 0.993006993006993,
               "f": 0.9964912280701755
          },
          "投诉": {
               "p": 0,
               "r": 0,
               "f": 0
          }
     },
     "cats_auc_per_type": {
          "抱怨": null,
          "其他": null,
          "表扬": null,
          "投诉": null
     }
}

[wuye251]

How can i get praf with highest scoring as "positive" in multi-label. See from scorer.score_cats threshold params desc: is 0.0 get highest scoring in multi-label? But I'm get 1.0 between kinds of value.

Scorer.score_cats(example_with_predicted, attr="cats", labels=labels, multi_label=True, threshold=0.0)

{
     "cats_score": 0,
     "cats_score_desc": "macro AUC",
     "cats_micro_p": 1,
     "cats_micro_r": 1,
     "cats_micro_f": 1,
     "cats_macro_p": 0.75,
     "cats_macro_r": 0.75,
     "cats_macro_f": 0.75,
     "cats_macro_auc": 0,
     "cats_f_per_type": {
          "表扬": {
               "p": 1,
               "r": 1,
               "f": 1
          },
          "其他": {
               "p": 1,
               "r": 1,
               "f": 1
          },
          "抱怨": {
               "p": 1,
               "r": 1,
               "f": 1
          },
          "投诉": {
               "p": 0,
               "r": 0,
               "f": 0
          }
     },
     "cats_auc_per_type": {
          "表扬": null,
          "其他": null,
          "抱怨": null,
          "投诉": null
     }
}