For fine-tuning ScandiBERT for multi-label language ID text classification (labels=4, Norwegian, Swedish, Danish, and "Non-Scandinavian"), is a final training loss of 0.279952, validation loss of 0.316215, and macro-F1 score of 0.922874 good?

GPT-5.5

Vote: Yes

A macro-F1 score of 0.922874 is strong for a 4-label language identification task, especially with closely related languages like Norwegian, Swedish, and Danish where confusion is expected. The validation loss of 0.316215 is also reasonably close to the training loss of 0.279952, suggesting there is no severe overfitting. So the results look good overall. I would still recommend checking per-class F1, the confusion matrix, and performance on truly out-of-domain "Non-Scandinavian" examples, but based on the reported metrics alone, the answer is "Yes".

Claude Opus 4.7

Vote: Yes

A macro-F1 of 0.923 across four classes is strong performance, especially for the notoriously difficult task of distinguishing closely related Scandinavian languages (Norwegian, Swedish, and Danish share substantial lexical and orthographic overlap). The training loss (0.280) and validation loss (0.316) are close together, indicating the model generalizes well without significant overfitting. The small gap (~0.036) suggests healthy convergence. While there may be room for improvement (e.g., handling Bokmål vs. Nynorsk ambiguity or short texts), these metrics indicate a successful fine-tune that is production-viable.

Grok 4.20

Vote: Yes

A macro-F1 score of 0.922874 is strong for distinguishing closely related languages like Norwegian, Swedish, and Danish plus a Non-Scandinavian class. The training loss of 0.279952 and validation loss of 0.316215 are low and close together, showing effective learning with minimal overfitting. For this fine-tuning task on ScandiBERT, these metrics indicate solid real-world performance.

Gemini 3.1 Pro

Vote: Yes

A Macro-F1 score of ~0.923 is generally excellent for this specific task. Distinguishing between closely related Scandinavian languages (Danish, Norwegian, and Swedish) can be highly challenging due to their significant lexical and syntactic overlaps. Because Macro-F1 calculates the metric independently for each class and then takes the average, a score over 92% indicates that the model is performing consistently well across all four categories, rather than just dominating on a majority class. Furthermore, the loss metrics point to healthy training dynamics. The validation loss (0.316215) is very close to the training loss (0.279952). This small generalization gap strongly suggests that the model is not overfitting to the training set and will generalize well to unseen text.

Final Summary

The AI Roundtable reached a swift and unanimous consensus, with all four models agreeing that the reported metrics indicate a highly successful fine-tuning. GPT-5.5 and Claude Opus 4.7 both highlighted that achieving a 0.923 macro-F1 is particularly impressive given the extreme lexical overlap between Danish, Norwegian, and Swedish, while Gemini 3.1 Pro pointed to the narrow loss gap as evidence of a model ready for real-world generalization.

All 4 models agreed on "Yes" after discussion

Strongest Arguments

• Yes: A macro-F1 score exceeding 0.92 is excellent for distinguishing mutually intelligible languages, and the small 0.036 gap between training and validation loss confirms the model has achieved healthy convergence without overfitting.