Skip to content

Commit 0046121

Browse files
committed
correct accuracy calculation when all samples are incorrect
1 parent c67df19 commit 0046121

File tree

2 files changed

+7
-1
lines changed

2 files changed

+7
-1
lines changed

NEWS.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# vitals (development version)
22

3+
* The default `accuracy()` metric will now report a score of 0 rather than
4+
`NaN` when all scores are 0.
5+
36
* Fixed bug where non-default grading systems in model-graded evals would
47
result in scores being wiped during logging (#139).
58

R/task.R

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -700,7 +700,10 @@ Task <- R6::R6Class(
700700
if (any(is.na(numeric_scores))) {
701701
return()
702702
}
703-
numeric_scores <- numeric_scores / max(numeric_scores, na.rm = TRUE)
703+
max_score <- max(numeric_scores, na.rm = TRUE)
704+
if (is.finite(max_score) && max_score > 0) {
705+
numeric_scores <- numeric_scores / max_score
706+
}
704707
private$metric_results <-
705708
list2(
706709
accuracy = logged(accuracy)(numeric_scores)

0 commit comments

Comments
 (0)