[Bug]: `tm_outliers()` (picks) errors on a child dataset with a compound primary key: "Join columns in `x` must be present in the data"

[wangben718]

What happened?

Summary

When tm_outliers() is used through the new picks() API on a dataset that is a child in the join graph and whose primary key has more columns than the foreign key it shares with its parent (e.g. ADLB with key STUDYID/USUBJID/PARAMCD/AVISIT, FK STUDYID/USUBJID), the module errors as soon as the tab renders:

Error in : Join columns in `x` must be present in the data.
✖ Problem with `PARAMCD` and `AVISIT`.
 when evaluating qenv code:
ANL_OUTLIER_EXTENDED <- dplyr::left_join(
  ANL_OUTLIER,
  dplyr::select(ADLB, ...),
  by = c("STUDYID", "USUBJID", "PARAMCD", "AVISIT")
)

The old data_extract_spec path (tm_outliers.default / srv_outliers) is not affected — this is a regression introduced by the picks migration.

Reproducible example

Uses only the example CDISC data shipped with {teal.data} (rADSL, rADLB, default_cdisc_join_keys):

library(teal.modules.general)

data <- teal_data()
data <- within(data, {
  ADSL <- teal.data::rADSL
  ADLB <- teal.data::rADLB
})

# ADSL: parent (key STUDYID/USUBJID).
# ADLB: child of ADSL, compound primary key STUDYID/USUBJID/PARAMCD/AVISIT.
join_keys(data) <- teal.data::default_cdisc_join_keys[c("ADSL", "ADLB")]

app <- init(
  data = data,
  modules = modules(
    tm_outliers(
      "Outliers",
      outlier_var = picks(
        datasets("ADLB", "ADLB"),
        variables(choices = "AVAL", selected = "AVAL", multiple = FALSE)
      ),
      # A grouping variable is offered but NONE is selected by default:
      categorical_var = picks(
        datasets("ADLB", "ADLB"),
        variables(choices = c("PARAMCD", "AVISIT"), selected = NULL, multiple = FALSE)
      )
    )
  )
)

shinyApp(app$ui, app$server)

Open the app → the Outliers tab is blank and the console shows the error above.

Root cause

srv_outliers.picks() rebuilds ANL_OUTLIER_EXTENDED by left-joining back onto the dataset's full primary key:

join_keys <- as.character(teal.data::join_keys(data_obj())[dataname_first, dataname_first])
# ADLB -> c("STUDYID", "USUBJID", "PARAMCD", "AVISIT")
ANL_OUTLIER_EXTENDED <- dplyr::left_join(ANL_OUTLIER, dplyr::select(dataname, ...), by = join_keys)

But the teal.picks merge (teal.picks:::.merge_expr) only retains a child dataset's foreign key (plus the selected variables), dropping the rest of its primary key:

this_foreign_keys <- .fk(join_keys, dataname)     # ADLB -> STUDYID, USUBJID
this_primary_keys <- join_keys[dataname, dataname] # ADLB -> STUDYID, USUBJID, PARAMCD, AVISIT
this_variables <- if (length(this_foreign_keys) == 0L) {
  union(this_primary_keys, this_mapping$variables)  # PARENT -> full primary key
} else {
  union(this_foreign_keys, this_mapping$variables)  # CHILD  -> FK only (drops PARAMCD/AVISIT)
}

So ANL_OUTLIER never contains PARAMCD/AVISIT, yet the join still requires them → error.

When it happens / when it doesn't

Outlier dataset	Result
Parent / standalone (no foreign key)	works — full primary key retained
Child, and every extra key column is selected	works — selecting carries them into the data
Child, with an unselected primary-key column beyond the FK	fails

Notes:

• categorical_var accepts only one variable — srv_outliers.picks uses ANL[[categorical_var]], as.name(categorical_var), group_by(...), get(categorical_var), all single-column. So a dataset with two extra key columns (e.g. ADLB: PARAMCD and AVISIT) cannot be made to work from the app side — there is only one categorical slot.
• Passing categorical_var = c("PARAMCD", "AVISIT") (multiple = TRUE) does not help; it fails earlier with Error in [[: Can't extract column with 'categorical_var'.

Why existing tests don't catch it

Both tests/testthat/test-tm_outliers.R and tests/testthat/test-shinytest2-tm_outliers.R:

1. Build outlier_var with teal.transform::data_extract_spec(...) → they exercise the old tm_outliers.default path, not the picks path.
2. Use CO2 with a single synthetic key (CO2$primary_key <- seq_len(nrow(CO2)), join_key("CO2","CO2","primary_key")) → a standalone, single-key dataset, which cannot reach the failing branch.

So the combination "new picks API × child dataset with a compound key" is uncovered.

Suggested fix

Intersect the join key with the columns actually present in the analysis data, e.g.:

join_keys <- intersect(join_keys, names(ANL_OUTLIER))

(or have the merge retain a child dataset's full primary key). A regression test should cover the picks path with a parent/child pair where the child has a compound primary key (e.g. rADSL + rADLB).

Session info

• teal.modules.general 0.8.0
• teal.picks 0.3.0
• teal 1.2.1
• teal.data 0.8.1
• teal.transform (installed with the above)
• R 4.6.1

sessionInfo()

Relevant log output

Code of Conduct

• I agree to follow this project's Code of Conduct.

Contribution Guidelines

• I agree to follow this project's Contribution Guidelines.

Security Policy

• I agree to follow this project's Security Policy.

[averissimo]

I think the problem actually is from teal.picks as the join_keys() of the merged object should keep existing primary keys.

@LeonidasZhak , thanks for the PR, but I think the fix should land on teal.picks instead, as the module should be fine as is.

What do you think?