Add comprehensive variable pre-filtering mechanism with categorical support to br_run and br_pipeline functions

[Copilot]

This PR implements a comprehensive pre-filtering mechanism for focal variables (x) in both br_run() and br_pipeline() functions to address issues with variables that lack analytical significance, including both numeric and categorical variables.

Problem

Focal variables in regression analysis may include terms with:

• Excessive NA values (reducing effective sample size)
• Near-constant or zero variance (causing numerical instability)
• Single-level categorical variables (providing no information for modeling)
• Insufficient variability for meaningful analysis

These problematic variables can cause modeling failures or produce unreliable results, requiring manual data cleaning before analysis.

Solution

Added optional pre-filtering capabilities with the following new parameters:

• filter_x: Enable/disable filtering (default: FALSE for backward compatibility)
• filter_na_prop: Maximum proportion of NA values allowed (default: 0.8)
• filter_sd_min: Minimum standard deviation required for numeric variables (default: 1e-6)
• filter_var_min: Minimum variance required for numeric variables (default: 1e-6)
• filter_min_levels: Minimum unique levels required for categorical variables (default: 2)

Key Features

Smart Filtering Logic:

• Numeric variables: Filters based on NA proportion, variance, and standard deviation
• Categorical variables: Filters based on NA proportion and minimum unique levels
• Complex terms: Handles appropriately (e.g., I(x^2), interactions)
• Variable types: Supports character, factor, logical, and other data types

Categorical Variable Filtering:

• Automatically removes single-level categorical variables (constant categories)
• Configurable minimum level requirements via filter_min_levels
• Applies to character, factor, and logical variable types

User Experience:

• Informative messages showing filtering results and proportions
• Lists filtered variable names (with truncation for large lists)
• Error handling when all variables are filtered out
• Fully backward compatible (disabled by default)

Usage Examples:

# Enable filtering with default thresholds
br_pipeline(data, y = "outcome", x = variables, method = "gaussian", filter_x = TRUE)

# Custom filtering criteria including categorical levels
br_run(obj, filter_x = TRUE, filter_na_prop = 0.5, filter_min_levels = 3)

# Pipeline-style usage
breg(data) |>
  br_set_y("response") |>
  br_set_x(focal_vars) |>
  br_set_model("logistic") |>
  br_run(filter_x = TRUE)

Example Output:

Pre-filtering removed 5 out of 9 focal variables (55.6%)
Filtered variables: "constant_numeric", "constant_character", "constant_factor", "constant_logical", and "high_na_var"

Implementation Details

• Enhanced filter_variables_x() helper function in 98-utils.R
• Updated both br_run() and br_pipeline() function signatures
• Integrated filtering logic before model template generation
• Comprehensive test suite covering numeric and categorical variable scenarios
• All existing tests pass without modification

Testing

The implementation includes extensive tests covering:

• Basic filtering functionality with different variable types (numeric, character, factor, logical)
• Integration with both br_run() and br_pipeline() workflows
• Custom parameter configurations (filter_min_levels)
• Edge cases (all variables filtered, no variables to filter)
• Backward compatibility (existing code unchanged when filter_x = FALSE)

Fixes #50.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[ShixiangWang]

@copilot

categorical variables 也支持过滤，比如如果一个x如果只有一种分类，这种列肯定要过滤掉对吧？

目前数据科学领域对一般分类变量过滤会采用什么策略呢？怎么实现呢？

请校对和测试所有的计算逻辑，并确保所有test通过

[Copilot]

@copilot

categorical variables 也支持过滤，比如如果一个x如果只有一种分类，这种列肯定要过滤掉对吧？
...

You're absolutely right! I've enhanced the filtering mechanism to support categorical variables. The implementation now:

Categorical Variable Filtering:

• Automatically filters out categorical variables with only one unique level (constant categories)
• Supports character, factor, and logical variable types
• Added filter_min_levels parameter (default: 2) to control minimum unique levels required

Common Data Science Strategies Implemented:

1. Single-level filtering: Removes categorical variables with only one unique value
2. NA proportion filtering: Applies to all variable types including categorical
3. Configurable thresholds: filter_min_levels parameter allows customization

Example Results:

# Before: categorical variables with single levels were kept
# After: they are properly filtered out
Pre-filtering removed 5 out of 9 focal variables (55.6%)
Filtered variables: "constant_numeric", "constant_character", "constant_factor", "constant_logical", and "high_na_var"

Usage:

# Default behavior (filters single-level categoricals)
br_pipeline(data, y = "outcome", x = variables, method = "gaussian", filter_x = TRUE)

# Custom threshold for categorical levels
br_run(obj, filter_x = TRUE, filter_min_levels = 3)  # Requires at least 3 levels

All computational logic has been verified with comprehensive tests covering numeric, character, factor, and logical variables. All existing tests continue to pass, ensuring backward compatibility.

Commit: 889f532