Fit a Neyman-Pearson Panel from Optimization Results

Takes an OptimizationResult from optimize_panel() and fits a Neyman-Pearson classifier on a selected Pareto solution. Returns a TransferablePanelResult with the NP classifier and threshold.

fit_np_panel(
  optimization_result,
  solution_id = NULL,
  features = NULL,
  x = NULL,
  y = NULL,
  cohort = NULL,
  minimize_FPR = TRUE,
  alpha = 0.05,
  delta = 0.05,
  method = "penlog",
  split = 1L,
  split.ratio = 0.5,
  n.cores = 1L,
  ...
)

Arguments

optimization_result

An OptimizationResult from optimize_panel().

solution_id

Integer specifying which Pareto solution to use. If NULL (default), automatically selects the solution with the best value on the first objective.

features

Character vector of features to use. If provided, overrides the features from solution_id. Must be a subset of the feature pool.

x

Optional training feature matrix. If NULL, uses the stored aggregated training data from the optimization result.

y

Optional training response vector. If NULL, uses the stored response from the optimization result.

cohort

Optional cohort factor. If NULL, uses the stored cohort from the optimization result.

minimize_FPR

Logical; if TRUE (default), the NP classifier controls the false positive rate (Type I error). If FALSE, labels are flipped internally so the classifier controls the false negative rate (miss rate), which is appropriate for rule-out diagnostic tests requiring high sensitivity.

alpha

Target error rate to control (default 0.05). When minimize_FPR = TRUE, this is the maximum FPR. When minimize_FPR = FALSE, this is the maximum FNR (i.e., minimum sensitivity = 1 - alpha).

delta

Tolerance for violating the alpha constraint (default 0.05). The error rate is guaranteed to be at most alpha with probability 1-delta.

method

NP classification method passed to nproc::npc(). One of: "penlog" (penalized logistic, default), "logistic", "svm", "lda", "slda", "nb", "nnb", "ada", "tree", "randomforest".

split

Number of random train/test splits for threshold estimation (default 1).

split.ratio

Proportion of data used for training in each split (default 0.5).

n.cores

Number of cores for parallel processing (default 1).

...

Additional arguments passed to nproc::npc().

Value

A TransferablePanelResult with the fitted NP classifier.

Details

The NP paradigm controls one type of error (FPR or FNR) at level alpha with probability at least 1-delta, while minimizing the other error type.

See also

optimize_panel(), fit_panel(), evaluate_panel()

Examples

if (FALSE) { # \dontrun{
# Run optimization targeting the high-sensitivity region
opt <- optimize_panel(
  x, y,
  objectives = define_objectives(
    metrics = c("pauc", "num_features"),
    params = list(pauc = list(sens_floor = 0.90))
  ),
  constraints = list(min_metric_constraint("sensitivity", 0.90))
)

# Fit NP classifier for rule-out test (control FNR to achieve high sensitivity)
panel <- fit_np_panel(opt, minimize_FPR = FALSE, alpha = 0.05)

# This guarantees sensitivity >= 95% with high probability
per_cohort_metrics(panel)

# Evaluate on held-out data
evaluate_panel(panel, x_test, y_test)
} # }