R/panel_diversity.R
compute_diversity.RdCurated pathway-diversity scores (e.g. counting MSigDB Hallmark gene sets a
panel touches) collapse to NA whenever the feature identifiers are not gene
symbols – a routine situation for proteomics panels (SomaScan/aptamer target
names, UniProt accessions). compute_diversity() is an annotation-free
replacement: it clusters the training feature pool into co-expression modules
and scores each panel by how many distinct modules it spans.
Score Pareto solutions by co-expression diversity
A data frame from summarize_solutions() (must contain a
solution_id column). As a convenience, an OptimizationResult may be
passed directly, in which case its summary is computed internally.
The OptimizationResult the solutions summary
came from. Supplies both the training feature matrix (@aggregated_x, whose
columns are the base-feature pool) and the per-solution base features.
Optional when solutions is itself an OptimizationResult.
Number of co-expression modules to cut the pool into
(default 10). Capped at the number of usable pool features (a warning is
issued if reduced). Note the score can only reach 1.0 when
n_modules >= panel size, and degenerates to all-1.0 once
n_modules >= pool size.
Clustering method. Currently only "hierarchical" is
supported. (See the TODO in Details – as an open-source project we should
add alternative methods such as k-means.)
Agglomeration method passed to stats::hclust()
(default "ward.D2").
Correlation type for the feature distance, one of
"pearson" (default) or "spearman".
Name of the appended diversity column
(default "coexpr_diversity").
The solutions data frame with one additional numeric column
(named by column) giving each solution's co-expression diversity in
(0, 1]. Row order and all existing columns are preserved. Solutions whose
features cannot be mapped to the pool receive NA.
Appends a co-expression diversity score to a summarize_solutions() data
frame. The feature pool stored in the OptimizationResult is clustered once
into n_modules co-expression modules (correlation distance 1 - |r|,
hierarchical clustering), and each solution is scored as the number of
distinct modules its base features span divided by the panel size. The result
lies in (0, 1]: values near 1 indicate the panel draws from many
independent co-expression programs (spread / non-redundant), while low values
indicate the features co-cluster (co-regulated / redundant).
Because every pool feature belongs to exactly one module (there is no
"unmapped" bucket), the score is never NA for panels whose features are in
the pool – unlike annotation-based pathway diversity on non-symbol features.
Multi-cohort handling. When the optimization spans multiple cohorts,
@aggregated_x stacks all cohorts (and, for the transferable pipeline, the
train + validation partitions). Correlating that pooled matrix directly would
conflate genuine within-cohort co-expression with between-cohort batch
mean-shifts (the effect metric_cohort_leakage is designed to detect),
biasing features to look more redundant. To avoid this, each feature is
centered within its cohort before the correlation matrix is computed, so
modules reflect co-expression rather than batch structure. For a single
cohort this is a no-op (correlation is invariant to centering).
TODO (open-source extension point). Only hierarchical clustering is
implemented. Other clustering strategies – k-means, DBSCAN, graph/community
detection on the correlation network – would be natural additions, ideally
exposed via a small clustering-method registry mirroring the feature-transform
registry in R/feature_transforms.R (new.env() + register_*() + getter,
with built-ins registered inline). Until then method accepts a single value.
if (FALSE) { # \dontrun{
opt <- optimize_panel(
x = list(Cohort1 = x1, Cohort2 = x2),
y = list(Cohort1 = y1, Cohort2 = y2),
objectives = define_objectives(metrics = c("auc", "cohort_auc_gap")),
feature_pool = ruv_features, max_features = 5,
feature_transform = "pairwise_ratios", algorithm = "NSGA-III",
fitness_mode = "within_cohort_val", train_ratio = 0.7, val_ratio = 0.2
)
solutions <- summarize_solutions(opt)
solutions <- compute_diversity(solutions, opt)
solutions[order(-solutions$coexpr_diversity), ]
} # }