Reference

Reference#

Testing#

spicyr.spicy(cells, condition=None, subject=None, covariates=None, image_id='imageID', cell_type='cellType', spatial_coords=('x', 'y'), r=None, from_=None, to=None, method='cell', effect='allocation', k=None, combine='maxT', adjust_abundance=False, variance='cr2', ref=None, cores=1, survival=None, table_key='table', image_info=None, **kwargs)#

Test whether the co-localisation of cell types differs between conditions, or is associated with survival.

Every ordered pair of cell types from_ -> to is tested: are to cells placed near from cells more than other cells are?

In each image, the share of to cells with at least one from cell within r is compared with its exact expectation q if the to cells were a random choice among the cells that are not from cells (random labelling of the observed cells). The effect (effect="allocation", the default) is the fraction of to cells placed next to (or kept away from) from cells. For a pair that attracts (more to cells next to from cells than q over all images together) it is (observed share - q) / (1 - q): if a fraction f of the to cells were moved next to from cells, the effect is f. For a pair that avoids it is (observed share - q) / q: if a fraction f of the to cells that would have a from cell nearby were moved away, the effect is -f. Either way it does not depend on how many from cells there are or how densely they are packed. The side is chosen once per pair from all images, without the conditions, and is reported in the side column. effect="count" gives the number of extra from cells within r of each to cell instead; it also reflects how many from cells surround a to cell (depth of infiltration), but it grows with how densely the from cells are packed. Images are combined within patients (subject) and patients within conditions by a frailty GEE with a second random effect for the images of a patient (each image is weighted by 1 / (its sampling variance + the between-image variance), so one large image cannot outweigh the patient’s other images), and the difference is tested with a CR2 variance on Satterthwaite degrees of freedom, with patients as the units. The difference is adjusted for any covariates and, with adjust_abundance=True, for the log share of the from type in each image; the unadjusted test is then reported alongside (unadjusted_* columns). When nearly every cell has a from cell within r (q close to 1), there is little room for attraction and an attracting pair’s images carry little information; a smaller r is more informative.

from is spelled from_ because from is a Python keyword (**{"from": ...} also works).

Parameters:

  • cells – A pandas DataFrame (one row per cell), an AnnData object (cells as observations; coordinates in obsm["spatial"] or obs columns), a SpatialData object (its annotation table table_key), or the path of a SpatialData store (.zarr) or of an AnnData zarr store. A store is read lazily: the image and cell-type columns are loaded once as integer codes, the coordinates (obsm["spatial"], or the obs columns spatial_coords) a batch of whole images at a time, and the condition, subject and covariates once per image, so the data never need to fit in memory. Reading is fastest when the rows are grouped by image (as spatialdata writes a table sorted by region); categorical image and cell-type columns keep the codes small.

  • condition – Column of the image-level condition (two or more groups).

  • subject – Column of the patient of each image. Images of one patient are combined; by default each image is a patient.

  • covariates – Image- or patient-level columns to adjust for. The effect of each is reported as <column>_effect and <column>_p_value (one column per level after the first for a categorical covariate).

  • r – Radius, or several radii combined by combine (“maxT”, up to 6 radii; “davies”, Davies’ bound for the same maximum, for many radii; or “cauchy”). Default 50. The max-T p-value is the multivariate normal probability of the largest absolute z, computed by the quasi-Monte Carlo method of Genz and Bretz with an absolute error of about 1e-4 (by quadrature for two radii); the same inputs always give the same value.

  • from_ – Cell types to test (all ordered pairs by default).

  • to – Cell types to test (all ordered pairs by default).

  • method – “cell” (spicyR Cell). The image-level method of spicyR is not yet available in Python.

  • effect – “allocation” (default): the fraction of to cells moved next to (or away from) from cells within r; “count”: the number of extra from cells within r of each to cell. With k, “within r” means among the cell’s k nearest neighbours.

  • k – Use the k nearest neighbours instead of a radius.

  • adjust_abundance – Adjust the test for the log share of the from type in each image (default False). Its effect is reported as abundance_effect. It does not separate more from cells from more densely packed ones, and it removes real effects when the share tracks the condition.

  • variance – “cr2” (default: CR2 on Satterthwaite df) or “hartung_knapp” (the model-based variance, scaled by the Pearson statistic when that exceeds its df and never below CR2, on m - 2 df). The variance used is in .variance of the result.

  • survival – A survival outcome: (time_column, event_column), one value per patient (the R package’s Surv(time, event)). Leave condition empty. (Older form: condition=time_column, survival=event_column.)

  • image_id – Columns of the image, the cell type and the coordinates. For SpatialData, images are the table’s regions (its region_key column) unless image_id names another column.

  • cell_type – Columns of the image, the cell type and the coordinates. For SpatialData, images are the table’s regions (its region_key column) unless image_id names another column.

  • spatial_coords – Columns of the image, the cell type and the coordinates. For SpatialData, images are the table’s regions (its region_key column) unless image_id names another column.

  • cores – Threads. Batches of whole images are read and summarised in parallel; results do not depend on it.

  • table_key – The annotation table of a SpatialData object or store.

  • image_info – A table with one row per image, for condition, subject, covariates or the survival columns that are not columns of the cell table: a DataFrame, an AnnData object (its obs), or the key of a table of the SpatialData object or store. Its image_id column, or else its index, gives the image IDs. Every image of the cells needs a row, and every row an image.

Return type:

SpicyResults

Writing a store#

spicyr.write_cell_store(cells, path, image_id='imageID', spatial_coords=('x', 'y'), table_key='table', image_info=None, info_key='images', chunk_rows=1048576)#

Write cells as the cell table of a SpatialData Zarr store.

The store holds the cells as an AnnData table under tables/ (Zarr v3), which spicy() reads a block of rows at a time and spatialdata.read_zarr() reads as a SpatialData object. The coordinates become the obsm array "spatial" and the other columns the table’s obs (categorical columns stay categorical). The image column is the table’s region key: a categorical column whose categories are the images in order of first appearance, unless it is categorical already. A column "instance_id" numbers the cells from 0 (unsigned integers of 16 bits at least). A table with one row per image can be added as a second table. Every array of the length of the table is written in chunks of chunk_rows rows compressed with zstd, one chunk at a time, so writing needs little memory beyond that of cells, however many cells it has.

Parameters:

  • cells – A pandas DataFrame with one row per cell, or an AnnData object (cells as observations; its obs is written, with the coordinates from the obs columns spatial_coords or else from obsm["spatial"]). Columns are categorical, strings, integers, floats or booleans (pandas’ nullable types included).

  • path – The directory of the store (replaced if it exists).

  • image_id – The column of the image.

  • spatial_coords – The two columns of the coordinates. They are written as float64 (float32 coordinates stay float32).

  • table_key – The name of the cell table.

  • image_info – Optional: a DataFrame (or an AnnData object, its obs) with one row per image, written as the table info_key. Its image_id column, or else its index, gives the image IDs.

  • info_key – The name of the per-image table.

  • chunk_rows – Rows per chunk of every array of the length of a table.

Return type:

The path of the store.

Examples

>>> import numpy as np, pandas as pd
>>> rng = np.random.default_rng(1)
>>> cells = pd.DataFrame({"imageID": pd.Categorical(np.repeat(["a", "b"], 50)),
...                       "cellType": pd.Categorical(np.tile(["T", "B"], 50)),
...                       "x": rng.uniform(0, 100, 100), "y": rng.uniform(0, 100, 100)})
>>> path = write_cell_store(cells, "cells.zarr")

Results#

class spicyr.SpicyResults(cell_results, levels, survival, radius_results=None, image_ids=<factory>, condition=None, subject=None, pairwise_assoc=<factory>, image_weights=<factory>, r=None, k=None, method='cell', effect='allocation', variance=None)#

Results of spicyr.spicy().

cell_results is the full table, one row per pair (and per level when there are more than two conditions): for the allocation effect the pair’s side (“attract” or “avoid”, which sets its scale), the effect (effect: “allocation”, the fraction of to cells moved next to or away from from cells within r, or “count”, the extra from cells per to cell) in the reference condition (excess_ref) and the comparison condition, the difference, its standard error, Satterthwaite df, p-value and BH-adjusted p-value, the frailty variance tau2, what the test was adjusted for (adjusted_for), the effect and p-value of each adjustment (abundance_effect, <covariate>_effect, …) and the unadjusted test (unadjusted_*). image_weights gives, for every pair, each image’s share of its condition’s information. For survival: the score test (score_coefficient, p_value) and the hazard ratio per SD of the shrunken excess.

bind(pair=None)#

Per-image excess of every pair (or one "from__to" pair) with the image’s condition and subject.

Return type:

DataFrame

box_plot(from_, to, ax=None, interactive=False)#

Per-image excess of one pair by condition, with a point per image behind each box.

The Python version of spicyR’s spicyBoxPlot().

Each point is sized by how much the image contributes to the test (its weight in the frailty model, relative to the average image of its condition). With interactive=True the result is a plotly figure: hover over a point to see its image, patient, excess and weight, which helps to find images worth looking at with spicyr.plot_image(). Images in which the pair was not tested are not shown.

signif_plot(fdr=False, breaks=None, comparison_group=None, colours=('#4575B4', 'white', '#D73027'), marks_to_plot=None, cutoff=0.05, ax=None)#

Bubble plot of every pair, as spicyR’s signifPlot().

Each pair is a disc at (to, from). Its left half is coloured by the excess in the reference condition and its right half by the excess in the comparison condition; the radius grows with -log10 p (the BH-adjusted p with fdr=True), and a black ring marks p (or, with fdr=True, the BH-adjusted p) below cutoff. breaks is (low, high, step) for the colour scale. For survival results each disc is one colour: the log hazard ratio per SD.

top_pairs(n=10, coef=None, adj='fdr', cutoff=None, figures=None)#

The most significant pairs, as spicyR’s topPairs().

Return type:

DataFrame

The attributes of a results object:

Attribute

Contents

cell_results

one row per pair (and per group with more than two groups): excess_ref, excess_comp, excess_difference, se, df, p_value, p_adj, tau2; with covariates or adjust_abundance=True, what the test was adjusted for (adjusted_for), the effect and p-value of each adjustment (<covariate>_effect, <covariate>_p_value, abundance_effect) and the unadjusted test (unadjusted_difference, unadjusted_se, unadjusted_df, unadjusted_p_value, unadjusted_p_adj). For survival: score_coefficient, p_value, p_adj, hazard_ratio_sd, hr_p_value and, with adjust_abundance=True, the p-value and hazard ratio without it. With several radii: r and p_value_best_radius

radius_results

with several radii, one row per pair and radius

pairwise_assoc

the effect of every pair in every image (what bind() returns)

image_weights

the weight of every image in the test of every pair: its share of its group’s information (the point sizes of box_plot())

levels, image_ids, condition, subject, r, k, effect

the groups, images and settings of the analysis (effect: “allocation” or “count”)

Proportions#

spicyr.get_prop(cells, feature='cellType', image_id='imageID')#

The proportion of each level of feature in each image, as spicyR’s getProp().

Parameters:

  • cells – A pandas DataFrame or an AnnData object with one row per cell.

  • feature – The column whose proportions are computed, e.g. cell types or regions.

  • image_id – The column of the units: images, or patients to pool the images of each patient.

Return type:

DataFrame

spicyr.col_test(df, condition, type=None, feature=None, image_id='imageID')#

Test each column of a table, such as proportions per image, as spicyR’s colTest().

Each column is tested for a difference between two groups (Wilcoxon or Welch t-test) or for an association with survival (Cox model).

Parameters:

  • df – A DataFrame with one row per image (or patient) and one column per feature, such as the output of get_prop(). With feature given, df is instead a table of cells (a DataFrame or an AnnData object), and the proportions of feature in each image_id are computed first.

  • condition – The group of each row of df (a sequence or Series in its row order), or two columns (time, event) for survival. With feature, the name of the cells’ group column.

  • type – "wilcox", "ttest" or "survival"; by default "ttest" for one condition and "survival" for two columns.

  • feature – The column of the cells whose proportions are tested, e.g. cell types or regions.

  • image_id – With feature: the units, images or patients (to pool the images of each patient).

Return type:

DataFrame

Returns:

  • A DataFrame with one row per column of ``df``, ordered by p-value (the group means (t-test), the test)

  • statistic (tval.W, tval.t) or the Cox coefficient and its standard error (survival), the p-value

  • (pval), the Benjamini-Hochberg adjusted p-value (adjPval) and the column name (cluster), all

  • rounded to two significant digits as in R.

Plotting an image#

spicyr.plot_image(cells, image, from_, to, r=None, image_id='imageID', cell_type='cellType', spatial_coords=('x', 'y'), table_key='table', ax=None, **kwargs)#

Plot one image, showing the from_ and to cells of a pair.

The density of all cells is shown in blue, with the from_ cells (gold) and to cells (dark red) on top. With r, a circle of radius r is drawn around each to cell: the from_ cells inside the circles are those that spicyr.spicy() counts.

Parameters:

  • cells – A pandas DataFrame, AnnData or SpatialData object, as for spicyr.spicy().

  • image – The image to plot (a value of the image_id column).

  • from_ – The two cell types (from also works as a keyword).

  • to – The two cell types (from also works as a keyword).

  • r – Optional radius of the circles around the to cells, in the units of the coordinates.

Return type:

The matplotlib axes.

Data#

spicyr.datasets.metabric_ali2020(path=None, as_anndata=False)#

Imaging mass cytometry of 483 breast tumours from the METABRIC cohort (Ali et al., Nature Cancer 2020).

One row per cell (433,001 cells): the core (file_id), the patient (metabricId), the cell type assigned by the authors (description), the coordinates in micrometres (x, y) and the patient’s clinical data, including ER status (ER.Status) and relapse-free survival (timeRFS, eventRFS).

The file (130 MB) is downloaded once from Bioconductor’s ExperimentHub, where the R package SpatialDatasets gets it (spe_Ali_2020), and cached in ~/.cache/spicyr (or $SPICYR_CACHE). Reading it needs the rds2py package (pip install "spicyr[data]").

With as_anndata=True the cells are returned as an AnnData object: the table as obs and the coordinates in obsm["spatial"] (needs anndata).

spicyr.datasets.schurch2020(path=None, as_anndata=False)#

CODEX imaging of colorectal cancer (Schürch et al., Cell 2020): 35 patients, four tissue cores each.

One row per cell (258,385 cells) with the core (imageID), the patient (patient), the cell type (cellType), the coordinates (x, y) and the patient group (group: 1 = Crohn’s-like reaction, 2 = diffuse inflammatory infiltration). Downloaded from ExperimentHub as spe_Schurch_2020 of the R package SpatialDatasets (82 MB), and cached as for metabric_ali2020(). as_anndata as for metabric_ali2020().

spicyr.datasets.example_cells()#

A small simulated data set: 14 patients (7 per condition, A and B), two images each, six cell types.

In condition B, about a third of the T cells were moved next to tumour cells. Both directions change; tumour -> T (are T cells placed near tumour cells? tumour cells counted around each T cell, the recruited type at the centre) is the direction whose excess does not depend on how common tumour cells are.

Return type:

DataFrame

spicyr.format_data(cells, image_id='imageID', cell_type='cellType', spatial_coords=('x', 'y'), table_key='table')#

One row per cell with columns imageID, cellType, x, y, and every other column kept.

Rows are ordered by image in order of first appearance, as spicyR’s .format_data().