ELIXIR Spatial course
Spatial & single-cell data integration
Day 2 - Practical 5
7th October 2026
Benedetta Manzato (LUMC) & Emma Vonk (LUMC)
- Dataset description
- Spatial imputation with SpaGE and Tangram
- Cell deconvolution with Cell2location
import os
import warnings
import anndata as ad
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import scanpy as sc
warnings.filterwarnings("ignore")
# Display the version of important packages for reproducibility
sc.logging.print_header()
# Set the default aesthetics for Scanpy plots
sc.set_figure_params(facecolor='white', figsize=(8, 8)) # White background, figure size 8x8
sc.settings.verbosity = 0 # Set verbosity to show only errors (0) # errors (0), warnings (1), info (2), hints (3)
We create an output folder in your local space to save outputs:
folder_path = './outputs/'
if not os.path.exists(folder_path):
os.makedirs(folder_path)
print(f"Folder created: {folder_path}")
else:
print(f"Folder already exists: {folder_path}")
Folder created: ./outputs/
Dataset: MERFISH Spatial Transcriptomics of Adult Mouse Brain (ZhuangāABCAā1)¶
We will use the Zhuang-ABCA-1 dataset: 4.2 million cell spatial transcriptomics dataset spanning 147 coronal sections with a 1122 gene panel. 2.8 million cells passed cell classification confidence score threshold and displayed in the ABC atlas.
We decided to work with one section only, the one that captures most cells (section 80).
Read data¶
We will read the AnnData object that has been filtered to have only section 80 and QC was performed from the authors of the study. The data can be found in the Zenodo directory provided.
adata_section = ad.read_h5ad("../data/Practical5/adata_section80.h5ad")
The dataset was obtain with a targeted / imaging-based technology and not the whole transcriptome is measured, just a limited gene panel.
- Question: how many genes are measured?
Reminder from Practical 0: Spatial coordinates, being metadata about cells/spots are stored either in .obs or .obsm Spot metadata also contains info about class, subclass, neurotransmitter, patient metadata and section.
adata_section.obs.head()
| brain_section_label | feature_matrix_label | donor_label | donor_genotype | donor_sex | cluster_alias | x | y | z | subclass_confidence_score | ... | neurotransmitter | class | subclass | supertype | cluster | neurotransmitter_color | class_color | subclass_color | supertype_color | cluster_color | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| cell_label | |||||||||||||||||||||
| 299794030110334465835331169565547130392 | Zhuang-ABCA-1.080 | Zhuang-ABCA-1 | Zhuang-ABCA-1 | wt/wt | F | 196 | 5.873824 | 1.246374 | 6.757411 | 0.951445 | ... | Glut | 01 IT-ET Glut | 006 L4/5 IT CTX Glut | 0027 L4/5 IT CTX Glut_5 | 0097 L4/5 IT CTX Glut_5 | #2B93DF | #FA0087 | #3DCCB7 | #639917 | #1C0099 |
| 66045375924099067525605874805924532486 | Zhuang-ABCA-1.080 | Zhuang-ABCA-1 | Zhuang-ABCA-1 | wt/wt | F | 196 | 5.066814 | 1.215444 | 6.757411 | 0.990889 | ... | Glut | 01 IT-ET Glut | 006 L4/5 IT CTX Glut | 0027 L4/5 IT CTX Glut_5 | 0097 L4/5 IT CTX Glut_5 | #2B93DF | #FA0087 | #3DCCB7 | #639917 | #1C0099 |
| 116791314340621211157298988846240231316 | Zhuang-ABCA-1.080 | Zhuang-ABCA-1 | Zhuang-ABCA-1 | wt/wt | F | 196 | 4.681061 | 1.287158 | 6.757411 | 0.972796 | ... | Glut | 01 IT-ET Glut | 006 L4/5 IT CTX Glut | 0027 L4/5 IT CTX Glut_5 | 0097 L4/5 IT CTX Glut_5 | #2B93DF | #FA0087 | #3DCCB7 | #639917 | #1C0099 |
| 13950435409550353848526753205083905255 | Zhuang-ABCA-1.080 | Zhuang-ABCA-1 | Zhuang-ABCA-1 | wt/wt | F | 174 | 4.708601 | 1.292049 | 6.757411 | 0.995666 | ... | Glut | 01 IT-ET Glut | 006 L4/5 IT CTX Glut | 0023 L4/5 IT CTX Glut_1 | 0069 L4/5 IT CTX Glut_1 | #2B93DF | #FA0087 | #3DCCB7 | #3B9900 | #997145 |
| 147436316185400258367147717470263928395 | Zhuang-ABCA-1.080 | Zhuang-ABCA-1 | Zhuang-ABCA-1 | wt/wt | F | 174 | 4.729919 | 1.285492 | 6.757411 | 0.999973 | ... | Glut | 01 IT-ET Glut | 006 L4/5 IT CTX Glut | 0023 L4/5 IT CTX Glut_1 | 0069 L4/5 IT CTX Glut_1 | #2B93DF | #FA0087 | #3DCCB7 | #3B9900 | #997145 |
5 rows Ć 22 columns
Some spatial tools require the spatial coordinates can be stored as spatial or X_spatial
# Add X_spatial to the right field
x_spatial = np.array(adata_section.obs[['x','y']])
adata_section.obsm['X_spatial'] = x_spatial
adata_section.obsm['X_spatial']
array([[5.87382373, 1.24637434],
[5.0668137 , 1.21544436],
[4.68106119, 1.28715849],
...,
[0.97304079, 4.80513099],
[0.86566165, 4.29416095],
[0.84111779, 3.8952338 ]], shape=(37068, 2))
import gc
# collect garbage
gc.collect()
35
Prediction of spatial patterns for unmeasured genes using scRNA-seq data with SpaGE¶
SpaGE: Spatial Gene Enhancement using scRNA-seq (Abdelaal et al. 2020)
SpaGE cannot be installed using pip or conda, so we have to clone the Git repository.
![ -d SpaGE ] || (wget -qO- https://github.com/tabdelaal/SpaGE/archive/refs/heads/master.tar.gz | tar xz && mv SpaGE-master SpaGE)
Now that the sourced vode of SpaGE is cloned to our folder, lets append the path.
import sys
sys.path.append("SpaGE")
from SpaGE.main import SpaGE
SpaGE is now all set up! Lets continue with the rest of the analysis.
import scipy.stats as st
SpaGE takes as input the (targeted) spatial dataset (query) and a single-cell RNA dataset used as reference. The two datasets should be generated under the same conditions (same species, tissue, region) as much as possible.
We will use Mouse whole-brain transcriptomic cell type atlas (Hongkui Zeng).This scRNAseq dataset profiled 7 million cells (approximately 4.0 million cells passing quality control). More in detail:
1.7 million single cell transcriptomes spanning the whole adult mouse brain using 10Xv2 chemistry (WMB-10Xv2)
2.3 million single cell transcriptomes spanning the whole adult mouse brain using 10Xv3 chemistry (WMB-10Xv3)
1687 single cell transcriptomes spanning the whole adult mouse brain using the 10X Multiome chemistry (WMB-10XMulti)
The size is of the combined single-cell object from AbcProjectCache is around 280GB. For this reason, the three objects have been previously downloaded, concatenated, preprocessed and subsetted to obtain 28981 cells. For your knowledge, you can find all the steps in obtain_scref.py.
Let's read the resulting single-cell AnnData object:
adata_sc = ad.read_h5ad("../data/Practical5/sc_adata_1percent.h5ad")
Inspect the object:
adata_sc
AnnData object with n_obs Ć n_vars = 28981 Ć 32285
obs: 'cell_barcode', 'barcoded_cell_sample_label', 'library_label', 'feature_matrix_label', 'entity', 'brain_section_label', 'library_method', 'region_of_interest_acronym', 'donor_label', 'donor_genotype', 'donor_sex', 'dataset_label', 'x', 'y', 'cluster_alias'
layers: None (.X)
adata_sc.obs.head()
| cell_barcode | barcoded_cell_sample_label | library_label | feature_matrix_label | entity | brain_section_label | library_method | region_of_interest_acronym | donor_label | donor_genotype | donor_sex | dataset_label | x | y | cluster_alias | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| cell_label | |||||||||||||||
| GTATTCTCAGCTATTG-045_C01 | GTATTCTCAGCTATTG | 045_C01 | L8TX_180829_01_C10 | WMB-10Xv2-TH | cell | NaN | 10Xv2 | TH | Snap25-IRES2-Cre;Ai14-407905 | Snap25-IRES2-Cre/wt;Ai14(RCL-tdT)/wt | F | WMB-10Xv2 | 12.420248 | -10.367616 | 5076 |
| GTGTGATCACCAGCGT-468_A02 | GTGTGATCACCAGCGT | 468_A02 | L8TX_201217_01_D07 | WMB-10Xv3-MB | cell | NaN | 10Xv3 | MB | Snap25-IRES2-Cre;Ai14-556231 | Ai14(RCL-tdT)/wt | F | WMB-10Xv3 | 1.571478 | 23.592803 | 14956 |
| TAACCAGCACCAAATC-204.2_A01 | TAACCAGCACCAAATC | 204.2_A01 | L8TX_200122_01_F12 | WMB-10Xv3-MB | cell | NaN | 10Xv3 | MB | Snap25-IRES2-Cre;Ai14-504535 | Snap25-IRES2-Cre/wt;Ai14(RCL-tdT)/wt | F | WMB-10Xv3 | -12.491472 | 11.322906 | 5231 |
| AGCGTCGGTTCAGCGC-420_C06 | AGCGTCGGTTCAGCGC | 420_C06 | L8TX_201106_01_G03 | WMB-10Xv3-MB | cell | NaN | 10Xv3 | MB | Snap25-IRES2-Cre;Ai14-551017 | Ai14(RCL-tdT)/wt | M | WMB-10Xv3 | -10.406498 | 11.711900 | 5230 |
| AGACAAAAGAACTCCT-278_B03 | AGACAAAAGAACTCCT | 278_B03 | L8TX_200702_01_H01 | WMB-10Xv3-TH | cell | NaN | 10Xv3 | TH | Snap25-IRES2-Cre;Ai14-529474 | Snap25-IRES2-Cre/wt;Ai14(RCL-tdT)/wt | F | WMB-10Xv3 | 5.455338 | -13.476950 | 5101 |
In this case the feature metadata are stored in a separate csv. We need to read read and then add them to the right field in the AnnData object
- Add feature metadata to the AnnData object
We previously made sure that the genes in the csv file are sorted in the same way as the ones already present in the AnnData object, so you will just need to add it to the right field now)
# read
genes = pd.read_csv("../data/Practical5/gene.csv",index_col=0)
genes.head()
| gene_symbol | name | mapped_ncbi_identifier | comment | |
|---|---|---|---|---|
| gene_identifier | ||||
| ENSMUSG00000051951 | Xkr4 | X-linked Kx blood group related 4 | NCBIGene:497097 | NaN |
| ENSMUSG00000089699 | Gm1992 | predicted gene 1992 | NaN | NaN |
| ENSMUSG00000102331 | Gm19938 | predicted gene, 19938 | NaN | NaN |
| ENSMUSG00000102343 | Gm37381 | predicted gene, 37381 | NaN | NaN |
| ENSMUSG00000025900 | Rp1 | retinitis pigmentosa 1 (human) | NCBIGene:19888 | NaN |
# add
adata_sc.var = genes
# Reset the index to turn the current index into a column
adata_sc.var.reset_index(inplace=True)
adata_sc.var.set_index('gene_symbol', inplace=True)
adata_sc.var.head()
| gene_identifier | name | mapped_ncbi_identifier | comment | |
|---|---|---|---|---|
| gene_symbol | ||||
| Xkr4 | ENSMUSG00000051951 | X-linked Kx blood group related 4 | NCBIGene:497097 | NaN |
| Gm1992 | ENSMUSG00000089699 | predicted gene 1992 | NaN | NaN |
| Gm19938 | ENSMUSG00000102331 | predicted gene, 19938 | NaN | NaN |
| Gm37381 | ENSMUSG00000102343 | predicted gene, 37381 | NaN | NaN |
| Rp1 | ENSMUSG00000025900 | retinitis pigmentosa 1 (human) | NCBIGene:19888 | NaN |
# collect garbage
gc.collect()
13
Do some pre-processing for SpaGE:
# 1) Filter cells with < 10 expressed genes (counts > 0)
# Scanpy expects "min_genes" to count number of genes per cell (non-zero entries)
sc.pp.filter_cells(adata_sc, min_genes=20) # modifies adata_sc in-place
print(f"After sc.pp.filter_cells: {adata_sc.n_obs} cells, {adata_sc.n_vars} genes")
# 2) Per-cell normalization to counts-per-million (CPM) and log1p
# normalize_total works row-wise (per observation). target_sum=1e6 replicates your earlier scale.
sc.pp.normalize_total(adata_sc, target_sum=1e6, inplace=True) # scales rows so row-sum == 1e6
sc.pp.log1p(adata_sc) # in-place log(1 + X). Works with sparse X.
print("Applied per-cell normalize_total(target_sum=1e6) + log1p.")
# Optional step: memory efficiency purpose only!
# Compute HVGs using Seurat v3 flavor. This writes adata_sc.var['highly_variable'].
# You can change flavor to 'cell_ranger' or 'seurat_v3' depending on preference.
sc.pp.highly_variable_genes(adata_sc, n_top_genes=10000, flavor='seurat_v3', subset=False, inplace=True)
# Subset to keep only HVGs (creates a smaller AnnData)
hv_mask = adata_sc.var['highly_variable'].values.astype(bool)
print(f"Found {hv_mask.sum()} HVGs (requested {10000}). Subsetting AnnData...")
adata_sc = adata_sc[:, hv_mask].copy()
After sc.pp.filter_cells: 28981 cells, 32285 genes
Applied per-cell normalize_total(target_sum=1e6) + log1p.
Found 10007 HVGs (requested 10000). Subsetting AnnData...
- How many genes do the spatial and single-cell data set have in common?
- How many genes are measured in the single-cell dataset that are not measured in the spatial dataset?
## use .intersection
gene_intersection = set(adata_sc.var_names) & set(adata_section.var_names)
## use a set operation
genes_not_intersection = set(adata_sc.var_names) - set(adata_section.var_names)
print(f"Number of genes measured both in the sc and spatial datasets: {len(gene_intersection)}")
print(f"Number of genes measured in the sc reference only (after HVG): {len(genes_not_intersection)}")
Number of genes measured both in the sc and spatial datasets: 881 Number of genes measured in the sc reference only (after HVG): 9114
sc.pp.normalize_total(adata_section, target_sum=1_000_000, inplace=True)
sc.pp.log1p(adata_section)
Let's find some brain-specific genes to predict with SpaGE
brain_region_specific_genes = [
"Bcl11b", "Cux2", "Fezf2", "Tbr1", "Satb2", # isocortex
"Foxp2", # Striatum specific
"Penk", # Basal ganglia specific
"Zic1", # Cerebellum specific
"Hoxb2", # Hindbrain specific
"Dlx6", # Subpallium specific
]
Let's split them in overlap_genes (measured in spatial dataset and sc reference) and non_overlap_genes (unmeasured in the spatial dataset)
# Checking for genes from brain_region_specific_genes that are measured by spatial data
overlap_genes = list(set(adata_section.var.index) & set(brain_region_specific_genes))
overlap_genes
['Bcl11b', 'Zic1', 'Satb2', 'Fezf2', 'Cux2', 'Dlx6', 'Penk']
# Checking for genes from brain_region_specific_genes that are NOT measured by spatial data
non_overlap_genes = list(set(brain_region_specific_genes) - set(overlap_genes))
non_overlap_genes
['Tbr1', 'Hoxb2', 'Foxp2']
Leave-one-gene-out cross validation¶
Since ground-truth is available, we can leave out the measured (spatial) genes, predict their expression with SpaGE and then compare it to the real expression.
# sc df
RNA_data = adata_sc.to_df()
RNA_data = RNA_data.T
# spatial
spatial_data = adata_section.to_df()
Define the function to predict genes measured in the spatial dataset to have a ground truth and plot the measured expression next to the predicted one.
def predicted_GE_LOOCV(Gene_set, spatial_data, RNA_data):
# Initialize an empty DataFrame to store the predicted values
predicted_df = pd.DataFrame(index=spatial_data.index)
for i in Gene_set:
# Predict the expression of the current gene using the SpaGE method
Imp_Genes = SpaGE(spatial_data.drop(i, axis=1), RNA_data.T, n_pv=30, genes_to_predict=[i])
Imp_Genes.index = spatial_data.index
# Add the predicted values for the current gene as a new column in the DataFrame
predicted_df[i] = Imp_Genes[i]
return predicted_df
predicted_df_loocv = pd.read_csv("../data/Practical5/predicted_GE_loocv_SpaGE.csv",index_col=0)
predicted_df_loocv
| Satb2 | Penk | Fezf2 | Dlx6 | Zic1 | Bcl11b | Cux2 | |
|---|---|---|---|---|---|---|---|
| cell_label | |||||||
| 299794030110334465835331169565547130392 | 4.992982 | 0.151337 | 0.406908 | 0.000000 | 0.000000 | 0.238104 | 4.540980 |
| 66045375924099067525605874805924532486 | 5.040087 | 0.075470 | 1.575786 | 0.000000 | 0.000000 | 0.223969 | 4.403609 |
| 116791314340621211157298988846240231316 | 4.944482 | 0.151654 | 1.982587 | 0.000000 | 0.000000 | 0.225307 | 4.584788 |
| 13950435409550353848526753205083905255 | 5.509757 | 0.089223 | 2.470048 | 0.109389 | 0.000000 | 0.475855 | 3.652432 |
| 147436316185400258367147717470263928395 | 4.692426 | 0.226915 | 2.497056 | 0.109751 | 0.079881 | 0.181779 | 2.278905 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 138771139060049685442837890240266124575 | 0.059332 | 0.000000 | 0.000000 | 0.000000 | 0.079726 | 0.069395 | 0.146549 |
| 293280579073114219480073630362346662120 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.159701 | 0.056213 | 0.266262 |
| 29789632307409269107735464912172084480 | 0.059382 | 0.165090 | 0.098243 | 0.000000 | 0.159671 | 0.000000 | 0.959135 |
| 63992048193478335418190347526482691865 | 0.237527 | 0.378606 | 0.098315 | 0.000000 | 0.000000 | 0.056286 | 0.892634 |
| 217820882131375786118651898970174983535 | 3.167188 | 1.230329 | 0.120103 | 0.000000 | 0.000000 | 0.984083 | 4.537077 |
37068 rows Ć 7 columns
Since we have a ground truth, create a Series to store Spearman correlation results for each gene.
Our gene_set in this case is a set of genes shared between the spatial and the sc datasets.
from matplotlib import gridspec
from scipy.stats import spearmanr
plt.style.use('dark_background')
def plot_measured_vs_predicted_spatial(Gene_set, spatial_data, predicted_df, adata_section):
"""
Plot measured and predicted spatial expression for genes, with calculated Spearman correlations.
Parameters:
Gene_set (list): List of genes to visualize.
spatial_data (DataFrame): Measured expression data (columns are genes, rows are spots).
predicted_df (DataFrame): Predicted expression data (columns are genes, rows are spots).
adata_section (AnnData): Spatial data with 'x' and 'y' coordinates.
Returns:
dict: Spearman correlations for each gene.
"""
correlations = {}
for gene in Gene_set:
# Calculate Spearman correlation
correlation = spearmanr(spatial_data[gene], predicted_df[gene])[0]
correlations[gene] = correlation
# Create a figure with two scatterplots
fig = plt.figure(figsize=(15, 8)) # Adjust figure size
gs = gridspec.GridSpec(1, 3, width_ratios=[1, 1, 0.05], wspace=0.3) # Create grid spec for layout
# --- Measured Gene Expression ---
ax0 = fig.add_subplot(gs[0])
cmap_measured = spatial_data[gene]
cmap_measured = np.clip(cmap_measured, None, np.percentile(cmap_measured, 99)) # Clip to 99th percentile
sc1 = ax0.scatter(adata_section.obs['x'], -adata_section.obs['y'], s=1, c=cmap_measured, cmap="viridis", alpha=1)
ax0.set_title(f"Measured Expression: {gene}", fontsize=12)
ax0.set_xlabel("X Coordinate")
ax0.set_ylabel("Y Coordinate")
ax0.set_aspect("equal", adjustable="box")
ax0.grid(False)
# --- Predicted Gene Expression ---
ax1 = fig.add_subplot(gs[1])
cmap_predicted = predicted_df_loocv[gene]
cmap_predicted = np.clip(cmap_predicted, None, np.percentile(cmap_predicted, 99)) # Clip to 99th percentile
ax1.scatter(adata_section.obs['x'], -adata_section.obs['y'], s=1, c=cmap_predicted, cmap="viridis", alpha=1)
ax1.set_title(f"Predicted Expression: {gene}", fontsize=12)
ax1.set_xlabel("X Coordinate")
ax1.set_ylabel("Y Coordinate")
ax1.set_aspect("equal", adjustable="box")
ax1.grid(False)
# --- Colorbar ---
cbar_ax = fig.add_subplot(gs[2]) # Third subplot for color bar
cbar = fig.colorbar(sc1, cax=cbar_ax)
cbar.set_label("Expression")
# Adjust layout and show
plt.tight_layout()
plt.show()
return correlations
correlations = plot_measured_vs_predicted_spatial(overlap_genes, spatial_data, predicted_df_loocv, adata_section)