smoother.models.impute

Impute spatial feature of interest with spatial loss

Classes

`SpatialLoss`	Spatial loss.
`ImputeModel`	Class for different implementations of spatial imputation.
`ImputeTorch`	Imputation solver implemented using pytorch.
`ImputeConvex`	Imputation solver implemented using cvxpy.

Module Contents

class smoother.models.impute.SpatialLoss(prior, spatial_weights: List[smoother.weights.SpatialWeightMatrix] = None, rho=1, use_sparse=True, standardize_cov=False)

Bases: torch.nn.Module

Spatial loss.

The spatial smoothing loss on a spatial random variable (num_group x num_spot).

prior

The prior spatial process model, can be one of ‘sma’,’sar’, ‘car’, ‘icar’.

Type:: str

spatial_weights

Spatial weight matrix collection of length num_group or 1. If 1 then all groups will be subject to the same covariance.

Type:: List[SpatialWeightMatrix]

rho

The spatial autocorrelation parameter (for SpatialWeightMatrix.get_inv_cov).

Type:: float

use_sparse

Whether to use sparse inverse covariance matrix in the calculation.

Type:: bool

standardize_cov

Whether to standardize the covariance matrix to have same variance (1) across locations. Only proper covariance can be standardized.

Type:: bool

inv_cov

Inverse covariance (precision) matrix of spatial variables of each group, (num_group or 1) x n x n. If the first dimension has length 1, all groups will have the same covariance structure.

Type:: 3D tensor

inv_cov_2d_sp

Sparse block diagonal inverse covariance (precision) matrix.

Type:: 2D sparse tensor

confidences

Relative prior confidence of each group. The higher the confidence, the stronger the smoothing will be. If float, all groups will have the same confidence.

Type:: 1D tensor or float

_sanity_check() → None: Check whether the spatial loss is defined properly.

estimate_confidence(ref_exp, st_exp, method='lr') → None

Estimate the relative confidence for each group.

The covariance matrix will be scaled accordingly.

Parameters:

ref_exp (2D tensor) – Bulk expression signiture matrix, num_gene x num_group.
st_exp (2D tensor) – Spatial expression matrix, num_gene x num_spot.
method (str) – Method used to estimate variance.

forward(coefs, normalize=True)

Calculate spatial loss.

Parameters:: coefs (2D tensor) – Columns of regression coefficients, num_group x num_spot.

calc_corr_decay_stats(coords, min_k=0, max_k=50, cov_ind=0, return_var=False)

Calculate spatial covariance decay over degree of neighborhoods.

Covariance measured between k-nearest neighbors. If the number of covariance matrices (i.e. self.inv_cov.shape[0]) is larger than 1, use ‘cov_ind’ to select the covariance matrix to use.

Parameters:

coords (2D tensor) – Coordinates of spots, num_spot x 2.
min_k – Minimum number of k in k-nearest neighbors. k = 0: self.
max_k – Maximum number of k in k-nearest neighbors.
cov_ind (int) – Index of the covariance matrix to use.
return_var (bool) – Whether to return variance stats.

Returns:

Correlation decay quantiles. var_quantiles_df (pd.DataFrame): Per-spot variance quantiles.

Return type:

corr_decay_quantiles_df (pd.DataFrame)

class smoother.models.impute.ImputeModel(y_obs, spatial_loss_all: smoother.losses.SpatialLoss, fixed_obs=True, nonneg=False, **kwargs)

Class for different implementations of spatial imputation.

This class is designed for coorperative inheritance (mixin).

y_obs

Observed spatial feature matrix, n_obs x n_feature.

Type:: 2D tensor

spatial_loss_all

Spatial loss object build from the combined spatial coordinates containing both observed (the first n_obs rows) and missing spots (the rest rows).

Type:: SpatialLoss

fixed_obs

Whether to allow updates in the observed data.

Type:: bool

nonneg

Whether to enforce nonnegativity on spatial features.

Type:: bool

n_feature

Number of spatial features to process.

Type:: int

n_all

Number of total spots. n_all = n_obs + n_missing.

Type:: int

n_obs

Number of observed spots.

Type:: int

n_missing

Number of missing spots.

Type:: int

var_obs: Observed spatial feature matrix, n_obs x n_feature.

var_missing: Missing spatial feature matrix to impute, n_missing x n_feature.

var_all: Concatenated observed and missing spatial feature matrix, n_all x n_feature.

impute_configs

Dictionary of imputation configurations.

Type:: dict

impute_time

Time spent on deconvolution.

Type:: float

impute_flag

Whether the imputation was successful.

Type:: bool

set_params(y_obs, spatial_loss_all, fixed_obs=True, nonneg=False): Set configuration parameters and initialize variables.

abstract set_variables(): Set variables to impute.

abstract get_recon_loss(): Get reconstruction loss.

abstract get_sp_loss(): Get spatial loss (weighted by lambda_spatial_loss).

abstract get_results(): Get spatial features after imputation.

class smoother.models.impute.ImputeTorch(y_obs, spatial_loss_all: smoother.losses.SpatialLoss, fixed_obs=True, nonneg=False, **kwargs)

Bases: ImputeModel, torch.nn.Module

Imputation solver implemented using pytorch.

var_obs

Observed spatial feature matrix, n_obs x n_feature.

Type:: 2D tensor

var_missing

Missing spatial feature matrix to impute, n_missing x n_feature.

Type:: 2D tensor

var_all

Concatenated observed and missing spatial feature matrix, n_all x n_feature.

Type:: 2D tensor

See super class attributes for additional arguments.

set_variables(): Initialize torch parameters to estimate.

forward(y_obs, **kwargs): Run imputation for a new set of spatial features.

get_recon_loss()

Calculate loss of deviation from observation.

If the observed data is not fixed, then the loss is used to make sure data does not deviate too much from the observed value after imputation.

get_sp_loss(): Calculate loss of spatial smoothness.

get_results()

Get spatial features of all spots after imputation.

The first n_obs spots are the observed spots and the rest are the missing ones. If fixed_obs is True, then the returned observed data is not updated.

final_sanity_check()

impute(lambda_spatial_loss=1.0, lr=0.001, max_epochs=1000, patience=10, tol=1e-05, verbose=True, quiet=False) → bool

Run imputation algorithm.

Parameters:

lambda_spatial_loss (float) – Specifies the strength of the spatial smoothing.
lr (float) – Learning rate.
max_epochs (int) – Maximum number of training epochs. If -1, iterate until convergence (d_loss < tol).
patient (int) – Number of epochs to wait for the loss to decrease before stopping.
tol (float) – Tolerance of loss convergence.
verbose (bool) – If True, print out loss while training.
quiet (bool) – If True, no output printed.

Returns:

True if imputation is successful.

Return type:

bool

class smoother.models.impute.ImputeConvex(y_obs, spatial_loss_all: smoother.losses.SpatialLoss, fixed_obs=True, nonneg=False, **kwargs)

Bases: ImputeModel

Imputation solver implemented using cvxpy.

var_obs: Observed spatial feature matrix, n_obs x n_feature.

var_missing: Missing spatial feature matrix to impute, n_missing x n_feature.

var_all: Concatenated observed and missing spatial feature matrix, n_all x n_feature.

recon_loss_exp

Reconstruction loss expression.

Type:: cp.Expression

sp_loss_exp

Spatial loss expression.

Type:: cp.Expression

See super class attributes for additional arguments.

set_variables(): Initialize parameters to estimate.

set_recon_loss()

Set loss expression of deviation from observation.

If the observed data is not fixed, then the loss is used to make sure data does not deviate too much from the observed value after imputation.

set_sp_loss(): Set spatial loss expression (before weighted by lambda_spatial_loss).

get_recon_loss()

Calculate loss of deviation from observation.

If the observed data is not fixed, then the loss is used to make sure data does not deviate too much from the observed value after imputation.

get_sp_loss(): Calculate loss of spatial smoothness (weighted by lambda_spatial_loss).

get_results()

Get spatial features of all spots after imputation.

The first n_obs spots are the observed spots and the rest are the missing ones. If fixed_obs is True, then the returned observed data is not updated.

impute(lambda_spatial_loss=1.0, verbose=False, quiet=False, solver=None, **kwargs) → bool

Run imputation algorithm.

Parameters:

lambda_spatial_loss (float) – Specifies the strength of the spatial smoothing.
verbose (bool) – If True, print out loss while training.
quiet (bool) – If True, no output printed.

Returns:

True if imputation is successful.

Return type:

bool