smoother.utils ============== .. py:module:: smoother.utils .. autoapi-nested-parse:: Utility functions Functions --------- .. autoapisummary:: smoother.utils.normalize_minmax smoother.utils._z_score smoother.utils._pca smoother.utils.calc_feature_similarity_sparse smoother.utils.calc_feature_similarity smoother.utils.get_histology_vector smoother.utils.calc_histology_similarity_sparse smoother.utils.calc_histology_similarity smoother.utils.quantile_feature_similarity_neighbor smoother.utils.quantile_feature_similarity_decay smoother.utils.get_neighbor_quantile_value_by_k smoother.utils.plot_similarity_cdf smoother.utils.plot_similarity_decay smoother.utils._mono_exp smoother.utils.estimate_decay_rate Module Contents --------------- .. py:function:: normalize_minmax(x, min_zero=True, return_scale=False) Normalize data to [0, 1] or [-1, 1]. :param x: data to be normalized. :type x: 2D array :param min_zero: If True, set column minimum to 0. :type min_zero: bool :param return_scale: If True, return the scaling factor. :type return_scale: bool .. py:function:: _z_score(features) Z-score standardization. .. py:function:: _pca(features, dim) Dimension reduction by PCA. .. py:function:: calc_feature_similarity_sparse(features: torch.Tensor, indices: torch.Tensor, dist_metric='cosine', reduce='none', dim=10, nonneg=False, return_type='flat') Calculate pairwise feature similarity between spots for a given set of spot pairs. Similarity `s` is transformed from distance `d` by 1) Cosine similarity: s = (1 - d) (if nonneg, (1 - d).clamp(min = 0)) 2) Others: s = exp(- d^2/(2 * band_width^2))) :param features: Feature matrix, num_feature x num_spot. :type features: 2D tensor :param indices: Pairs of spot indices of which to calculate the similarity, 2 x num_pairs. :type indices: 2D tensor :param dist_metric: Distance metric. :type dist_metric: str :param reduce: If `PCA`, calculate distance on the reduced PCA space. :type reduce: str :param dim: Number of dimension of the reduced space. :type dim: int :param nonneg: If True, set negative similarity to 0. :type nonneg: bool :param return_type: If `sparse`, return a sparse tensor. If `dense`, return a dense tensor. If `flat`, return a flat tensor of length num_pairs. :type return_type: str :returns: A 2D tensor containing pairwise similarities, num_spot x num_spot. :rtype: feature_sim .. py:function:: calc_feature_similarity(features: torch.Tensor, dist_metric='cosine', reduce='pca', dim=10, nonneg=False) Calculate pairwise feature similarity between spots. Similarity `s` is transformed from distance `d` by 1) Cosine similarity: s = (1 - d) (if nonneg, (1 - d).clamp(min = 0)) 2) Others: s = exp(- d^2/(2 * band_width^2))) :param features: Feature matrix, num_feature x num_spot. :type features: 2D tensor :param dist_metric: Distance metric. :type dist_metric: str :param reduce: If `PCA`, calculate distance on the reduced PCA space. :type reduce: str :param dim: Number of dimension of the reduced space. :type dim: int :param nonneg: If True, set negative similarity to 0. :type nonneg: bool :returns: A 2D tensor containing pairwise similarities, num_spot x num_spot. :rtype: feature_sim .. py:function:: get_histology_vector(image, x_pixel, y_pixel, spot_radius, scale_factor, padding=True) Get the histology image vector of one spot. :param image: Histology image, num_pixel x num_pixel x num_channel. :type image: 3D array :param x_pixel: Spot centric position (in fullres). :type x_pixel: float :param y_pixel: Spot centric position (in fullres). :type y_pixel: float :param spot_radius: Spot size (in fullres). :type spot_radius: float :param scale_factor: Scale factor that transforms fullres image to the given image. :type scale_factor: float :param paddings: Whether to pad for boundary spots. If False, will return the averaged color vector. :type paddings: bool :returns: A vector containing histology information around the spot. :rtype: spot_vec (1D array) .. py:function:: calc_histology_similarity_sparse(coords, indices, image, scale_factors, dist_metric='euclidean', reduce='none', dim=3, nonneg=True) Calculate pairwise histology similarity between spots for a given set of points. :param coords: Spatial coordinate matrix (in fullres pixel), num_spot x 2. :type coords: 2D array :param indices: Pairs of spot indices of which to calculate the similarity, 2 x num_pairs. :type indices: 2D tensor :param image: Histology image, num_pixel x num_pixel x num_channel. :type image: 3D array :param scale_fators: The JSON dictionary from 10x Visium's `scalefactors_json.json` 'spot_diameter_fullres' (float): Spot size (fullres) 'tissue_hires_scalef' (float): Scale factor that transforms fullres image to the given image. :type scale_fators: dict :param reduce: How to compute histological similarity. Can be one of 'pca', 'mean', and 'none'. If 'none', will concatenate pixel-level histology vectors of each spot and calculate distance. If 'pca', will concatenate pixel-level histology vectors of each spot, apply PCA to reduce the dimension of the histology space, then calculate the distance. If 'mean', will average the histology vector of each spot over its covering area. :type reduce: str :param dim: Number of dimension of the reduced space. :type dim: int :param nonneg: Whether to remove negative similarity. :type nonneg: bool :returns: A 2D tensor containing pairwise histology similarities, num_spot x num_spot. :rtype: hist_sim .. py:function:: calc_histology_similarity(coords, image, scale_factors, dist_metric='euclidean', reduce='pca', dim=3, nonneg=True) Calculate pairwise histology similarity between spots. :param coords: Spatial coordinate matrix (in fullres pixel), num_spot x 2. :type coords: 2D array :param image: Histology image, num_pixel x num_pixel x num_channel. :type image: 3D array :param scale_fators: The JSON dictionary from 10x Visium's `scalefactors_json.json` 'spot_diameter_fullres' (float): Spot size (fullres) 'tissue_hires_scalef' (float): Scale factor that transforms fullres image to the given image. :type scale_fators: dict :param reduce: How to compute histological similarity. Can be one of 'pca', 'mean', and 'none'. If 'none', will concatenate pixel-level histology vectors of each spot and calculate distance. If 'pca', will concatenate pixel-level histology vectors of each spot, apply PCA to reduce the dimension of the histology space, then calculate the distance. If 'mean', will average the histology vector of each spot over its covering area. :type reduce: str :param dim: Number of dimension of the reduced space. :type dim: int :param nonneg: Whether to remove negative similarity. :type nonneg: bool :returns: A 2D tensor containing pairwise histology similarities, num_spot x num_spot. :rtype: hist_sim .. py:function:: quantile_feature_similarity_neighbor(features, coords, k, reduce='pca', dim=20, n_null=100) Calculate cosine similarity of features between physical k-nearest neighbors. :param features: Feature matrix, num_gene x num_spot. :type features: 2D tensor :param coords: Coordinates of spots, num_spot x 2. :type coords: 2D tensor :param k: Number of neighbors. :type k: int :param reduce: Dimension reduction method, 'pca' or 'none'. :type reduce: str :param dim: Number of dimensions to reduce to. :type dim: int :param n_null: Number of null distribution samples. :type n_null: int :returns: Quantiles of cosine similarity between neighbors. :rtype: df (dataframe) .. py:function:: quantile_feature_similarity_decay(features, coords, max_k=50, topk=False, reduce='pca', dim=20) Calculate cosine similarity of features between physical neighbors of varying degrees. :param features: Feature matrix, num_gene x num_spot. :type features: 2D tensor :param coords: Coordinates of spots, num_spot x 2. :type coords: 2D tensor :param max_k: Maximum number of neighbors. :type max_k: int :param topk: Whether to keep all neighbors within the threshold or only the k nearest neighbors. :type topk: bool :param reduce: Dimension reduction method, 'pca' or 'none'. :type reduce: str :param dim: Number of dimensions to reduce to. :type dim: int :returns: Quantiles of cosine similarity between neighbors. :rtype: df (dataframe) .. py:function:: get_neighbor_quantile_value_by_k(sq_mat, coords, min_k=0, max_k=50) Extract values of neighboring pairs of spots from the square matrix. :param sq_mat: Data to extract, num_spot x num_spot. :type sq_mat: 2D tensor :param coords: Coordinates of spots, num_spot x 2. :type coords: 2D tensor :param min_k: Minimum number of neighbors. k = 0: self. :type min_k: int :param max_k: Maximum number of neighbors. :type max_k: int .. py:function:: plot_similarity_cdf(df_k, title='', k=None) Plot cumulative distribution of pairwise cosine similarity between neighboring spots. :param df_k: Quantiles of cosine similarity between neighbors. Output of quantile_feature_similarity_neighbor. :type df_k: dataframe :param title: Plot title. :type title: str :param k: Number of neighbors (labeling only). :type k: int .. py:function:: plot_similarity_decay(df_all, title='', plot_type='shadow') Plot decay of pairwise cosine similarity between neighboring spots. :param df_all: Quantiles of cosine similarity between neighbors. Output of quantile_feature_similarity_decay. :type df_all: dataframe :param title: Plot title. :type title: str :param plot_type: Type of plot, "shadow", "boxplot" or "lineplot". :type plot_type: str .. py:function:: _mono_exp(x, alpha, beta) .. py:function:: estimate_decay_rate(x, y, return_all_params=False) Estimate decay rate of a mono-exponential decay function. :param x: x values. :type x: array :param y: y values. :type y: array :param return_all_params: If False, only return decay rate (beta). :type return_all_params: bool