How can I convert coordinates between H&E image and Xenium data?

There are two directions for coordinate conversion:

1. Convert coordinates in H&E image to Xenium data, so we can map H&E image onto Xenium data.
2. Convert coordinates in Xenium data to H&E image, so we can map Xenium data onto H&E image

This article focuses on H&E images, however the strategy is similarly applicable to immunofluorescence (IF) images.

First, use Xenium Explorer to align H&E image to Xenium DAPI image (find the detailed instructions here). After alignment, download the transformation matrix CSV file from Xenium Explorer (find how to export transformation matrix csv here).

Here, we are using this breast cancer Xenium prime data as an example.

 

1. Convert coordinates in H&E image to Xenium data

After H&E image alignment in Xenium Explorer, we have the following transformation matrix exported from Xenium Explorer. This transformation matrix can be used to convert coordinates in H&E image to Xenium data.

0.011071028838929093,1.2895790134339569,807.9788953065927
-1.2895790134339569,0.011071028838929093,73097.655174759
0,0,1

First, we need to understand how to use this matrix for the conversion. If we have a transformation matrix M and a coordinate [x, y] in H&E image, the converted coordinate [x', y'] in Xenium data can be calculated as below.

For example, in the H&E image, the coordinate of the top-left corner is [0, 0]. After applying the calculation with the transformation matrix as above, its converted coordinate in Xenium data should be [x', y'] =[807.9788953065927, 73097.655174759]:

x' = (0*0.011071028838929093) + (0*1.2895790134339569) + 807.9788953065927 = 807.9788953065927

y' = (0*(-1.2895790134339569)) + (0*0.011071028838929093) + 73097.655174759 = 73097.655174759

Since here we are showing pixel values, while Xenium Explorer uses micron values, we need to convert this pixel coordinate to micron coordinate. We can find the pixel-to-micron conversion factor in the experiment.xenium file. Open experiment.xenium in any text editor and find "pixel_size" (pixel size in micron). Here, 1 pixel size = 0.2125 micron. 

Hence, the pixel coordinate [807.9788953065927, 73097.655174759] can be converted to micron coordinate as follows:

[807.9788953065927*0.2125, 73097.655174759*0.2125] = [171.69551525,15533.25172464]. 

Indeed, the micron coordinates displayed in Xenium Explorer for that H&E image are [171.7, 15533.3].

If you use third-party tools to do cell segmentation based on post-Xenium images, you can use Xenium Ranger import-segmentation pipeline to automatically convert all cell coordinates from the post-Xenium images to Xenium data. You will need to provide transformation matrix csv file to the parameter --coordinate-transform when running Xenium Ranger import-segmentation pipeline (find the more detailed instructions here).

2. Convert coordinates in Xenium data to H&E image

The transformation matrix exported from Xenium Explorer can be used to convert coordinates from H&E image to Xenium data. How can we convert coordinates from Xenium data to H&E image?  In the above case, since we have transformation matrix M, coordinates in the H&E image C can be converted to coordinates in Xenium data C'. In theory, if C' = M x C, then C = M^-1 x C'. If we have coordinates in Xenium data C' and want to get their converted coordinates in H&E image C, then we only need to get the inverse of the transformation matrix M^-1.

There are many packages that can calculate the inverse of a matrix. Below are two examples in R and python.

R:

library(matlib)
transform_mat <- read.csv("./imagealignment.csv", header = FALSE)
transform_mat <- as.matrix(transform_mat)
transform_mat_inv <- inv(transform_mat)

Python:

import numpy as np
import pandas as pd
mat = pd.read_csv("./imagealignment.csv", header=None)
mat_inv = np.linalg.inv(mat.values)

Using the above case, if we get the following transformation matrix (M) from Xenium Explorer:

0.011071028838929093,1.2895790134339569,807.9788953065927
-1.2895790134339569,0.011071028838929093,73097.655174759
0,0,1

Accordingly, we can calculate its inverse matrix (M^-1) as below:

0.00665672,-0.77538971,56673.792
0.77538971,0.00665672,-1113.089
0,0,1

For example, we have a nucleus in the same Xenium data whose coordinate is [5372.5, 12779.5] (in microns).

• First, we can convert micron coordinate to pixel coordinate: [5372.5/0.2125, 12779.5/0.2125]= [25282.35, 60138.82]. 
• Next, based on the above inverse matrix, we can convert this coordinate from Xenium data to H&E image: M^-1 x [25282.35, 60138.82, 1] = [25282.35*0.00665672 + 60138.82*(-0.77538971) + 1*56673.792, 25282.35*0.77538971 + 60138.82*0.00665672 + 1*(-1113.089), 25282.35*0 + 60138.82*0 + 1*1] = [10211.07, 18890.91, 1]. Therefore, the coordinates of that nucleus in the H&E image should be [10211.07, 18890.91]. 

Indeed, the real coordinate of that nucleus in the H&E image is the same as calculate