for Cell Instance Segmentation in Microscopy Images
|
|
|
|
|
|
|
|
|
 |
|
Method overview and example segmentations on diverse
datasets.
Top row: An unsupervised learning objective gives rise to
object-centric embeddings (OCEs), such that patches extracted
from the same object (green boxes) maintain their relative
position to each other. Predicted densely, these OCEs allow
instance segmentation of cells in microscopy images, by using
a post-processing step such as mean-shift clustering. Bottom
row: Example raw images and dense OCEs/instance segmentations
on four datasets spanning different imaging modalities, cell
sizes and shapes.
|
Segmentation of objects in microscopy images is required for many biomedical applications. We introduce object-centric embeddings (OCEs), which embed image patches such that the spatial offsets between patches cropped from the same object are preserved. Those learnt embeddings can be used to delineate individual objects and thus obtain instance segmentations. Here, we show theoretically that, under assumptions commonly found in microscopy images, OCEs can be learnt through a self-supervised task that predicts the spatial offset between image patches. Together, this forms an unsupervised cell instance segmentation method which we evaluate on nine diverse large-scale microscopy datasets. Segmentations obtained with our method lead to substantially improved results, compared to state-of-the-art baselines on six out of nine datasets, and perform on par on the remaining three datasets. If ground-truth annotations are available, our method serves as an excellent starting point for supervised training, reducing the required amount of ground-truth needed by one order of magnitude, thus substantially increasing the practical applicability of our method.
Overview
|
|
Unsupervised Learning of Object-Centric Embeddings. During learning, small image patches are randomly cropped
from the raw image and embedded through a learnable function
into a 2D embedding space. The objective of the loss is to
ensure that the spatial offset between pairs of patches in the
raw image (green arrows) is preserved in the embedding space.
|
Results
|
|
The pretrained models of StarDist and Cellpose are compared with
Cellulus on nine diverse microscopy image datasets. Two instance
segmentation metrics F1 and SEG are evaluated by comparing the
quality of predicted instance segmentation with the ground truth
instance segmentation. Best performing method on each dataset is
shown in bold. The last row TissueNet (all) shows a weighted
average (weights proportional to the number of images) of
results for Immune, Lung, Pancreas and Skin.
|
|
|
|
|
Acknowledgements
S.W., H.W. and K.M. are supported by the Medical Research Council,
as part of UK Research and Innovation [MCUP1201/23].
|