Title: Content-Adaptive Biomedical Image Understanding

Duration: June 2020 – October 2025

Research Area: Applied Data Science and AI, Life Science and Medicine

Biomedical images contain rich contextual information in which structures were labeled and how these structures are spatially related. However, biomedical images are often 3D volumes, acquired by tomography or light-sheet microscopy, of large size. This prevents the use of standard computer-vision techniques, and the question of how to extract and characterize information about shapes and their mutual spatial relations from such images remains open. In this project, we build on our previous work in content-adaptive image representation [1] to enable content-adaptive understanding of shapes and spatial arrangements in biomedical images using deep learning methods. This also includes the visualization of large distributed volume data for visual analytics and validation by human observers.

Aims

• Adopt the Adaptive Particle Representation (APR) [1] for end-to-end pipelines in biomedical imaging.
• Enable APR-native image processing and visualization.
• Develop a novel Convolutional Neural Network architecture to natively work on APRs.
• Enable interactive (low latency) visualization of large distributed volume data at high frame rates and in situ (i.e., without moving the data).
• Establish a scalable platform for Petabyte-scale biomedical imaging studies.

Problem

There are mainly two issues with understanding spatial structure in biomedical images: First, the image data are large (100s of Gigabytes to Terabytes for a single image), preventing single-GPU use of standard machine-learning frameworks. Second, information is mainly contained in the spatiotemporal dynamics of the signal intensity field, as morphology and texture are often dominated by the optics of the imaging equipment. This project provides a mathematically proven framework (proofs of global optimality, asymptotic convergence, error bounds, and computational equivalence) for addressing these questions and implements the solutions in practically useful and scalable software.

Practical Example

We are currently using the first results of this project in a collaboration with the Wyss Center for Neuroscience in Geneva to enable 3D neuro-histology of large human brain samples at sub-cellular resolution (see Scholler et al., 2023).

Technology

1. Adaptive Particle Representation of images [1]
2. Multi-level/multi-scale discrete convolutions
3. Deep learning
4. Asymptotic function approximation theory
5. Distributed parallel computing
6. View-dependent explorable volume representations

Outlook

Going forward, the project will expand along two orthogonal directions: First, we will research topological approaches to characterizing shapes in APRs of images. This will enable us to exploit the natural invariance properties of biomedical images. Second, we will bring the technology into applications, for example in large-scale neuro-histology or in high-content drug screening.

Publications

• J. Scholler, J. Jonsson, T. Jordá-Siquier, I. Gantar, L. Batti, B. Cheeseman, S. Pagès, I. F. Sbalzarini, C. M. Lamy. Efficient image analysis for large-scale next generation histopathology using pAPRica. bioRXiV, 10.1101/2023.01.27.525687v1, 2023.
• A. Gupta, P. Incardona, A. Brock, G. Reina, S. Frey, S. Gumhold, U. Günther, and I. F. Sbalzarini. Parallel compositing of volumetric depth images for interactive visualization of distributed volumes at high frame rates. In Proc. Eurographics Symposium on Parallel Graphics and Visualization (EGPGV), pages 25–35. The Eurographics Association, 2023. Best Paper Award.
• A. Gupta, U. Günther, P. Incardona, G. Reina, S. Frey, S. Gumhold, and I. F. Sbalzarini. Efficient raycasting of volumetric depth images for remote visualization of large volumes at high frame rates. In Proc. 16th IEEE Pacific Visualization Symposium (PacificVis), pages 61–70, Seoul, Korea, 2023. IEEE.
• J. Jonsson, B. L. Cheeseman, S. Maddu, K. Gonciarz, and I. F. Sbalzarini. Parallel discrete convolutions on adaptive particle representations of images. IEEE Trans. Image Process., 31:4197–4212, 2022.

Team

Lead

• Prof. Dr. Ivo F. Sbalzarini

Team Members

• Roua Rouatbi
• Aryaman Gupta
• Joel Jonsson

Partners

• Center for Systems Biology Dresden
• Wyss Center for Neuroscience Geneva
• University of Geneva, University of Stuttgart
• Oxford University

[1] B. L. Cheeseman, U. Günther, K. Gonciarz, M. Susik, and I. F. Sbalzarini. Adaptive particle representation of fluorescence microscopy images. Nat. Commun., 9:5160, 2018.