Introduction to the project

The impact of AI in scientific image processing. - Dr. Deborah Schmidt (HELMHOLTZ IMAGING - Max Delbrück Center)

Background on Helmholtz Imaging

• Helmholtz is Germany’s largest research organization
• Helmholtz Imaging is a platform initiated by the Helmholtz Information & Data Science Incubator
• Helmholtz Imaging’s mission is to unlock the potential of imaging in the Helmholtz Association across al research domains and along the entire imaging pipeline.

Advances in scientific image processing through AI

AI in Enhanced Image Reconstruction

• AI-Enhanced Visualization of Cosmic Phenomena
• The image of the M87 black hole reconstructed with PRIMO.

AI in Image Enhancement

• CARE
  Content-aware image restoration: pushing the limits of fluorescence microscopy.

AI in Denoising

• Noise2Void
  Mpose2vpod-learning denoising from single noisy images.

AI in Segmentation

• nnU-Net
  A self-configuring method for deep learning-based biomedical image segmentation.
  
  
  
  

• CellPose
  A generalist algorithm for cellular segmentation.

AI in Image Exploration

• Terrapattern
  Open-Ended, Visual Query-By-Example for Satellite Imagery using Deep Learning.

AI in multimodal Data Handling

• Multi-sensor data fusion using deep learning for bulky waste image classification.
  Deep learning and data fusion to estimate surface soil moisture from multi-sensor satellite images.

AI in Synthetic Data Generation

• Generative Lung Architecture Modeling
  Segmentation and mesh generation -> Generative lung tissue datasets -> 3d bioprinting
  
  

Challenges of AI applications in scientific image processing

Challenges in applying AI based methods.

• Software publications in Computer Vision are hard to reproduce and lack user friendliness.
• Novel AI image processing methods often optimized for natural images (2D), not coping with diversity of scientific image datasets.
• Blurred line between user and developer.
• Validation of results can be challenging.
• (Undetected) bias in training data.
• Resource availability / conscious usage of resources.
  
  

Need of high quality annotations

• Ilastik - https://ilastik.org

• Labkit - https://imagej.net/plugins/labkit/

Choosing the right metrics

• Metrics reloaded: recommendations for image analysis validation.

Can we trust images?

• Will we be able to detect deep fakes in scientific imaging?

What’s next?

Individualized workflows based on Large Language Models

“We are currently living in a transformative period in which the century-old promises of AI are rapidly becoming reality.” ~ Royer, L.A. The future of bioimage analysis: a dialog between mind and machine. Nat Methods 20, 951-952 (2023).

Multimodal image processing

“A complete picture of how life functions can only be attained if we leverage all these technologies together. In essence, we reimagine the future of optical microscopy wherein we can image anything anywhere at any time.” ~ Balasubramanian, H., Hobson, C.M., Chew, TL. et al. Imagining the future of optical microscopy: everything, everywhere, all at once. Commun Biol 6, 1096 (2023).

Foundation Models

• Multimodal biomedical AI.

“The problem is that neither individuals nor governments seem to be able to follow the pace of these technological developments.” ~ Vinuesa, R., Azizpour, H., Leite, I. et al.The role of artificial intelligence in achieving the Sustainable Development Goals. Nat Commun 11, 233 (2020).