# Research Interests

We are interested in all aspects of mathematical imaging: the use of mathematical techniques to analyse and to improve real-world images, ranging from photographs made with consumer cameras to the images made with professional imaging devices in the sciences and medicine. These include techniques such as MRI (magnetic resonance imaging) and PET (positron emission tomography). Our current research concentrates in particular on higher order PDEs for image inpainting, and discontinuity-preserving higher-order variational approaches for the recovery of sparsely sampled data. Further themes include parameter learning, with the goal of building “black box” imaging tools suitable for use by non-professionals.

## Geometric Integration Methods for Optimisation

This project is concerned with the development and analysis of optimisation schemes based on geometric numerical integration methods. Discrete gradient methods are popular numerical schemes for solving systems of ODEs, and are known for preserving structures of the continuous system such as energy dissipation/conservation. Applying discrete gradients to dissipative ODEs/PDEs yields optimisation schemes that preserve the dissipative structure. For example, we consider a derivative-free discrete gradient method for optimising nonsmooth, nonconvex problems in a blackbox setting. This method has been shown to converge to optimal points of the objective function in a general, nonsmooth setting, while retaining favourable properties of gradient flow. This blackbox optimisation framework is useful, for instance, for bilevel optimisation of regularisation parameters in image processing.

## Flow of Microtubules in the Drosophila Oocyte

The focus of this project is to characterise directionality of plus ends of microtubules in confocal microscopy images of Drosophila embryos. This goal is particularly challenging due to the high noise level in such data, making it almost impossible to distinguish EB1 fluorescently labelled comets from randomly distributed noise. To overcome this problem, we employ recently developed methods for joint motion estimation and image reconstruction. As a result we are able to estimate motion in image sequences where other state of the art methods fail.

## Cellular Mechanics of Drosophila

The main goal of this project is to understand cellular processes in the developing embryo of Drosophila. We aim to identify the mechanisms and driving forces of tissue formation, tissue regrowth after incurring significant damage, and the collective motion of cells. Mathematical models of, for instance, cell membranes typically include unknown physical parameters. A well-established way to infer these from measurements is to probe tissue mechanics with the help of laser ablations in order to measure e.g. recoil velocities. In this project, we aim to utilise image analysis and inverse problems to estimate these parameters automatically and reliably from confocal microscopy images of live Drosophila embryos.

## Mathematical Challenges in Electron Tomography

Electron microscopy is a powerful tool in the physical, biological, and industrial sciences advancing areas from nanotechnology to drug discovery. Tomography is a mathematical technique used to recover full 3D information from a sequence of 2D images. One of the classical challenges here is to get the best quality reconstruction from the smallest amount of data. Some of our work has been to introduce novel and customised regularisation techniques to address such problems. Recent hardware advances have also extended this to spectral images, where pixel-wise values of the 2D images are vectors rather than greyscale. This data is also very slow to acquire so we need new methods to reconstruct from little and very noisy data. [Image from O. Nicoletti et al., Nature, 502 80-84 (2013)]

## Faster PET Reconstruction by Stochastic Optimisation

This project is concerned with the efficient reconstruction of positron emission tomography by means of stochastic optimisation. In the last decade, many mathematical tools have been developed that have the ability to enhance clinical imaging in various ways. On the forefront of this wave are non-smooth priors that allow the reconstruction of a smooth image but do not prohibit jumps across meaningful areas like organs in medical imaging. Beside this these new tools also allow the incorporation of a-priori structural knowledge about the solution at hand. However, most of this progress has not been translated into clinical practice as most modern algorithms are too demanding for the huge data sizes encountered. In the past, algorithms have been made “applicable” to clinical practice by only considering a subset of the data at a time. While for some models this leads satisfactory results, in general this ad-hoc strategy may yield to spurious artefacts. Motivated by the success of similar techniques in machine learning, in this project we extend modern algorithms for imaging that can handle non-smooth priors in a rigorous way to the subset setting by means of “randomisation”. While the algorithm and thus its iterates are random, the variance of these are low and converge quickly to the desired deterministic solution. For more information on this project, see the following publications:

[1] Ehrhardt, M. J., Markiewicz, P. J., Schönlieb, C.-B. (2018). Faster PET Reconstruction with Non-Smooth Priors by Randomization and Preconditioning, https://arxiv.org/abs/1808.07150.

[2] Ehrhardt, M. J., Markiewicz, P. J., Richtárik, P., Schott, J., Chambolle, A., & Schönlieb, C.-B. (2017). Faster PET Reconstruction with a Stochastic Primal-Dual Hybrid Gradient Method. In Proceedings of SPIE (Vol. 10394, pp. 1–12). San Diego. http://doi.org/10.1117/12.2272946.

[3] Chambolle, A., Ehrhardt, M. J., Richtárik, P., & Schönlieb, C.-B. (2017). Stochastic Primal-Dual Hybrid Gradient Algorithm with Arbitrary Sampling and Imaging Applications. to appear in SIAM Journal on Optimization. http://arxiv.org/abs/1706.04957.

## Segmentation and Relaxation Methods for Integer-Constrained Problems

In image segmentation problems, we look for a partition of the image domain into regions with certain characteristics. Such problems appear in areas such as image editing (separating foreground from background, merging multiple images), medical applications (separating gray and white matter, finding structures in medical images), and biological imaging (finding cells and nuclei, detecting cancerous cells). They can be viewed as variational problems with *integer* constraints, which are generally very hard to solve. We work on new ways of approximating such problems through convex relaxation techniques, encoding prior geometrical knowledge, and solving them numerically.

## Reconstruction of Digital Elevation Maps

In order to obtain high-quality height maps of terrain data, the usually sparse measurements need to be interpolated to a dense digital elevation map. We develop new higher-order regularizers for interpolating such maps that allow high-quality reconstruction from few measurements. The high order and non-convexity of these schemes makes them analytically as well as numerically very challenging.

## Model Selection by Bilevel Optimisation

A key issue in image denoising, and in inverse problems as a whole, is the correct choice of data priors and fidelity terms. Depending on this choice, different results are obtained. Several strategies, both physical - dictated by the physics behind the acquisition process - and statistically grounded (e.g. by estimating or learning noise and structure in the data), have been considered in the literature. We consider approaches that learn the model and the parameter choice by bilevel optimisation techniques.

## Non-smooth and Higher-order Variational & PDE Regularisation

One of the most successful image processing approaches is PDEs and variational models. Given a noisy image, its processed (denoised) version is computed as a solution of a PDE or as a minimiser of a functional (variational model). Both of these processes are regularising the given image. In favourable imaging approaches this is done by eliminating high-frequency features (noise) while preserving or even enhancing low-frequency features (object boundaries, edges). This gives rise to non-smooth, nonlinear terms in the model of possibly high differential order. The total variation regularizer is a typical example in this class. Beyond image denoising, such regularization procedures are successfully applied to image deblurring, inpainting and inverse problems in imaging in general. We are interested in theoretical and numerical aspects of non-smooth and, in particular, higher-order regularisation.

## High-resolution Magnetic Resonance Imaging & Emission Tomography

The quality of images reconstructed from measurements acquired with medical imaging tools such as magnetic resonance imaging (MRI) and emission tomography (PET and SPECT) usually suffers from acquisition noise and undersampling. For still being able to reconstruct high-resolution images the solution of the respective inverse problem is equipped with non-smooth regularizers - as outlined above. In this context we are interested in PET and dynamic MRI.

## Domain Decomposition Methods

We are interested in domain decomposition methods used in image processing. The following link gives more information on our research in this area: Domain decomposition methods for TV-minimization.

## Limited-Angle Tomography

In limited-angle tomography one can not look at an object from certain angles resulting in bad reconstructions of the image. To address this we apply a specialized inpainting method to the sinogram to fill in the missing data, which greatly improves the reconstruction.

## Unusual Boundary Conditions in Image Inpainting

Most inpainting algorithms implicitly assume that image data is to be extrapolated across all boundaries of the inpainting domain. This is not always appropriate - for example, the inpainting problems arising in 3D conversion and novel viewpoint synthesis give rise to two different types of boundaries - a data boundary across which we wish to extrapolate image information, as well as a free boundary where we do not. This issue together with the performance requirements of inpainting HD content in real or near to real time necessitate the development of specialized inpainting methods.

## Analysis of Neurological Structures in High Resolution MRI

The thalamus is a deep brain structure made of gray-matter, responsible to route nearly all afferent impulses to the cortex. It is subdivided in four major nuclei (each of them having specific functions and containing additional subdivisions). Thalamic changes have been related to several diseases, including Alzheimer's disease, Parkinson's disease, multiple sclerosis and other neurodegenerative dementias. The aim of the project is to divide the thalamus in its major nuclei using high resolution, multi-contrast MRI information.

## Dictionary-based Segmentation by Graph Clustering

A male pied flycatcher is a bird characterised by a distinctive bright blaze on his forehead whose shape and brightness vary among the other birds in the group. Such properties reflect behavioural and biological attitudes of the bird in the group, so a careful and accurate detection and measurement of this blaze is essential for statistical studies. We segment the blaze by using a dictionary-method that, by modelling the image as a graph and looking in the image for some characterising features of the blaze (such as texture and brightness) by a comparison with examples provided by the user, detects the blaze accurately.

## Quantitative Analysis of LiDAR, Hyperspectral, and RGB Images for Remote Sensing

The goal of this project is detection, segmentation, and quantitative analysis of each individual tree in forest regions from aerial images. Steps include registration and alignment of data from several sources, including LiDAR, hyperspectral, as well as RGB images, and the development of new detection methods for finding individual trees in large-scale datasets.

## Segmentation in Image Guided Radiotherapy for the Treatment of Prostate Cancer

We apply image segmentation methods in detecting the rectum in treatment day CT scan (MVCT) using prior manual segmentation information in planning day CT scan (kVCT). The segmentation result will be used to help clinicians to deliver proper dose to the cancer position of the patients and also help to analysis side effects quantitatively.

# Research Projects

## Current Projects

- Leverhulme Trust project on
*Unveiling the invisible*. Duration: January 2019 - December 2021. P.I.: C.-B. Schönlieb. Co.I.: S. Bucklow, A. Launaro, S. Panayotova. - Unilever and EPSRC IAA Partnership Development Award for
*Mathematical Image Analysis and Machine Learning for Better Food Microstructures*. Duration: March 2018 - February 2021. P.I.: C.-B. Schönlieb and P. Schütz. - NPL postdoctoral fellowship grant for
*The mathematics of measurement*. Duration: March 2018 - February 2021. P.I.: A. Forbes and C.-B. Schönlieb. - Global Alliance funding for
*Mathematical and statistical theory of imaging*. Duration: January 2017 - December 2018. P.I.: C.-B. Sch\"onlieb. - EPSRC Centre for
*Mathematical and Statistical Analysis of Multimodal Clinical Imaging*. Duration: March 2016 - February 2020. P.I.: C.-B. Schönlieb. Co-Is: J. Aston, S. Bohndiek, E. Bullmore, N. Burnet, T. Fokas, F. Gilbert, A. Hansen, S. Reichelt, J. Rudd, R. Samworth, G. Treece, G. Williams. - MSCA-RISE project
*Challenges in Preservation of Structure (CHiPS)*, EU Horizon 2020 program. Duration: January 2016 - December 2019. P.I.: Elena Celledoni (NTNU). Cambridge lead: C.-B. Schönlieb. - Leverhulme Trust project on
*Breaking the non-convexity barrier*. Duration: November 2015 - October 2018. P.I.: C.-B. Schönlieb. Co.I.: M. Benning, L. Gladden, M. Möller.

## Past Projects

- Alan Turing Institute seed funding for
*Personalised breast cancer screening*. Duration: October 2017 - April 2018 P.I.: M. van der Schaar and C.-B. Schönlieb. - LMS Undergraduate Research Bursary for
*Bilevel optimisation for learning the sampling pattern in Magnetic Resonance Tomography*, P.I.: C.-B. Schönlieb. Duration: July 2016 - August 2016. Co.I.s: M. Benning, M. Ehrhardt. - EPSRC project on Efficient computational tools for inverse imaging problems. Duration: 01 January 2015 - 31 December 2017. P.I: C.-B. Schönlieb. Co-I.: T. Valkonen.
- CCI Collaborative Fund on Assessing the conservation quality of tropical forest unmanned aerial vehicles. Duration: 01 September 2014 -- 01 September 2016. PIs: D. Coomes, J. Lindsell, C.-B. Schönlieb, T. Swinfield.
- Wellcome Trust/ University of Cambridge Senior ISSF internship for the project Development of Image Analysis Algorithms for Monitoring Forest Health from Aircraft. Duration: 01 May 2014 -- 31 March 2015. PIs: X. Cai, D. Coomes, C.-B. Schönlieb.
- LMS-Scheme 3 funding for four meetings to be held in the UK on Current frontiers in inverse problems: from theory to applications, 2014.
- Leverhulme Early Career Fellowship awarded by the Leverhulme Trust and the Newton Trust. Duration: 01 November 2013 -- 31 October 2016. PI: L. Lellmann
- EPSRC first grant Nr. EP/J009539/1 Sparse & Higher-order Image Restoration. Duration: 03 May 2012 – 02 May 2014. PI: C.-B. Schönlieb
- Royal Society International Exchange Award Nr. IE110314 High-order Compressed Sensing for Medical Imaging. Duration: 01 Jan 2012 – 31 Dec 2013. PI’s: M. Burger & C.-B. Schönlieb. See online news of the University of Münster.
- EPSRC / Isaac Newton Trust Small Grant Non-smooth geometric reconstruction for high resolution MRI imaging of fluid transport in bed reactors. Duration: 01 July 2012 - 30 June 2013. PI: C.-B. Schönlieb
- Mathworks Academic Support for Development of Matlab Tools for the Numerical Analysis Tripos. Duration: July-September 2012. P.I.’s: S. Cowley, A. Iserles, C.-B. Schönlieb and A. Shadrin.