Variational Regularisation for Inverse Problems
In this project, we introduce a new higher-order total directional variation (TDV) regulariser for inverse imaging problems by taking into account the image gradient weighted by the structural content.
We generalise previous work on the directional total variation, e.g. the definitions (DTV, wTV, dTV) given in Ehrhardt and Betcke (2016) and in Kongsgov and Dong (2017), by considering spatially dependent directions within the image domain and higher-order derivatives. Also, we extend previous work on total generalised variation (TGV) regulariser, see Bredies, Kunish and Pock (2010), Bredies (2013) and Bredies and Holler (2014), by showing that TGV is a special case of our regulariser.
We describe both the theoretical and the numerical details for its use within a variational formulation for solving inverse problems and give examples for the reconstruction of noisy images and videos, image zooming and the interpolation of scattered surface heigh value data from Digital Elevation Maps (DEM) and Atomic Force Microscopy (AFM).
Image Denoising
Image Zooming (4x)
Scattered Data Interpolation (DEM)
Scattered Data Interpolation (AFM)
Journal Articles |
Parisotto, Simone; Lellmann, Jan; Masnou, Simon; Schönlieb, Carola-Bibiane: Higher-order total directional variation: Imaging Applications. In: SIAM Journal on Imaging Sciences, 13 (4), pp. 2063–2104, 2020. (Type: Journal Article | Abstract | Links)@article{ParLelMasSch2020,We introduce a class of higher-order anisotropic total variation regularizers, which are defined for possibly inhomogeneous, smooth elliptic anisotropies, that extends the total generalized variation regularizer and its variants. We propose a primal-dual hybrid gradient approach to approximating numerically the associated gradient flow. This choice of regularizers allows us to preserve and enhance intrinsic anisotropic features in images. This is illustrated on various examples from different imaging applications: image denoising, wavelet-based image zooming, and reconstruction of surfaces from scattered height measurements. |
Parisotto, Simone; Masnou, Simon; Schönlieb, Carola-Bibiane: Higher-order Total Directional Variation: Analysis. In: SIAM Journal on Imaging Sciences, 13 (1), pp. 474–496, 2020. (Type: Journal Article | Abstract | Links)@article{ParMasSch2020,We analyze a new notion of total anisotropic higher-order variation which, differently from total generalized variation in [K. Bredies, K. Kunisch, and T. Pock, SIAM J. Imaging Sci., 3 (2010), pp. 492–526], quantifies for possibly nonsymmetric tensor fields their variations at arbitrary order weighted by possibly inhomogeneous, smooth elliptic anisotropies. We prove some properties of this total variation and of the associated spaces of tensors with finite variations. We show the existence of solutions to a related regularity-fidelity optimization problem. We also prove a decomposition formula which appears to be helpful for the design of numerical schemes, as shown in a companion paper, where several applications to image processing are studied. |
Parisotto, Simone; Calatroni, Luca; Caliari, Marco; Schönlieb, Carola-Bibiane; Weickert, Joachim: Anisotropic osmosis filtering for shadow removal in images. In: Inverse Problems, 35 (5), 2019. (Type: Journal Article | Abstract | Links)@article{ParCalCalSchWei2019,We present an anisotropic extension of the isotropic osmosis model that has been introduced by Weickert et al (2013 Energy Minimization Methods in Computer Vision and Pattern Recognition (Berlin: Springer)) for visual computing applications, and we adapt it specifically to shadow removal applications. We show that in the integrable setting, linear anisotropic osmosis minimises an energy that involves a suitable quadratic form which models local directional structures. In our shadow removal applications we estimate the local structure via a modified tensor voting approach (Moreno et al 2012 New Developments in the Visualization and Processing of Tensor Fields (Berlin: Springer)) and use this information within an anisotropic diffusion inpainting that resembles edge-enhancing anisotropic diffusion inpainting (Galić et al 2008 J. Math. Imaging Vis. 31 255–69; Weickert and Welk 2006 Visualization and Processing of Tensor Fields (Berlin: Springer)). Our numerical scheme combines the nonnegativity preserving stencil of Fehrenbach and Mirebeau (2014 J. Math. Imaging Vis. 49 123–47) with an exact time stepping based on highly accurate polynomial approximations of the matrix exponential. The resulting anisotropic model is tested on several synthetic and natural images corrupted by constant shadows. We show that it outperforms isotropic osmosis, since it does not suffer from blurring artefacts at the shadow boundaries. |
Burger, Martin; Korolev, Yury; Parisotto, Simone; Schönlieb, Carola-Bibiane: Total Variation Regularisation with Spatially Variable Lipschitz Constraints. In: (submitted, ArXiv: 1912.02768), 2019. (Type: Journal Article | Abstract | Links)@article{BurKorParSch2019,We introduce a first order Total Variation type regulariser that decomposes a function into a part with a given Lipschitz constant (which is also allowed to vary spatially) and a jump part. The kernel of this regulariser contains all functions whose Lipschitz constant does not exceed a given value, hence by locally adjusting this value one can determine how much variation is the reconstruction allowed to have. We prove regularising properties of this functional, study its connections to other Total Variation type regularisers and propose a primal dual optimisation scheme. Our numerical experiments demonstrate that the proposed first order regulariser can achieve reconstruction quality similar to that of second order regularisers such as Total Generalised Variation, while requiring significantly less computational time. |
Inproceedings |
Parisotto, Simone; Schönlieb, Carola-Bibiane: Total Directional Variation for Video Denoising. In: Burger M. Lellmann J., Modersitzki J. (Ed.): Scale Space and Variational Methods in Computer Vision (SSVM 2019), Lecture Notes in Computer Science, Springer, 2019. (Type: Inproceedings | Abstract | Links)@inproceedings{ParSch2019,In this paper we propose a variational approach for video denoising, based on a total directional variation (TDV) regulariser proposed in Parisotto et al. [20, 21] for image denoising and interpolation. In the TDV regulariser, the underlying image structure is encoded by means of weighted derivatives so as to enhance the anisotropic structures in images, e.g. stripes or curves with a dominant local directionality. For the extension of TDV to video denoising, the space-time structure is captured by the volumetric structure tensor guiding the smoothing process. We discuss this and present our whole video denoising workflow. The numerical results are compared with some state-of-the-art video denoising methods. |
PhD Theses |
Parisotto, Simone: Anisotropic variational models and PDEs for inverse imaging problems. University of Cambridge, 2019. (Type: PhD Thesis | Abstract | Links)@phdthesis{Parisotto2019,Supervisor: Prof Carola-Bibiane Schönlieb (University of Cambridge) Co-supervisor: Prof Simon Masnou (Université Lyon 1) In this thesis we study new anisotropic variational regularisers and partial differential equations (PDEs) for solving inverse imaging problems that arise in a variety of real-world applications. Firstly, we introduce a new anisotropic higher-order total directional variation regulariser. We describe both the theoretical and the numerical details for its use within a variational formulation for solving inverse problems and give examples for the reconstruction of noisy images and videos, image zooming and the interpolation of scattered surface data. Secondly, we focus on a non-symmetric drift-diffusion equation, called osmosis. We propose an efficient numerical implementation of the osmosis equation, based on alternate directions and operator splitting techniques. We study their scale-space properties and show their efficiency in processing large images. Moreover, we generalise the osmosis equation to accommodate suitable directional information: this modification turns out to be useful to correct for the well-known blurring artefacts the original osmosis model introduces when applied to shadow removal in images. Last but not least, we explore applications of variational models and PDEs to cultural heritage conservation. We develop a new non-invasive technique that uses multi-modal imaging for detecting sub-superficial defects in fresco walls at sub-millimetre precision. We correct light-inhomogeneities in these imaging measurements that are due to measurement errors via osmosis filtering, in particular making use of the efficient computational schemes that we introduced before for dealing with the large-scale nature of these measurements. Finally, we propose a semi-supervised workflow for the detection and inpainting of defects in damaged illuminated manuscripts. Keywords: Total directional variation, anisotropic diffusion, osmosis filter, cultural heritage conservation, primal-dual hybrid gradient, dimensional splitting, inverse problems, image denoising, video denoising, image zooming, surface interpolation, digital elevation maps, shadow removal, thermal quasi-reflectography, non-destructive imaging, dual-mode mid-infrared imaging, inpainting, illuminated manuscripts. |