Cultural Heritage Imaging Science

Journal Articles

Parisotto, Simone; Calatroni, Luca; Bugeau, Aurélie; Papadakis, Nicolas; Schönlieb, Carola-Bibiane: Variational Osmosis for Non-linear Image Fusion. In: IEEE Transactions on Image Processing, 2020. (Type: Journal Article | Abstract | Links) @article{ParCalBugPapSch2020, title = {Variational Osmosis for Non-linear Image Fusion}, author = {Simone Parisotto and Luca Calatroni and Aurélie Bugeau and Nicolas Papadakis and Carola-Bibiane Schönlieb}, url = {https://arxiv.org/abs/1910.02012 https://github.com/simoneparisotto/Variational-Osmosis-for-Non-Linear-Image-Fusion}, doi = {10.1109/tip.2020.2983537}, year = {2020}, date = {2020-04-01}, journal = {IEEE Transactions on Image Processing}, abstract = {We propose a new variational model for non-linear image fusion. Our approach is based on the use of an osmosis energy term related to the one studied in Vogel et al. [44] and Weickert et al. [45]. The minimization of the proposed non-convex energy realizes visually plausible image data fusion, invariant to multiplicative brightness changes. On the practical side, it requires minimal supervision and parameter tuning and can encode prior information on the structure of the images to be fused. For the numerical solution of the proposed model, we develop a primal-dual algorithm and we apply the resulting minimization scheme to solve multi-modal face fusion, color transfer and cultural heritage conservation problems. Visual and quantitative comparisons to state-of-the-art approaches prove the out-performance and the flexibility of our method.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close We propose a new variational model for non-linear image fusion. Our approach is based on the use of an osmosis energy term related to the one studied in Vogel et al. [44] and Weickert et al. [45]. The minimization of the proposed non-convex energy realizes visually plausible image data fusion, invariant to multiplicative brightness changes. On the practical side, it requires minimal supervision and parameter tuning and can encode prior information on the structure of the images to be fused. For the numerical solution of the proposed model, we develop a primal-dual algorithm and we apply the resulting minimization scheme to solve multi-modal face fusion, color transfer and cultural heritage conservation problems. Visual and quantitative comparisons to state-of-the-art approaches prove the out-performance and the flexibility of our method. Close https://arxiv.org/abs/1910.02012 https://github.com/simoneparisotto/Variational-Osmosis-for-Non-Linear-Image-Fusi[…] doi:10.1109/tip.2020.2983537 Close

Daffara, Claudia; Parisotto, Simone; Ambrosini, Dario: A multipurpose, dual-mode imaging in the MWIR range for artwork diagnostic: a systematic approach. In: Optics and Lasers in Engineering, 2017. (Type: Journal Article | Abstract | Links) @article{DafParAmb2017, title = {A multipurpose, dual-mode imaging in the MWIR range for artwork diagnostic: a systematic approach}, author = {Claudia Daffara and Simone Parisotto and Dario Ambrosini}, doi = {10.1016/j.optlaseng.2017.10.006}, year = {2017}, date = {2017-10-01}, journal = {Optics and Lasers in Engineering}, abstract = {We present a multi-purpose, dual-mode imaging method in the Mid-Wavelength Infrared (MWIR) range (from 3 µm to 5 µm) for a more efficient nondestructive analysis of artworks. Using a setup based on a MWIR thermal camera and multiple radiation sources, two radiometric image datasets are acquired in different acquisition modalities, the image in quasi-reflectance mode (TQR) and the thermal sequence in emission mode. Here, the advantages are: the complementarity of the information; the use of the quasi-reflectance map for calculating the emissivity map; the use of TQR map for a referentiation to the visible of the thermographic images. The concept of the method is presented, the practical feasibility is demonstrated through a custom imaging setup, the potentiality for the nondestructive analysis is shown on a notable application to cultural heritage. The method has been used as experimental tool in support of the restoration of the mural painting “Monocromo” by Leonardo da Vinci. Feedback from the operators and a comparison with some conventional diagnostic techniques is also given to underline the novelty and potentiality of the method.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close We present a multi-purpose, dual-mode imaging method in the Mid-Wavelength Infrared (MWIR) range (from 3 µm to 5 µm) for a more efficient nondestructive analysis of artworks. Using a setup based on a MWIR thermal camera and multiple radiation sources, two radiometric image datasets are acquired in different acquisition modalities, the image in quasi-reflectance mode (TQR) and the thermal sequence in emission mode. Here, the advantages are: the complementarity of the information; the use of the quasi-reflectance map for calculating the emissivity map; the use of TQR map for a referentiation to the visible of the thermographic images. The concept of the method is presented, the practical feasibility is demonstrated through a custom imaging setup, the potentiality for the nondestructive analysis is shown on a notable application to cultural heritage. The method has been used as experimental tool in support of the restoration of the mural painting “Monocromo” by Leonardo da Vinci. Feedback from the operators and a comparison with some conventional diagnostic techniques is also given to underline the novelty and potentiality of the method. Close doi:10.1016/j.optlaseng.2017.10.006 Close

Book Chapters

Daffara, Claudia; Parisotto, Simone; Mariotti, Paola Ilaria: Infrared Analysis and ‘Thermal Quasi-Reflectography’. In: Leonardo Da Vinci – The Sala delle Asse of the Sforza Castle, pp. 186-199, Silvana Editrice, 2017, ISBN: 9788836636778. (Type: Book Chapter | Links) @inbook{ParisottoLeonardoBook2017, title = {Infrared Analysis and 'Thermal Quasi-Reflectography'}, author = {Claudia Daffara and Simone Parisotto and Paola Ilaria Mariotti}, url = {https://www.silvanaeditoriale.it/libro/9788836636778}, isbn = {9788836636778}, year = {2017}, date = {2017-06-01}, booktitle = {Leonardo Da Vinci - The Sala delle Asse of the Sforza Castle}, pages = {186-199}, publisher = {Silvana Editrice}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } Close https://www.silvanaeditoriale.it/libro/9788836636778 Close

Inproceedings

Daffara, Claudia; Parisotto, Simone; Mazzocato, Sara; Mariotti, Paola Ilaria; Ambrosini, Dario: Thermal imaging in the 3-5 micron range for precise localization of defects: application on frescoes at the Sforza Castle. In: Groves, Roger; Liang, Haida (Ed.): Optics for Arts, Architecture, and Archaeology VIII, SPIE, 2021. (Type: Inproceedings | Links) @inproceedings{Daffara2021, title = {Thermal imaging in the 3-5 micron range for precise localization of defects: application on frescoes at the Sforza Castle}, author = {Claudia Daffara and Simone Parisotto and Sara Mazzocato and Paola Ilaria Mariotti and Dario Ambrosini}, editor = {Roger Groves and Haida Liang}, url = {https://doi.org/10.1117/12.2593993}, doi = {10.1117/12.2593993}, year = {2021}, date = {2021-06-25}, booktitle = {Optics for Arts, Architecture, and Archaeology VIII}, publisher = {SPIE}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Close https://doi.org/10.1117/12.2593993 doi:10.1117/12.2593993 Close

Parisotto, Simone; Calatroni, Luca; Daffara, Claudia: Digital Cultural Heritage Imaging via Osmosis Filtering. In: Mansouri, Alamin; Moataz, Abderrahim El; Nouboud, Fathallah; Mammass, Driss (Ed.): ICISP 2018: Image and Signal Processing, Lecture Notes in Computer Science, pp. 407-415, Springer, 2018. (Type: Inproceedings | Abstract | Links) @inproceedings{ParCalDaf2018, title = {Digital Cultural Heritage Imaging via Osmosis Filtering}, author = {Simone Parisotto and Luca Calatroni and Claudia Daffara}, editor = {Alamin Mansouri and Abderrahim El Moataz and Fathallah Nouboud and Driss Mammass}, doi = {10.1007/978-3-319-94211-7_44}, year = {2018}, date = {2018-06-30}, booktitle = {ICISP 2018: Image and Signal Processing, Lecture Notes in Computer Science}, volume = {10884}, pages = {407-415}, publisher = {Springer}, abstract = {In Cultural Heritage (CH) imaging, data acquired within different spectral regions are often used to inspect surface and sub-surface features. Due to the experimental setup, these images may suffer from intensity inhomogeneities, which may prevent conservators from distinguishing the physical properties of the object under restoration. Furthermore, in multi-modal imaging, the transfer of information between one modality to another is often used to integrate image contents. In this paper, we apply the image osmosis model proposed in [4,10,12] to solve correct these problems arising when diagnostic CH imaging techniques based on reflectance, emission and fluorescence mode in the optical and thermal range are used. For an efficient computation, we use stable operator splitting techniques to solve the discretised model. We test our methods on real artwork datasets: the thermal measurements of the mural painting “Monocromo” by Leonardo Da Vinci, the UV-VIS-IR imaging of an ancient Russian icon and the Archimedes Palimpsest dataset.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Close In Cultural Heritage (CH) imaging, data acquired within different spectral regions are often used to inspect surface and sub-surface features. Due to the experimental setup, these images may suffer from intensity inhomogeneities, which may prevent conservators from distinguishing the physical properties of the object under restoration. Furthermore, in multi-modal imaging, the transfer of information between one modality to another is often used to integrate image contents. In this paper, we apply the image osmosis model proposed in [4,10,12] to solve correct these problems arising when diagnostic CH imaging techniques based on reflectance, emission and fluorescence mode in the optical and thermal range are used. For an efficient computation, we use stable operator splitting techniques to solve the discretised model. We test our methods on real artwork datasets: the thermal measurements of the mural painting “Monocromo” by Leonardo Da Vinci, the UV-VIS-IR imaging of an ancient Russian icon and the Archimedes Palimpsest dataset. Close doi:10.1007/978-3-319-94211-7_44 Close

Parisotto, Simone; Calatroni, Luca; Daffara, Claudia: Mathematical osmosis imaging for multi-modal and multi-spectral applications in Cultural Heritage conservation. In: Image Processing for Art Investigation, Ghent, 2018. (Type: Inproceedings | Abstract | Links) @inproceedings{ParCalDaf2018b, title = {Mathematical osmosis imaging for multi-modal and multi-spectral applications in Cultural Heritage conservation}, author = {Simone Parisotto and Luca Calatroni and Claudia Daffara}, url = {https://www.ip4ai.ugent.be/IP4AI2018_proceedings.pdf}, year = {2018}, date = {2018-06-22}, booktitle = {Image Processing for Art Investigation}, address = {Ghent}, abstract = {In this work we present a dual-mode mid-infrared workflow [6], for detecting sub-superficial mural damages in frescoes artworks. Due to the large nature of frescoes, multiple thermal images are recorded. Thus, the experimental setup may introduce measurements errors, seen as inter-frame changes in the image contrast, after mosaicking. An approach to lowering errors is to post-process the mosaic [10] via osmosis partial differential equation (PDE) [12, 13], which preserves details, mass and balance the lights: efficient numerical study for osmosis on large images is proposed [2, 11], based on operator splitting [8]. Our range of Cultural Heritage applications include the detection of sub-superficial voids in Monocromo (L. Da Vinci, Castello Sforzesco, Milan) [5], the light-balance for multi-spectral imaging and the data integration on the Archimedes Palimpsest [10].}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Close In this work we present a dual-mode mid-infrared workflow [6], for detecting sub-superficial mural damages in frescoes artworks. Due to the large nature of frescoes, multiple thermal images are recorded. Thus, the experimental setup may introduce measurements errors, seen as inter-frame changes in the image contrast, after mosaicking. An approach to lowering errors is to post-process the mosaic [10] via osmosis partial differential equation (PDE) [12, 13], which preserves details, mass and balance the lights: efficient numerical study for osmosis on large images is proposed [2, 11], based on operator splitting [8]. Our range of Cultural Heritage applications include the detection of sub-superficial voids in Monocromo (L. Da Vinci, Castello Sforzesco, Milan) [5], the light-balance for multi-spectral imaging and the data integration on the Archimedes Palimpsest [10]. Close https://www.ip4ai.ugent.be/IP4AI2018_proceedings.pdf Close

Daffara, Claudia; Parisotto, Simone; Mariotti, Paola Ilaria: Mid-infrared thermal imaging for an effective mapping of surface materials and sub-surface detachments in mural paintings: integration of thermography and thermal quasi-reflectography. In: Optics for Arts, Architecture, and Archaeology, International Society for Optics and Photonics 2015. (Type: Inproceedings | Abstract | Links) @inproceedings{DafParMar2015, title = {Mid-infrared thermal imaging for an effective mapping of surface materials and sub-surface detachments in mural paintings: integration of thermography and thermal quasi-reflectography}, author = {Claudia Daffara and Simone Parisotto and Paola Ilaria Mariotti}, doi = {10.1117/12.2184836}, year = {2015}, date = {2015-06-30}, booktitle = {Optics for Arts, Architecture, and Archaeology}, volume = {V}, number = {9527}, organization = {International Society for Optics and Photonics}, abstract = {Cultural Heritage is discovering how precious is thermal analysis as a tool to improve the restoration, thanks to its ability to inspect hidden details. In this work a novel dual mode imaging approach, based on the integration of thermography and thermal quasi-reflectography (TQR) in the mid-IR is demonstrated for an effective mapping of surface materials and of sub-surface detachments in mural painting. The tool was validated through a unique application: the “Monocromo” by Leonardo da Vinci in Italy. The dual mode acquisition provided two spatially aligned dataset: the TQR image and the thermal sequence. Main steps of the workflow included: 1) TQR analysis to map surface features and 2) to estimate the emissivity; 3) projection of the TQR frame on reference orthophoto and TQR mosaicking; 4) thermography analysis to map detachments; 5) use TQR to solve spatial referencing and mosaicking for the thermal-processed frames. Referencing of thermal images in the visible is a difficult aspect of the thermography technique that the dual mode approach allows to solve in effective way. We finally obtained the TQR and the thermal maps spatially referenced to the mural painting, thus providing the restorer a valuable tool for the restoration of the detachments.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Close Cultural Heritage is discovering how precious is thermal analysis as a tool to improve the restoration, thanks to its ability to inspect hidden details. In this work a novel dual mode imaging approach, based on the integration of thermography and thermal quasi-reflectography (TQR) in the mid-IR is demonstrated for an effective mapping of surface materials and of sub-surface detachments in mural painting. The tool was validated through a unique application: the “Monocromo” by Leonardo da Vinci in Italy. The dual mode acquisition provided two spatially aligned dataset: the TQR image and the thermal sequence. Main steps of the workflow included: 1) TQR analysis to map surface features and 2) to estimate the emissivity; 3) projection of the TQR frame on reference orthophoto and TQR mosaicking; 4) thermography analysis to map detachments; 5) use TQR to solve spatial referencing and mosaicking for the thermal-processed frames. Referencing of thermal images in the visible is a difficult aspect of the thermography technique that the dual mode approach allows to solve in effective way. We finally obtained the TQR and the thermal maps spatially referenced to the mural painting, thus providing the restorer a valuable tool for the restoration of the detachments. Close doi:10.1117/12.2184836 Close

PhD Theses

Parisotto, Simone: Anisotropic variational models and PDEs for inverse imaging problems. University of Cambridge, 2019. (Type: PhD Thesis | Abstract | Links) @phdthesis{Parisotto2019, title = {Anisotropic variational models and PDEs for inverse imaging problems}, author = {Simone Parisotto}, url = {http://simoneparisotto.com/math/thesis/phd/slides_phdthesis.pdf}, doi = {https://www.repository.cam.ac.uk/handle/1810/291064}, year = {2019}, date = {2019-10-26}, school = {University of Cambridge}, abstract = {Supervisor: Prof Carola-Bibiane Schönlieb (University of Cambridge) Co-supervisor: Prof Simon Masnou (Université Lyon 1) In this thesis we study new anisotropic variational regularisers and partial differential equations (PDEs) for solving inverse imaging problems that arise in a variety of real-world applications. Firstly, we introduce a new anisotropic higher-order total directional variation regulariser. We describe both the theoretical and the numerical details for its use within a variational formulation for solving inverse problems and give examples for the reconstruction of noisy images and videos, image zooming and the interpolation of scattered surface data. Secondly, we focus on a non-symmetric drift-diffusion equation, called osmosis. We propose an efficient numerical implementation of the osmosis equation, based on alternate directions and operator splitting techniques. We study their scale-space properties and show their efficiency in processing large images. Moreover, we generalise the osmosis equation to accommodate suitable directional information: this modification turns out to be useful to correct for the well-known blurring artefacts the original osmosis model introduces when applied to shadow removal in images. Last but not least, we explore applications of variational models and PDEs to cultural heritage conservation. We develop a new non-invasive technique that uses multi-modal imaging for detecting sub-superficial defects in fresco walls at sub-millimetre precision. We correct light-inhomogeneities in these imaging measurements that are due to measurement errors via osmosis filtering, in particular making use of the efficient computational schemes that we introduced before for dealing with the large-scale nature of these measurements. Finally, we propose a semi-supervised workflow for the detection and inpainting of defects in damaged illuminated manuscripts. Keywords: Total directional variation, anisotropic diffusion, osmosis filter, cultural heritage conservation, primal-dual hybrid gradient, dimensional splitting, inverse problems, image denoising, video denoising, image zooming, surface interpolation, digital elevation maps, shadow removal, thermal quasi-reflectography, non-destructive imaging, dual-mode mid-infrared imaging, inpainting, illuminated manuscripts.}, keywords = {}, pubstate = {published}, tppubtype = {phdthesis} } Close Supervisor: Prof Carola-Bibiane Schönlieb (University of Cambridge) Co-supervisor: Prof Simon Masnou (Université Lyon 1) In this thesis we study new anisotropic variational regularisers and partial differential equations (PDEs) for solving inverse imaging problems that arise in a variety of real-world applications. Firstly, we introduce a new anisotropic higher-order total directional variation regulariser. We describe both the theoretical and the numerical details for its use within a variational formulation for solving inverse problems and give examples for the reconstruction of noisy images and videos, image zooming and the interpolation of scattered surface data. Secondly, we focus on a non-symmetric drift-diffusion equation, called osmosis. We propose an efficient numerical implementation of the osmosis equation, based on alternate directions and operator splitting techniques. We study their scale-space properties and show their efficiency in processing large images. Moreover, we generalise the osmosis equation to accommodate suitable directional information: this modification turns out to be useful to correct for the well-known blurring artefacts the original osmosis model introduces when applied to shadow removal in images. Last but not least, we explore applications of variational models and PDEs to cultural heritage conservation. We develop a new non-invasive technique that uses multi-modal imaging for detecting sub-superficial defects in fresco walls at sub-millimetre precision. We correct light-inhomogeneities in these imaging measurements that are due to measurement errors via osmosis filtering, in particular making use of the efficient computational schemes that we introduced before for dealing with the large-scale nature of these measurements. Finally, we propose a semi-supervised workflow for the detection and inpainting of defects in damaged illuminated manuscripts. Keywords: Total directional variation, anisotropic diffusion, osmosis filter, cultural heritage conservation, primal-dual hybrid gradient, dimensional splitting, inverse problems, image denoising, video denoising, image zooming, surface interpolation, digital elevation maps, shadow removal, thermal quasi-reflectography, non-destructive imaging, dual-mode mid-infrared imaging, inpainting, illuminated manuscripts. Close http://simoneparisotto.com/math/thesis/phd/slides_phdthesis.pdf doi:https://www.repository.cam.ac.uk/handle/1810/291064 Close