My research interests are in computer vision, image analysis, machine learning, and multimodal perception. I approach these problems with methods from Bayesian statistics, signal processing, and applied mathematics. Specific projects I have worked on include:

Perturb-and-MAP Random Fields: Reducing Random Sampling to Optimization


We have been developing a new Perturb-and-MAP framework for one-shot random sampling in Gaussian or discete-label Markov random fields (MRF). With Perturb-and-MAP random fields we turn powerful deterministic energy minimization methods into efficient random sampling algorithms. By avoiding costly MCMC, we can generate in a fraction of a second independent random samples from million-node networks. Applications include model parameter estimation and solution uncertainty quantification in computer vision applications.
[Read more…]

Bayesian Inference in Large-Scale Sparse Models: Efficient Monte-Carlo and Variational Approaches

Sparse Bayes 

We study linear models under heavy-tailed priors from a probabilistic viewpoint. Instead of computing a single sparse most probable (MAP) solution as in standard deterministic techniques, the focus in the Bayesian compressed sensing framework shifts towards capturing the full posterior distribution on the latent variables. This allows quantifying the estimation uncertainty and learning model parameters using maximum likelihood. The exact posterior distribution under the sparse linear model is intractable and we propose both Monte-Carlo and variational Bayesian methods to approximate it. Efficient Gaussian sampling by local perturbations turns out to be a key computational module that allows both of these classes of algorithms to handle large-scale datasets with essentially the same memory and time complexity requirements as conventional MAP estimation techniques. We experimentally demonstrate these ideas in Bayesian total variation (TV) signal estimation, visual receptive field learning, and blind image deconvolution, among other applications.
[Read more…]

Digital Restoration of Missing Parts in the Pre-historic Wall Paintings of Thera


We have been working on PDE and wavelet-based techniques for the digital restoration of missing parts in paintings. This is part of an ongoing project on the virtual restoration of the 3,600 years old wall paintings excavated in the pre-historic Aegean settlement in Akrotiri, Thera, Greece.
[Read more…] [Project page…]

Audio-Visual Speech Recognition


Audiovisual speech recognition refers to the problem of recognizing speech by lipreading. We have developed highly adaptive multimodal fusion rules based on uncertainty compensation which are compatible with synchronous and asynchronous multimodal interaction architectures. Further, our work on AAM-based face representations leads to highly informative visual speech features which can be extracted in real-time.
[Read more…] [Software…]

Multigrid Geometric Active Contours


We investigate multigrid techniques for the solution of the time-dependent PDEs of geometric active contour models in Computer Vision. The method allows interactive solution of models whose numerical implementation with conventional techniques has been prohibitively slow.
[Read more…] [Software…]

Audio-Visual Speech Inversion


We focus on recovering aspects of vocal tract's geometry and dynamics from speech, a problem referred to as speech inversion. In our inversion scheme ambiguities inherent to audio-only inversion are resolved by also exploiting visual information from the speaker's face.
[Read more…] [Software…]

Automatic Scale Selection in Nonlinear Scale-Spaces


We consider optimal scale selection for fully automatic image denoising in nonlinear diffusion or morphological scale-spaces. The problem is studied from a statistical model selection viewpoint and we employ cross-validation statistical techniques to address it in a principled way.