Publications:
Peer reviewed papers:
Matrix factorization in the
presence of missing data is at the core of many computer vision problems
such as structure from motion (SfM), non-rigid SfM and photometric
stereo. We formulate the problem of matrix factorization with missing
data as a low-rank semidefinite program (LRSDP) with the advantage that:
1) an efficient quasi-Newton implementation of the LRSDP enables us to
solve large-scale factorization problems, and 2) additional constraints
such as orthonormality, required in orthographic SfM, can be directly
incorporated in the new formulation. Our empirical evaluations suggest
that, under the conditions of matrix completion theory, the proposed
algorithm finds the optimal solution, and also requires fewer
observations compared to the current state-of-the-art algorithms. We
further demonstrate the effectiveness of the proposed algorithm in
solving the affine SfM problem, non-rigid SfM and photometric stereo
problems.
The earth mover’s distance
(EMD) is an important perceptually meaningful metric for
comparing histograms, but it suffers from high (O(N3
log N)) computational complexity. We present a novel linear
time algorithm for approximating the EMD for low dimensional
histograms using the sum of absolute values of the weighted
wavelet coefficients of the difference histogram. EMD
computation is a special case of the Kantorovich-Rubinstein
transshipment problem, and we exploit the Hölder continuity
constraint in its dual form to convert it into a simple
optimization problem with an explicit solution in the wavelet
domain. We prove that the resulting wavelet EMD metric is
equivalent to EMD, i.e. the ratio of the two is bounded. We
also provide estimates for the bounds.
The weighted wavelet transform can be computed in time
linear in the number of histogram bins, while the comparison
is about as fast as for normal Euclidean distance or
Χ2 statistic. We experimentally show that
wavelet EMD is a good approximation to EMD, has similar
performance, but requires much less computation.
This work was also presented at the
Optimal transportation: Theory and applications summer
school in Grenoble, France in June 2009 by Prof. David
Jacobs and
Monge-Kantorovich Optimal Transport - Theory and
Applications workshop in Santa Fe, NM, USA in October
2009 by me.
Peter N.
Belhumeur, Daozheng Chen, Steven Feiner, David Jacobs, W.
John Kress, Haibin Ling, Ida Lopez, Ravi Ramamoorthi, Sameer
Sheorey, Sean White and Ling Zhang, Searching the
World's Herbaria: A System of Visual Identification of
Plant Species, European Conference on Computer
Vision, Marseille, France, 2008. Poster
(ppt) Demo video
(YouTube) Abstract
©2008 IEEE
We describe a working
computer vision system that aids in the identification of plant
species. A user photographs an isolated leaf on a blank back
ground, and the system extracts the leaf shape and matches it to
the shape of leaves of known species. In a few seconds, the
system displays the top matching species, along with textual
descriptions and additional images. This system is currently in
use by botanists at the Smithsonian Institution National Museum
of Natural History. The primary contributions of this paper
are: a description of a working computer vision system and its
user interface for an important new application area; the
introduction of three new datasets containing thousands of
single leaf images, each labeled by species and verified by
botanists at the United States National Herbarium; recognition
results on two of the three leaf datasets; and descriptions
throughout of practical lessons learned in constructing this
system.
Gaurav
Agarwal, Peter Belhumeur, Steven Feiner, David Jacobs, W.
John Kress,Ravi Ramamoorthi, Norman A. Bourg, Nandan Dixit,
Haibin Ling, Dhruv Mahajan, Rusty Russell, Sameer Shirdhonkar,
Kalyan Sunkavalli and Sean White
First Steps Toward an Electronic Field Guide for Plants
, Taxon, 55(3): 597-610, August, 2006.
Abstract
We describe an ongoing
project to digitize information about plant specimens and make
it available to botanists in the field. This first requires
digital images and models, and then effective retrieval and
mobile computing mechanisms for accessing this information.
We have almost completed a digital archive of the collection
of type specimens at the Smithsonian Institution Department of
Botany. Using these and additional images, we have also
constructed prototype electronic field guides for the flora of
Plummers Island. Our guides use a novel computer vision
algorithm to compute leaf similarity. This algorithm is
integrated into image browsers that assist a user in
navigating a large collection of images to identify the
species of a new specimen. For example, our systems allow a
user to photograph a leaf and use this image to retrieve a set
of leaves with similar shapes. We measured the effectiveness
of one of these systems with recognition experiments on a
large dataset of images, and with user studies of the complete
retrieval system. In addition, we describe future directions
for acquiring models of more complex, 3D specimens, and for
using new methods in wearable computing to interact with data
in the 3D environment in which it is acquired.
Project website: Electronic Field
Guide
Recognition of specular objects
is particularly difficult because their appearance is much
more sensitive to lighting changes than that of Lambertian
objects. We consider an approach in
which we use a 3D model to deduce the lighting that best
matches the model to the image. In this case, an important
constraint is that incident lighting should be non-negative
everywhere. In this paper, we propose a new method to
enforce this constraint and explore its usefulness in
specular object recognition, using the spherical harmonic
representation of lighting. The method follows from a novel
extension of Szego’s eigenvalue distribution theorem
to spherical harmonics, and uses semidefinite programming to
perform a constrained optimization. The new method is
faster as well as more accurate than previous methods.
Experiments on both synthetic and real data indicate that
the constraint can improve recognition of specular objects
by better separating the correct and incorrect models.
PhD Thesis:
Recognition and matching in the presence of
deformation and lighting change.
(pdf)
Department of Computer Science, University of Maryland, College
Park.
Advisor: Prof. David Jacobs
Technical Reports:
IEEE copyright:
Personal use of this material is permitted. However, permission to
reprint/republish this material for advertising or promotional
purposes or for creating new collective works for resale or
redistribution to servers or lists, or to reuse any copyrighted
component of this work in other works must be obtained from the
IEEE.