publications

2022

1. NeurIPS Workshop

   A Reproducible and Realistic Evaluation of Partial Domain Adaptation Methods

   Tiago Salvador, Kilian Fatras, Ioannis Mitliagkas, and Adam Oberman

   In NeurIPS Distribution Shifts Workshop, 2022

   abstract arXiv html pdf code

   Unsupervised Domain Adaptation (UDA) aims at classifying unlabeled target images leveraging source labeled ones. In this work, we consider the Partial Domain Adaptation (PDA) variant, where we have extra source classes not present in the target domain. Most successful algorithms use model selection strategies that rely on target labels to find the best hyper-parameters and/or models along training. However, these strategies violate the main assumption in PDA: only unlabeled target domain samples are available. Moreover, there are also inconsistencies in the experimental settings - architecture, hyper-parameter tuning, number of runs - yielding unfair comparisons. The main goal of this work is to provide a realistic evaluation of PDA methods with the different model selection strategies under a consistent evaluation protocol. We evaluate 7 representative PDA algorithms on 2 different real-world datasets using 7 different model selection strategies. Our two main findings are: (i) without target labels for model selection, the accuracy of the methods decreases up to 30 percentage points; (ii) only one method and model selection pair performs well on both datasets. Experiments were performed with our PyTorch framework, BenchmarkPDA, which we open source.

2. ICML Workshop

   ImageNet-Cartoon and ImageNet-Drawing: two domain shift datasets for ImageNet

   Tiago Salvador, and Adam M. Oberman

   In ICML Shift Happens Workshop, 2022

   abstract html pdf code Slides

   Benchmarking the robustness to distribution shifts traditionally relies on dataset collection which is typically laborious and expensive, in particular for datasets with a large number of classes like ImageNet. An exception to this procedure is ImageNet-C (Hendrycks & Dietterich, 2019), a dataset created by applying common real-world corruptions at different levels of intensity to the (clean) ImageNet images. Inspired by this work, we introduce ImageNet-Cartoon and ImageNet-Drawing, two datasets constructed by converting ImageNet images into cartoons and colored pencil drawings, using a GAN framework (Wang & Yu, 2020) and simple image processing (Lu et al., 2012), respectively.

3. ICLR

   FairCal: Fairness Calibration for Face Verification

   Tiago Salvador, Stephanie Cairns, Vikram Voleti, Noah Marshall, and Adam Oberman

   In International Conference on Learning Representations, 2022

   abstract arXiv html pdf code Poster Slides

   Despite being widely used, face recognition models suffer from bias: the probability of a false positive (incorrect face match) strongly depends on sensitive attributes such as the ethnicity of the face. As a result, these models can disproportionately and negatively impact minority groups, particularly when used by law enforcement. The majority of bias reduction methods have several drawbacks: they use an end-to-end retraining approach, may not be feasible due to privacy issues, and often reduce accuracy. An alternative approach is post-processing methods that build fairer decision classifiers using the features of pre-trained models, thus avoiding the cost of retraining. However, they still have drawbacks: they reduce accuracy (AGENDA, PASS, FTC), or require retuning for different false positive rates (FSN). In this work, we introduce the Fairness Calibration (FairCal) method, a post-training approach that simultaneously: (i) increases model accuracy (improving the state-of-the-art), (ii) produces fairly-calibrated probabilities, (iii) significantly reduces the gap in the false positive rates, (iv) does not require knowledge of the sensitive attribute, and (v) does not require retraining, training an additional model, or retuning. We apply it to the task of Face Verification, and obtain state-of-the-art results with all the above advantages.

2021

1. arXiv

   Frustratingly Easy Uncertainty Estimation for Distribution Shift

   Tiago Salvador, Vikram Voleti, Alexander Iannantuono, and Adam Oberman

   arXiv preprint arXiv: Arxiv-2106.03762, 2021

   abstract arXiv html pdf

   Distribution shift is an important concern in deep image classification, produced either by corruption of the source images, or a complete change, with the solution involving domain adaptation. While the primary goal is to improve accuracy under distribution shift, an important secondary goal is uncertainty estimation: evaluating the probability that the prediction of a model is correct. While improving accuracy is hard, uncertainty estimation turns out to be frustratingly easy. Prior works have appended uncertainty estimation into the model and training paradigm in various ways. Instead, we show that we can estimate uncertainty by simply exposing the original model to corrupted images, and performing simple statistical calibration on the image outputs. Our frustratingly easy methods demonstrate superior performance on a wide range of distribution shifts as well as on unsupervised domain adaptation tasks, measured through extensive experimentation.

2020

1. ICML Workshop

   Calibrated Top-1 Uncertainty estimates for classification by score based models

   Adam Oberman, Chris Finlay, Alexander Iannantuono, and Tiago Salvador

   In ICML Workshop on Uncertainty & Robustness in Deep Learning (Spotlight Talk), 2020

   abstract arXiv pdf code

   While the accuracy of modern deep learning models has significantly improved in recent years, the ability of these models to generate uncertainty estimates has not progressed to the same degree. Uncertainty methods are designed to provide an estimate of class probabilities when predicting class assignment. While there are a number of proposed methods for estimating uncertainty, they all suffer from a lack of calibration: predicted probabilities can be off from empirical ones by a few percent or more. By restricting the scope of our predictions to only the probability of Top1 error, we can decrease the calibration error of existing methods to less than one percent. As a result, the scores of the methods also improve significantly over benchmarks.

2. JSC

   A Partial Differential Equation Obstacle Problem for the Level Set Approach to Visibility

   Adam Oberman, and Tiago Salvador

   Journal of Scientific Computing, 2020

   abstract arXiv html pdf code

   In this article we consider the problem of finding the visibility set from a given point when the obstacles are represented as the level set of a given function. Although the visibility set can be computed efficiently by ray tracing, there are advantages to using a level set representation for the obstacles, and to characterizing the solution using a partial differential equation (PDE). A nonlocal PDE formulation was proposed in Tsai et al. (2004): in this article we propose a simpler PDE formulation, involving a nonlinear obstacle problem. We present a simple numerical scheme and show its convergence using the framework of Barles and Souganidis. Numerical examples in both two and three dimensions are presented.

2019

1. JSC

   A Simplified Threshold Dynamics Algorithm for Isotropic Surface Energies

   Tiago Salvador, and Selim Esedoglu

   Journal of Scientific Computing, 2019

   abstract arXiv html pdf

   We present a simplified version of the threshold dynamics algorithm given in Esedoḡlu and Otto (2015). The new version still allows specifying N-choose-2 possibly distinct surface tensions and N-choose-2 possibly distinct mobilities for a network with N phases, but achieves this level of generality without the use of retardation functions. Instead, it employs linear combinations of Gaussians in the convolution step of the algorithm. Convolutions with only two distinct Gaussians is enough for the entire network, maintaining the efficiency of the original thresholding scheme. We discuss stability and convergence of the new algorithm, including some counterexamples in which convergence fails. The apparently convergent cases include unequal surface tensions given by the Read and Shockley model and its three dimensional extensions, along with equal mobilities, that are a very common choice in computational materials science.

2018

1. JSC

   Higher-Order Adaptive Finite Difference Methods for Fully Nonlinear Elliptic Equations

   Brittany Froese Hamfeldt, and Tiago Salvador

   Journal of Scientific Computing, 2018

   abstract arXiv html pdf code

   We introduce generalised finite difference methods for solving fully nonlinear elliptic partial differential equations. Methods are based on piecewise Cartesian meshes augmented by additional points along the boundary. This allows for adaptive meshes and complicated geometries, while still ensuring consistency, monotonicity, and convergence. We describe an algorithm for efficiently computing the non-traditional finite difference stencils. We also present a strategy for computing formally higher-order convergent methods. Computational examples demonstrate the efficiency, accuracy, and flexibility of the methods.

2. IMAJNA

   Numerical methods for motion of level sets by affine curvature

   Adam M. Oberman, and Tiago Salvador

   IMA Journal of Numerical Analysis, 2018

   abstract arXiv html pdf

   We study numerical methods for the nonlinear partial differential equation that governs the motion of level sets by affine curvature. We show that standard finite difference schemes are nonlinearly unstable. We build convergent finite difference schemes using the theory of viscosity solutions. We demonstrate that our approximate solutions capture the affine invariance and morphological properties of the evolution. Numerical experiments demonstrate the accuracy and stability of the discretization.

2017

1. IMAJNA

   Numerical methods for the 2-Hessian elliptic partial differential equation

   Brittany D. Froese, Adam M. Oberman, and Tiago Salvador

   IMA Journal of Numerical Analysis, 2017

   abstract arXiv html pdf

   The elliptic 2-Hessian equation is a fully nonlinear partial differential equation (PDE) that is related to intrinsic curvature for three-dimensional manifolds. We ntroduce two numerical methods for this PDE: the first is provably convergent to the viscosity solution and the second is more accurate and convergent in practice but lacks a proof. The PDE is elliptic on a restricted set of functions: a convexity-type constraint is needed for the ellipticity of the PDE operator. Solutions with both iscretizations are obtained using Newton’s method. Computational results are presented on a number of exact solutions which range in regularity from mooth to nondifferentiable and in shape from convex to nonconvex.

2015

1. JCP

   Filtered schemes for Hamilton–Jacobi equations: A simple construction of convergent accurate difference schemes

   Adam M. Oberman, and Tiago Salvador

   Journal of Computational Physics, 2015

   abstract arXiv html pdf code

   We build a simple and general class of finite difference schemes for first order Hamilton–Jacobi (HJ) Partial Differential Equations. These filtered schemes are convergent to the unique viscosity solution of the equation. The schemes are accurate: we implement second, third and fourth order accurate schemes in one dimension and second order accurate schemes in two dimensions, indicating how to build higher order ones. They are also explicit, which means they can be solved using the fast sweeping method. The accuracy of the method is validated with computational results for the eikonal equation and other HJ equations in one and two dimensions, using filtered schemes made from standard centered differences, higher order upwinding and ENO interpolation.