September 2019
Volume 19, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2019
A new method to compute classification error
Author Affiliations & Notes
  • Abhranil Das
    The University of Texas at Austin
  • R Calen Walshe
    The University of Texas at Austin
  • Wilson S Geisler
    The University of Texas at Austin
Journal of Vision September 2019, Vol.19, 87. doi:https://doi.org/10.1167/19.10.87
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Abhranil Das, R Calen Walshe, Wilson S Geisler; A new method to compute classification error. Journal of Vision 2019;19(10):87. doi: https://doi.org/10.1167/19.10.87.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

The sensitivity index d’ is used ubiquitously to express the error rate in classification and detection tasks. In the general case with multiple cues that can have arbitrary variances and pairwise correlations, and unequal prior class probabilities, the decision boundary separating the classes is a complex surface. This makes it challenging to compute the error rate integral, and there currently exists no single algorithm that works reliably in all cases. Standard integration procedures may require grids that are inefficiently large and fine, converge slowly, yet may miss relevant parts of the space unless tailored case-by-case. These obstacles impede the testing and optimization of models. We present a new method to compute error rates simply, reliably and fast, for all cases up to 3 dimensions. This is founded on a mathematical transformation of the feature space that converts all cases into a single canonical form that is simple and spherically symmetric. In polar coordinates that can exploit this symmetry, the radius integral has a known analytical form, and the angle integral is computed over the same bounded grid of angles, for all cases. Our open-source MATLAB function based on this method computes error rates and d’, returns the optimal decision boundary, accommodates custom suboptimal boundaries, and produces plots to visualize the distributions and the decision boundary. We have applied this method to compute error rates for an ideal observer analysis of a detection task in natural scenes. This was a particularly challenging case for a traditional integration method because: i.) there are three correlated cues used for detection, ii.) error rates are extremely small (below machine precision) in some conditions, and iii.) the decision boundary varies unpredictably across conditions. We find that this method recovers the true error rate quickly and accurately, and to an arbitrary level of precision.

Acknowledgement: NIH grant EY11747 
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×