Attacks on Real-World Uses of Machine Learning

Machine-learning (ML) classifiers have numerous real-world applications, some of which are security-critical.  Motivated by such applications, much research has been devoted to better understanding adversarial examples, which are specially crafted inputs to a machine-learning model that are perceptually similar to benign inputs but are classified differently (i.e., misclassified). Both algorithms that create adversarial examples and strategies for defending against adversarial examples typically use L_p-norms (usually, p = 0, 2, or ∞) to measure the perceptual similarity between an adversarial input and its benign original.  That is, attacks can leverage any perturbation of sufficiently small L_p-norm to modify benign inputs so as to change their classifications.

Unfortunately, relying on L_p-norm bounds to enforce that adversarial examples are perceptually similar to the benign inputs from which they are derived is problematic, for several reasons.  First, we have shown that a small L_p-norm is neither sufficient nor necessary for a human to classify an adversarial example the same as the benign input from which it was derived.  That is, small L_p-norms are poor surrogates for perceptual similarity.  Second, in many real-world scenarios, the constraints that dictate how an adversary can manipulate a benign input cannot be captured simply by bounded L_p-norms.  For example, in face-recognition applications, an adversary might be able to modify his own appearance only through wearing accessories (e.g., eyeglasses) that are inconspicuous to an onlooker.  As another example, if the classifier is used to determine whether an executable is malware, the adversary might be able to modify the malware arbitrarily as long as its functionality is not changed.