From point to surface: generalizable manifold adversarial attacks, from individual adversarial to manifold adversarial attacks

Is the facial recognition system that claims to be 99% accurate really unbreakable? In fact, the face recognition system can be easily broken by making some changes in face photos that do not affect visual judgment. For example, the girl next door and the male celebrity can be judged as the same person. This is an adversarial attack. The goal of adversarial attacks is to find adversarial samples that are natural and can confuse the neural network. In essence, finding adversarial samples is to find the vulnerabilities of the neural network.

Recently, a research team from Dongfang University of Technology proposed a paradigm of generalized manifold adversarial attack (Generalized Manifold Adversarial Attack, GMAA),

Promote the traditional "point" attack mode to the "surface" attack mode, which greatly improves the generalization ability of the adversarial attack model and develops a new idea for the work of adversarial attacks.

This research improves previous work from two aspects: target domain and adversarial domain. On the target domain, this study finds more powerful adversarial examples with high generalization by attacking the set of states of the target identity. For the adversarial domain, previous work was looking for discrete adversarial samples, that is, finding several "loopholes" (points) of the system, while this research is looking for continuous adversarial manifolds, that is, finding the fragile integral parts of the neural network. Piece "area" (face). In addition, this research introduces domain knowledge of expression editing and proposes a new paradigm based on expression state space instantiation. By continuously sampling the generated adversarial manifold, we can obtain highly generalizable adversarial samples with continuous expression changes. Compared with methods such as makeup, lighting, and adding perturbations,

the expression state space is more universal and natural, and is not affected by Effects of gender and lighting. Research paper has been accepted for CVPR 2023.

##Paper link: https://arxiv.org/abs/2301.06083

Code link https://github.com/tokaka22/GMAA

Method introduction

In the target domain part, previous work They all design adversarial samples for a specific photo of target identity A. However, as shown in Figure 2, when the adversarial sample generated by this attack method is used to attack another photo of A, the attack effect will be significantly reduced. In the face of such attacks, regularly changing the photos in the facial recognition database is naturally an effective defense measure. However, the GMAA proposed in this study not only trains on a single sample of the target identity, but also looks for adversarial samples that can attack the set of target identity states.

Such highly generalized adversarial samples face updated people The face recognition library has better attack performance. These more powerful adversarial samples also correspond to the weaker areas of the neural network and are worthy of in-depth exploration. In the adversarial domain part, previous work has been to find one or several discrete adversarial samples, which is equivalent to finding one or several vulnerable neural networks in high-dimensional space. "Point", and this study believes that the neural network may be fragile on the entire "face", and the adversarial samples on this "face" should be "caught in one sweep". Therefore, this research is devoted to finding adversarial manifolds in high-dimensional space.

In summary, GMAA is a new attack paradigm that uses

adversarial manifolds to attack the state set of the target identity. The core idea of ​​the article is shown in Figure 1.

Specifically, this research introduces the domain knowledge of expression editing, Facial Action Coding System (FACS), and uses the expression state space to instantiate the proposed new attack paradigm. FACS is a facial expression coding system that divides the face into different muscle units. Each element in the AU vector corresponds to a muscle unit. The size of the vector element value represents the degree of muscle activity of the corresponding unit, thereby encoding Expression status. For example, in the figure below, the first element AU1 in the AU vector represents the degree of lifting the inner eyebrow.

From "The Anatomy of Facial Expressions"

For the target domain, this research attacks target sets containing multiple expression states, thereby achieving better attack performance on unknown target photos; for the adversarial domain, this research establishes an adversarial manifold corresponding to the AU space. Adversarial samples can be sampled on the adversarial manifold by changing the AU value, and by continuously changing the AU value, adversarial samples with continuously changing expressions can be generated. ​

#It is worth noting that this research uses expression state space to instantiate the GMAA attack paradigm. This is because expression is the most common state in human facial activities, and the expression state space is relatively stable and will not be affected by race or gender (light can change skin color, and makeup can affect gender) . In fact, as long as other suitable state spaces can be found, this attack paradigm can be generalized and applied to other adversarial attack tasks in nature.

Model results

The animation below shows the visual results of the study. Each frame of the animation is an adversarial sample sampled on the adversarial manifold. By continuously sampling, a series of adversarial samples (left side) with continuously changing expressions can be obtained. The red value indicates the adversarial sample and target sample of the current frame. (Right) Similarity under Face facial recognition system.

In Table 1, the study lists the black box attack success rates of 4 face recognition models on two data sets. Among them, MAA is a shortened version of GMAA, and MAA is only In the adversarial domain, the point attack model is extended to the manifold attack, and in the target domain, a single target photo is still attacked. The state set of attack targets is a common experimental setting. The article adds this setting to three methods including MAA in Table 2 (the bold part in the table is the result of adding this setting, in Table 2 (A “G” is added to the name of the method to distinguish it), which proves that the expansion of the target domain can improve the generalization of adversarial samples.

Figure 4 shows the results of attacking the APIs of two commercial facial recognition systems.

#The study also explored the impact of different expressions on attack performance, and the impact of the number of samples contained in the state set on attack generalization performance .

Figure 6 shows the comparison of visual results of different methods. MAA is better at fighting 20 adversarial samples were sampled on the manifold, and you can see that the visualization effect is more natural.

Of course, not all data sets have pictures of different statuses of an identity. How to expand the target domain in this case? This research also provides a feasible solution, which is to use AU vectors and expression editing models to generate target state sets. The article also presents the results of attacking the synthesized target state set, and it can be found that the generalization performance has also been improved.

Principles and Methods

The backbone of the model includes a WGAN-GP-based generation module, expression supervision module, and transferability enhancement Module, generalized attack module. Among them, the generalized attack module implements the function of attacking target state sets, and the transferability enhancement module comes from previous work. For fair comparison, this module is added to all baselines. The expression supervision module is composed of 4 well-trained expression editors, and realizes the expression transformation of adversarial samples through global structure supervision and local detail supervision.

For the expression supervision module, the corresponding ablation experiments are given in the supporting materials of the paper, which verify that local detail supervision can reduce the artifacts of the generated images. It effectively improves the visual quality of adversarial samples and improves the expression synthesis accuracy of adversarial samples.

In addition, the paper defines the concepts of continuous adversarial manifolds and semantic continuous adversarial manifolds , and proved in detail that the adversarial manifold generated by is homeomorphic to the AU vector space.

Summary

Summary As mentioned above, this research proposes a new attack paradigm called GMAA, while expanding the target domain and countermeasure domain, improving the performance of the attack. For the target domain, GMAA improves the generalization ability to the target identity by attacking a collection of states instead of a single image. Additionally, GMAA extends the adversarial domain from discrete points to semantically continuous adversarial manifolds ("point-to-face"). This study instantiates the GMAA attack paradigm by introducing domain knowledge of expression editing. Extensive comparative experiments prove that GMAA has better attack performance and more natural visual quality than other competing models.

The above is the detailed content of From point to surface: generalizable manifold adversarial attacks, from individual adversarial to manifold adversarial attacks. For more information, please follow other related articles on the PHP Chinese website!