Cover illustration

  • Date: 2026-09-21
  • For graduate students, early-career researchers and research engineers
  • Scope of reproduction: two classification linear models, continuous disturbance and evaluation audit; the text template provides design examples, and the model reasoning is waited for manual verification.

Abstract

Add a little noise to the input, and the model is still correct, which is not enough to show that it can resist intentional disturbances. The more hidden error is that the number of attack program steps is too few, the gradient is disconnected or the budget unit is wrong, but the attack failure is regarded as a model success. This paper trains a resolvable classifier, and solves the worst-case scenarios of random search, single-step gradient and multistep attack. The goal is to establish a trusted evaluation chain, rather than re-enact the list results of large-scale visual models.

Contents

  1. Reproducing value and paper problems
  2. Core Formulas and Threat Models
  3. A guide to the official implementation
  4. Minimum experiment and operation
  5. Evaluation protocol
  6. Results and failure analysis
  7. Semantic boundary of text attack template
  8. Failure Troubleshooting and Verification Records
  9. Follow-up research questions and summary

The problem of reproducing value and paper

The adversarial sample is a wrong input that is actively sought according to a certain restriction, and random noise has different purposes. What the paper reports. The work of Goodfellow et al. proposes a fast gradient symbol method, that is, FGSM, and explains the influence of small disturbances with linear accumulation; Madry et al. wrote the problem as a robust optimization, and separated the two links of "Looking for Bad Input" and "Learning Good Parameters". [1][2]

AutoAttack shows how step size, the objective function, and gradient problems can affect evaluation. Complementary attacks can reveal failures that a single attack misses. [3] Even a suite of attacks cannot prove that every allowed perturbation has been ruled out. We chose a simple model with an exact answer so that we could check directly for failures the evaluator missed, without having to trust a stronger attack as a reference.

The core formula and threat model

The threat model specifies what the attacker knows, what can be changed, how big it can be, and how it is successful. This time, it is allowed to read weights and real tags, which belong to white box and non-directed attacks; attack input features, do not modify training data, labels or model parameters. Non-directed means that only the prediction error is required, and the error category is not specified.

The common goals of robust training are:

min⁡θE(x,y)[max⁡∥δ∥∞≤ϵL(θ,x+δ,y)].\min_\theta\mathbb E_{(x,y)}\left[\max_{\|\delta\|_\infty\leq\epsilon}L(\theta,x+\delta,y)\right].

Here the parameter is θ, the sample and label are x, y, and the disturbance is δ; the absolute change of each dimension is restricted by the infinite norm. The inner layer increases the loss, and the outer layer adjusts the parameters. This article only trains ordinary classifiers, and then tests the inner layer to solve, No confrontation training was performed.

Let the score of the 20-dimensional input be f(x)=wᵀx+b, the label is plus or minus one, the logical loss is log(1+exp(−yf(x))). The batch input shape is [N,20], the weight is [20], score and label are [N]. signed interval m=yf(x) For positive, the classification is correct, and the smaller the value, the closer to failure.

The minimum interval can be precisely written as:

m∗=y(w⊤x+b)−ϵ∥w∥1,δ∗=−ϵy sign(w).m_* = y(w^\top x+b)-\epsilon\|w\|_1, \qquad \delta_*=-\epsilon y\,\mathrm{sign}(w).

The sum of the absolute value of the weight determines how much interval the budget can be reduced. Each dimension moves in the unfavorable direction, just to reach this lower bound. The accuracy here depends on the two-classified linear fraction, the monotonic logic loss and the declared disturbance set, and cannot be directly generalized to the deep network. There is no zero interval juxtaposition in this data; generally, the implementation needs to be clearly juxtaposed how to judge the class.

FGSM uses a symbol for one input gradient; the projection gradient attack PGD repeats "Rise along the gradient, and then project back to the original input near the original input". Although the name follows the gradient descent abbreviation, the attack step is increasing the loss. Each projection surrounds the original X, and the budget cannot be redone around the previous step, otherwise the total disturbance will cross the line.

3. Guide to the official implementation

Read MadryLab first pgd_attack.py.[4] LinfPGDAttack Save budget, step, step, and random starting point configuration; tf.gradients It is the gradient of loss to input. perturb Update the input first, then crop to x_nat±epsilon, and finally crop to the pixel interval. The former limits the attack strength, the latter guarantees that the input is legal, and the two have different responsibilities.

The implementation returns to the last step, which cannot assume that it has preserved all historical failures. This article adds the worst candidate record by sample, and then checks the AutoAttack's run_standard_evaluation:[5] It initializes with clean prediction first robust_flags, and then label the sample that was defeated by either attack as a failure. This is the union of the failed sets, not the average of several attacks Accuracy .

Track while reading set_version: The current standard configuration will overwrite the default values of some constructors, you cannot just copy the number of restarts in the initialization. Both repositories are masters during retrieval, and the commits are not locked; static reading is completed here, and the official framework, weight and dynamic dependence compatibility are all manual verification.

4. Minimum experiment and operation method

Fighting robustness evaluation process

The clean input in the figure enters the baseline at the same time, attack and parse the check branch. The attack candidates are handed over to the freezing model after the budget check is passed; the parsing branch and the actual prediction are jointly entered into the failure audit. The cross line is not a meeting point, and the lock indicates the same set of fixed parameters. Icons only show feature changes and do not represent real image experiments.

Fixed seeds 79, 80, 81, each independently generate training, development, test set, the number is 1024, 256, 2048. The probability of labeling is taken as positive and negative; the first dimension is the Gaussian noise of the label plus the standard deviation of 0.6, and the remaining nineteen dimensions are the independent noise of the label and the standard deviation of 0.3. The full input line check is not duplicated.

These are abstract measurement features, and the value range is the entire real number space, which is not normalized, nor is it cropped to the pixel range. Labels represent the latent categories at the time of generation and are still used after disturbances. This is a clearly defined measurement interference task, not natural data that has proven human semantics. The budget value therefore cannot be directly compared to the image pixel budget.

The model has only 21 parameters, using ordinary logic loss plus L2 regular; full batch gradient drops by 1,000 steps, the learning rate is 0.2, and the regular coefficient is 0.01. The development set only records the scores, no parameters are selected; the last step is uniformly taken. The three runs jointly change the data and the initialization, and the standard deviation describes the repeated changes of the composition setting.

From the article directory, run:

python3 -m pip install -r code/requirements.txt
python3 code/robustness.py
python3 code/audit.py

The actual environment is Python 3.12.14, NumPy 2.3.5, CPU double precision, already dependencies, no need to install. The script prints key shapes, saving all division, weight, training trajectory and worst attack candidates. document explanation see How to run the example.

5. Evaluation protocol

Five budgets are pre-fixed: zero, 0.05, 0.1, 0.2, and 0.3. Each input of the random baseline takes 16 uniform disturbances; the weak PGD takes two steps from the clean input, and the step size is 1/20 of the budget; the FGSM takes a complete budget; the full PGD Use three random starting points, twenty steps each time, with a step size of one-fifth of the budget.

For each type of search, clean input is always included as a candidate; PGD keeps the minimum interval in all iterations and restarts. In this way, the subsequent steps accidentally change the prediction back to the correct one, and do not erase the found failure. The scores of each sample can be compared with the lower bound of the resolution, and in addition to Accuracy, the number of missing attack samples is also reported.

The experience robust Accuracy takes the entire test set as the denominator, and requires clean input and the search candidates are correct; the already wrong sample is also a failure. The success rate of the conditional attack is only the denominator with the correct classification of the correct classification. Under normal circumstances, the robust accuracy obtained by a limited search is the upper realm of the true worst case of the test set. The weaker the attack, the looser the upper world may be.

These configurations write scripts before reading test results, without picking the best torrent or callback parameters. Different attack calculation budgets are not the same, so the table is used for audit adequacy and cannot be ranked as speed. To compare efficiency, the number of gradients, forward times and run time should be saved separately.

6. Analysis of actual results and failures

What we tested. The clean accuracy of three runs is 0.9946±0.0018. The following fixed disturbance budget is 0.2, the report mean and the standard deviation between seeds, not Confidence interval; see for details experimental results.

Candidate search method Experience Robust Accuracy Conditional attack success rate
random sixteen times 0.9800 ± 0.0017 0.0147
Two steps, small steps PGD 0.9910 ± 0.0015 0.0036
FGSM 0.7697 ± 0.0124 0.2262
Twenty steps, three starting points PGD 0.7697 ± 0.0124 0.2262

Each interval between FGSM and full PGD reaches analytical lower bounds; random search misses an average of about 431 attackable samples, and weak PGD about 453. The total displacement of the two-step displacement is only 10% of the budget, and even if the direction is correct, it may not be able to reach the boundary. "Using gradients" is not evidence of attack adequacy.

Here, there is a model structure for the equality of FGSM and full PGD, which cannot be explained as multistep attacks are generally useless. As the budget increases, the analysis of the three models is robust to Accuracy; this rule is aimed at the complete set of disturbances of the nest, and the finite search curves for random sampling are not necessarily monotonous.

We also added a fault example: directly return to the original input, the simulated attack is not moved. It shows zero success rate, but misses an average of about 461 attackable samples. It only verifies that the auditor can identify obvious failures, and is not an empirical evidence of gradient masking for a certain real defense.

7. Semantic boundaries of text attack templates

A small distance between continuous feature vectors need not correspond to a small change in the meaning of a text. CheckList distinguishes tests where behavior should stay the same from those where a change is expected. [6] For example, replacing “movie” with “film” in a positive review should preserve its label. Replacing “excellent” with “not excellent” changes the label, so a negative prediction can no longer count as a successful attack.

Additional Six template records, including punctuation, blank, synonyms, instruction format, and negative and antonym two label change control. Each article retains the original text, candidates, revised budget and expected labels; these labels are designed and judged by the author and wait for manual verification. Here only the record format is checked, no language model is called, so there is no text attack success rate.

During the real operation, it should be divided into development and testing according to the template family, first fixed the total query budget, output analysis rules and decoding seeds. The condition success rate is only calculated on the samples that were originally correct and the label is maintained, and the ratio of invalid rewrites is reported. Changes in semantics, output format changes and answer errors should be separated, otherwise a high attack rate may be just that the gold label has not been updated.

8. Failure investigation and verification records

Check the threat model first: the budget before and after normalization is not the same amount; pixel field cropping changes the feasible set. This audit sets another boundary example, which proves that the real-number field analytical formula cannot be used without modification for the cropped pixels. Check the gradient and statistics again: Do not disable input derivation during attack phase, and do not use average loss improvement instead of sample-by-sample failure checking.

The two scripts are checked through syntax with the CPU. The independent audit does not import training code, and recalculates 153,600 candidate predictions and 3072,000 coordinate budget checks with scalar operations; the maximum value difference is about 4.44e−15. The maximum error of the finite difference check between the training and the input gradient is about 7.25e−12, see the verification record for details.

The worst candidates were saved for each search, and all intermediate paths were not persisted; the independent audit also did not retrain the model. These restrictions should be delivered together with the results. This time, the official neural network, real images, GPU, AutoAttack integration or language model are not run, and it does not claim to reproduce the benchmark value of any original paper.

9. Follow-up scientific research issues and summary

Author’s interpretation. The next step can be replaced with a small nonlinear network, maintain data, budget and candidate record rules, and observe the gap between FGSM and multi-restart PGD; at this time, the resolution of the answer is invalid, and a new verification method is required. Comparing different training goals can we discuss whether robust training improves the failure set, and we cannot treat weaker attacks as training benefits.

The text direction should first solve the rewriting effectiveness: under the same budget, is the manual approved label maintenance ratio changed with the template family? Does the failure focus on negative, entity replacement or output parsing? Let each attack leave revisitable evidence, so that the "model is not broken" can be rewritten into a more accurate scientific research conclusion: under the constraints and search budget that have been declared, what we have found, but still cannot be ruled out.

References

The following sources are 2026-09-21 Retrieve and actually open; all the numbers in the body are from local experiments.

  1. Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy. Explaining and Harnessing Adversarial Examples. ICLR 2015; preprint starts in 2014, this time read V3 (2015-03-20). Version page; Full text. Support FGSM and linear analysis.
  2. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu. Towards Deep Learning Models Resistant to Adversarial Attacks. ICLR 2018; preprint starts in 2017, this time read V4 (2019-09-04). Version page; Full text. Support robust optimization with PGD.
  3. Francesco Croce, Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. ICML 2020,PMLR 119:2206–2216. Conference page. Support the evaluation of traps and combination attacks.
  4. MadryLab official pgd_attack.py, master, unlocked commits. Support gradient, projection and pixel field crop reading.
  5. AutoAttack Official Repository; autoattack.py, master, unlocked commits. Support failure sets cumulative and standard configuration reading.
  6. Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, Sameer Singh. Beyond Accuracy: Behavioral Testing of NLP Models with CheckList. ACL 2020:4902–4912. Conference page; Full text. Support the classification of behavioral tests, and do not support the text grades that are not run in this article.