AI Frontier Observation · Interpretation Date November 13, 2026 · Information Deadline 20:30:38
A rule that simply counted length achieved an AUROC of 0.9988 in the Llama test. It almost perfectly distinguished answers that fell into repetition from normal ones, yet it never sounded the alarm before a loop began. At 21:43 Beijing time on November 12, Marco De Negri and colleagues at ETH Zurich submitted their PREWARN paper to arXiv. The apparent contradiction raises a practical question: if a monitor is supposed to give someone time to stop generation, how far has the model already written when the first alarm sounds? In the submission record, the authors note that the paper was accepted at the NeurIPS 2026 Interpretability as a Science workshop. Paper and revision history

Schematic of original concept. The lights on the paper tape mark an alarm, and the ring on the right indicates that it starts to repeat.
A rule that only lights at the end
It is assumed that the model is solving a problem, and after writing "Substitute into the above formula" and then back to the same step, write a paragraph of derivation as it is, and write it all the way until the output is cut off. When you look back at the record, you can move the cursor to the place where the derivation first appeared and mark the starting point of the cycle. But those who read the answer in real time in that position, there is only an ordinary derivation in front of them. He hasn't seen it a second time. This poor information is the reason why the early warning problem deserves a separate study.
The author limits the research object as an answer that continues to repeat until the output quota is exhausted, and generates a maximum of 4096 tokens at a time, that is, the text unit that the model outputs one after another. Repeated answers that can end on their own are still normal according to the definition of research. In this way, reaching the upper limit is a very strong clue. The length rule only gives a high score at the last position of the touch. If the highest score of all positions is taken when scoring the entire answer, it can naturally put a large number of failed samples in front of the normal sample. It is this sorting ability that AUROC measures. Leave the high score until the last minute, the ranking is still, and the time for intervention has been exhausted. Paper body, section 2 and table 1
How to find a scale for "early"
In order to have a comparable time for each alarm, the researcher first asked the model referee to point out where to start, and then manually review the answer to the upper limit in the test set. When artificially agreeing that a certain answer is indeed stuck in a loop, it is also possible to move the starting point. In the three models, the proportion of the referee's starting point for artificial retention is between 74% and 92%. Select the beginning of the first loop as zero point, which is more strict than waiting until the second time of the same sentence appears. In another standard, the same lamp will appear earlier. Experimental settings and limitations
It's just the monitor. The author first selects the threshold on the Validation Set, so that the normal answer of up to 5% has triggered the alarm, and then fixed the threshold to see the test set . Read the result twice. Of all the failed answers, how many have been ringing from start to finish at least once? In those failed answers that have been ringing, how many times before the starting point? The former is the detection rate and the latter is the early warning rate. If a system only speaks on the few most certainly, even if it is timely, the reader still needs to know how many lines it misses. Evaluation protocol, Section 3
The same ruler, measure the opposite alarm timing
What can you really see in advance? The author trains the Linear Probe, which is equivalent to a simple reader at a certain layer of the model. Every time a token is generated, the risk score is read out from the internal state at that time. Let's first look at the solution to maintain the fixed weight of the generated model. In the same source test for manual review, Llama-3.1-8B-Instruct has 284 circular responses. The probe detection rate is 1.00 in the table, the early warning rate is 59.4%, and the median position of the first alarm is 137 tokens before the starting point. The previous length rule also detected all the loops, but the median position of the first sound was 2958 tokens after the starting point. These two systems are handed over to a person who only wants to screen bad answers afterwards, and to a person who wants to stop the generation in time, the value will be very different. Experiment Table 1
Change the reader to the 7b instruction model of Mistral, about 99% of the 315 loop answers still triggered the alarm, but only 26.2% of the loop answer that triggers the alarm It rang in advance, and the median position for the first alarm was 108 tokens late. The 8b instruction model of Apertus is closer to Llama, and the median position of the first alarm on the 116 loop answers is 172 tokens ahead of the fixed weight scheme. These probe results are summaries of three training random seeds. When facing a new answer, the median advance cannot be regarded as a fixed buffer time reserved for the user by the system. Experiment Table 1
What should we test after the alarm sounds?
The authors also used a low-rank adapter to adjust the internal representations read by the probe. At the same layer in Llama, the share of flagged positions in the 256 positions before a loop rose from 8.7% to 48.8%. The share of answers receiving an early warning rose from 59.4% to 65.5%. Sounding ten more alarms for an answer that already had an early warning raises position coverage without warning anyone about a new answer. More positions were flagged in earlier intervals, too. Would persistent alarms become distracting on a correct solution that genuinely takes time to write? The authors’ repository documents the training settings and the format for recording loop onsets, making it possible to check by answer length. Experiment Table 1 · Author code repository
If you change the question, this advance will be changed. The authors left the two topics for the test, and the Llama fixed weight plan has an early warning rate of 20% on these sources, and the median position of the first sound has also moved to the beginning of the cycle. The above figures are all from the author's experiments, and no independent reproduction was found in this search. Section 4.3
I would like to see a comparison of the real action according to the alarm: let the same batch of questions generate as usual, stop after the alarm, and resample after the alarm, and at the same time record the token saved, the number of questions that were answered correctly, and the normal long answer that was wrongly truncated. The paper measures the timing of the alarm, and the author reserves the intervention after the alarm for follow-up work. If the savings are mainly invalid loops, and the correct answers can be saved, then "137 tokens in advance" will have a clear use value. The most reliable use of this number at the moment is to help us see when a beautiful classification transcript is missed. Paper conclusion and limitations
References
- arXiv Submission Records and Workshop Receipt Instructions
- The full text of the PREWARN paper, method, table 1 and limitations
- Swiss-AI/Degeneration-Probe, description of training and evaluation
Apps by the author
I also build small tools for everyday life. Here are two of my apps:
- Daily Expense Journal: Track your income and spending, plan a budget and set savings goals. View on the App Store.
- One Small Action: Pick a small action that fits the time and energy you have today, then record how it felt and what you noticed. View on the App Store.
Find more articles and projects on my website: zffjzz.top. I’d love to hear how the apps work for you and what I could improve.