If we look at these papers together today, I think the most noteworthy line is not "the model has grown bigger", but:
Everyone began to deal with the fragile links of AI systems in real use more seriously.
These vulnerabilities are not always on the model parameters themselves, more often at the edge of the system:
- Will the Agent make the wrong key action in the long-horizon tasks?
- Will the benchmark be "score" by the model instead of being really solved
- In what structured conditions will VLM fail in security scenarios?
- Is the bottleneck of GUI Grounding in the generation stage or the prefill stage earlier?
- How does the model learn the preferences that users do not express from a small number of interactions?
They seem to be in different directions:
- Embodied Agent
- Agent Evaluation
- Vision-Language Model Safety
- GUI Agent / UI Understanding
- Human Preference Learning
But together, it actually points to the same problem:
The AI system is transitioning from "being to do problems" to "being able to do things stably", and what really determines the upper limit is often the layers that were easily overlooked in the past, such as verification, evaluation, failure analysis and preference modeling.
I’ve picked 5 papers for today’s roundup. Some may attract fewer headlines than others, but each gave me ideas about how to design future systems.
1) VeGAS: Embodied Agent Don't rush to output actions, first add a "verification"
- arXiv:2605.12620
- Title: Think Twice, Act Once: Verifier-Guided Action Selection For Embodied Agents
- Orientation: Embodied Agent / MLLM Agent / Test-Time Verification
The problem of this paper is very practical.
Now many Embodied Agents have used the multimodal model to understand the environment, generate chain reasoning, and then output actions.
The problem is, Once the scene becomes long link, multi-objective, and distributed, the model can easily make mistakes at a certain step, and then the entire trajectory will collapse together.
The authors call their approach VeGAS (Verifier-Guided Action Selection). The idea is simple, and I think it illustrates a broader trend:
- policy doesn't just sample one action
- Instead, first sample a set of candidate actions
- Then choose the more reliable one by a Generative Verifier
The core idea is actually a bit like moving "self-consistency" from text reasoning to action selection:
Instead of directing the first reaction, the system has an explicit screening layer before execution.
What's more interesting is that the paper is very straightforward:
- Directly take the ready-made MLLM as a Verifier, the effect is not good
- So they also did the LLM-based failure case data synthesis, and specially trained Verifier to identify potential errors
In other words, this paper really emphasizes not "one more model is stronger", but:
The validator has not seen how the system will fail, it will be difficult to work at critical moments.
On embodied reasoning benchmarks such as Habitat and ALFRED, VeGAS made clear gains on the harder tasks involving multiple objects and long action sequences. The paper reports improvements of up to 36% over a strong chain-of-thought baseline.
I think the inspiration for this work is:
In the future, the Agent system will not only rely on a "reasonable policy" individual to fight alone.
A more stable form may become more and more like:
- generation candidate
- do explicit verification
- then decide whether to execute
This route is very important for robots, GUI agents, and automation execution.
2) BenchJack: A lot of agents, maybe not as safe as you think
- arXiv:2605.12673
- Title: Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack
- Direction: Agent Benchmark / Reward Hacking / Evaluation Security
If you used to ask "Which Agent Benchmark is more difficult", then this paper asks another more heart-wrenching question:
Are these Benchmarks seriously prevented from cheating?
The author's argument is clear:
Today, many Benchmarks have become an important basis for model capabilities, investment and financing judgments and product routes, but their own design may not have a strong enough adversarial safety perspective.
The paper first sorts out the eight common Flaw patterns of Reward Hacking, and then does an automated Red-Teaming system BenchJack to drive the Coding Agent to actively audit Benchmark, looking for "high score but not really accomplishing the task" Exploit.
The most terrible thing about this is the result:
- They have found a lot of exploitable vulnerabilities on 10 popular Agent Benchmark
- A total of 219 different problems were exposed
- Some Benchmarks can almost get almost full marks without "doing a task"
- The extended iterative patching process can push some of the hackable-task ratios of some Benchmark from close to 100% to below 10%
This conclusion is actually quite heavy.
Because it means:
Some Benchmark's high scores do not necessarily mean that the model can really do things, only that it is enough to exploit the loopholes in the evaluation of the rules.
I think this paper is worth watching for all people who do Agent reviews, especially now that everyone is more and more like to take it:
- WebArena
- OSWorld
- Desktop / Browser / Terminal benchmark
To do the headline ranking.
Ranking is of course important, but if the evaluation surface itself can be attacked systematically, the explanatory power of the leaderboard will decline rapidly.
In a word, this work is supplemented by:
"Safety Engineering" of Agent Evaluation.
Such papers may not be as eye-catching as new models, but the long-term value is very high.
3) REVELIO: don't just look at the average accuracy, the real danger of VLM is the "structured failure mode"
- arXiv:2605.12674
- Title: Revealing Interpretable Failure Modes of VLMs
- Direction: VLM Safety / Failure Analysis / Interpretability
I love this paper because it captures a problem that is often masked by Benchmark:
The model does not make mistakes evenly, but persists in errors under certain combinations of certain conditions.
The author defines this situation as Failure Mode, and emphasizes that such failures are often combined by multiple "interpretable concepts".
For example, in autonomous driving scenarios, errors are not necessarily just "wrong objects", but may be:
- Catch
- Mine weather
- close-range pedestrian
- Weakness of spatial relationship
These factors are combined to trigger the stable failure of the model.
In order to systematically dig out this kind of model, the paper proposes REVELIO, and uses two search strategies to find the vulnerable conditions of VLM in the discrete combination space:
- diversity-aware beam search
- Gaussian-process Thompson Sampling
The author then applied it to autonomous and indoor robot scenarios, and found several vulnerabilities that were not systematically disclosed before.
The conclusions given by the paper are very straightforward:
- The model may be space Grounding is very weak
- Major occlusion may be ignored
- Safety hazards may be missed
- It may also be too conservative, resulting in too many false alarms and reduced efficiency
Why do I think this is important?
Because in many cases, the security scenario can't just accept a sentence "The overall accuraacy is not bad".
What really needs to be answered is:
Under what conditions will the model overturn the car, and whether these conditions can be understood and avoided by human beings.
The value provided by this paper is to take the "model unsafe" from a general judgment, and take a step forward to the Failure Mode map that is "locatable, enumerable, and repairable".
4) Re-Prefill: the key to GUI grounding, not necessarily in decode, but in prefill
- arXiv:2605.12549
- Title: What Happens Before Decoding? Prefill Determines GUI Grounding in VLMs
- Direction: GUI Agent/VLM Grounding/Inference-Time Improvement
This paper is very accurate.
Many GUI Agent jobs are trying to improve Grounding, that is:
When the user says "click the button in the lower left corner", whether the model can accurately position the target element.
In the past, many training-free methods would do:
- Reasoning multiple times
- Cut repeatedly
- Candidate Aggregation
But this paper points to a frequently overlooked problem:
Many Grounding errors are actually decided before decode.
The author splits the GUI grounding into two stages:
- Prefill stage determines the candidate UI element range
- The decoding phase refines the final coordinates
If the first step of the candidate is wrong, and then decode is fine, it is only "make a more precise choice in the wrong candidate".
I think this observation is very important, because it shifts everyone's attention from "how to adjust the output coordinates" to "what the model has noticed in the input stage".
Based on this insight, they propose Re-Prefill:
- Do the first round of prefill first
- Extract high-corresponding visual tokens based on Query position's attention information
- Put these tokens and instruction hidden states back into the input
- Do the second round of prefill and let the model think again
Essentially, this is a No training, only change the reasoning process method.
Experimental, it brings stable gains on multiple VLM and GUI Benchmark, including:
- ScreenSpot-Pro
- ScreenSpot-V2
- OSWorld-G
- UI-Vision
- MMBench-GUI
I think the greatest value of this paper is not just the 4.3% increase in the number, but it reminds us:
The bottlenecks of many multimodal systems are not "will not generate answers", but "the wrong focus has been selected in the input interpretation stage".
This is of great reference for those who do GUI Agent, web page operation Agent, and mobile Agent later.
5) CLIPR: The real preference of the user is often not in the clear rule
- arXiv:2605.12682
- Title: Learning Transferable Latent User Preferences for Human-Aligned Decision Making
- Direction: Preference Learning / Human Alignment / LLM Decision Making
In the last article, I chose a little "non-mainstream", but I think it is worth reading.
Many human-aligned decision making studies have a premise by default:
- Users will clarify their preferences
- The system can turn these preferences into rules
- Then make decisions according to the rules
But this is usually not the case.
Many important preferences are actually implicit:
- Users are more conservative or more aggressive
- More emphasis on speed or more attention to Accuracy
- More concerned about long-term stability or short-term income
- How do you want the system to choose when faced with blurred scenes?
These things are often not said to be complete at the beginning.
CLIPR's paper is:
Let the LLM infer the Latent User Preferences from the limited conversation and organize them into migrating natural language rules.
Its focus is not one-time personalization, but rather these preference rules:
- learn from a small number of interactions
- Can migrate in different tasks and in different environments
- Reduce the cost of frequent and repeated inquiries
This question will become more and more important to many agents.
Because when the Agent starts to take on longer chains and more vague goals, it is often not the task objective itself that really decides to "do it like what you want", but the trade-off logic that defaults and doesn't say it.
From the abstract, CLIPR performs better in three datasets and user studies than existing methods in alignment and inference costs.
I think the long-term value of this work is:
Alignment not only trains a more obedient model, but also includes letting the system gradually learn "the implicit principles for this user to make choices".
This is critical for personal assistants, decision-making assistants, and medical/financial assistance systems.
Final summary: The core keyword of today's batch of papers is not "ability", but "guardrail"
If you only look at the Headline, these papers today do not have the strong drama of that "one model swept the full list".
But from the perspective of system evolution, I think they are more interesting.
Because they are all the most prone to problems when making up the AI real landing:
- Verification before the supplementary execution of VeGAS
- BenchJack to make up benchmark to prevent cheating
- REVELIO Structured Failure Analysis of VLM Complementing VLM
- Re-Prefill in supplementing the pre-attention bottleneck of GUI GROUNDING
- The modeling of CLIPR in supplementing user implicit preferences
Summarize today's papers in one sentence:
In the next stage of AI competition, it is not necessarily just who will generate content, but who is more able to identify the fragile points in the system in advance and use structured methods to make up for it.
If you are looking at Agent, VLM, evaluation system or human-machine collaboration recently, I will give priority to these articles today.