AI Frontier Observation · Interpretation Date: November 10, 2026 · Data Deadline: Beijing Time 20:28:04

Imagine that you are looking for clothes with a photo of a jacket, first say "find a red hooded one", and then add the sentence "change the color to navy blue". If the front is the red hooded jacket, the system remembers what you said, but you can't figure out which sentence counts. If it handed back a navy blue stand-up collar jacket, it seems that only the last sentence was heard. Alibaba and Beihang University of Beihang University and others designed the multimodal retrieval benchmark TRIAD, which put these two confusing mistakes into the same experiment. Its Pre-printed first edition Submitted at 21:11 on November 9th, Beijing time; it is worth reading about how the author makes the same requirement test twice.

Three jackets are juxtaposed, and the translucent tracing paper highlights the hooded features that still need to be retained after the color change is highlighted

Original schematic. Jackets are examples given in this paper to explain the control design, and are not paper samples or search results.

How to make the same requirement test twice

One question is to give a reference picture, and once clarify what you are looking for at the end; another question starts from the same reference object, distributes the conditions into several rounds, and allows users to revise the previous requirements. The last thing I want to find. If the system is good at identifying colors and styles, but performs poorly in the second question method, there is a reason to go back and check how it uses the history of dialogue. Compare a simple single-sentence question with a complex pair of topics, and you can't get such specific clues.

This control is one of the three types of TRIAD tasks. The other two categories examine whether the specific picture and the upper-level concept can be searched separately, and whether the conditions of affirmation and negative can be met at the same time. Each of the six subsets has a thousand queries, each of which is sorted in a fixed one thousand candidates. A query can have multiple eligible answers. The author uses the average precision mean (mAP) to score: it can be understood as looking down along the list. Every time you encounter a qualified item, it depends on how accurate the results ranked in front of it, and then the average item and query are averaged. Even if the first dress happens to be the right one, and many unqualified clothes are mixed in the back, the score will be affected. body text

Forty times, what is the decline?

In the two fields of clothing and automobiles, the author compares the single-round and multi-round scores of 20 configurations respectively. These twenty configurations come from seventeen systems. In the 40 sets of "configuration × field", there are thirty-nine groups with lower multi-round maps. This number describes the direction of the average score change: it does not mean that the thirty-nine of the forty searches are not as big as the difficulty that each system encounters.

See Paper table 4 You can understand this difference. On the apparel subset, the Qwen3-VL-Embedding-2B is reduced from 0.442 in a single round to 0.212 in multiple rounds; the configuration of the same series 8B has been reduced from 0.508 to 0.454. There is one more exception: the Embed-RL-4B with T-CoT has risen from 0.360 to 0.376 on clothing, but it still falls on the car. Just looking at these lines, you can judge that the difficulties are stigmatic, but it is not appropriate to identify the scale or a certain reasoning method as the cause according to this.

There is also a detail that is easily overlooked: the candidate pool of the original pairing question is highly overlapping, but it is not exactly the same. Even if the two questions are looking for the same clothes, as long as one of the more almost the same styles are mixed in, the score may change. In the appendix, the author uses the same candidate pool for multiple rounds of questions to re-score the single round of input, and the overall downward trend is still there. This contrast enhances confidence in the trend, but cannot attribute every difference to a word or a round of dialogue.

You can also read this comparison backwards. A single round of input is equivalent to someone sorting out the needs of the system and giving it a clean query. If even this one is not well arranged, improving the memory of the dialogue may not help you. Therefore, the single round score itself should be retained with the difference between the two inputs.

Why is it useful and harmful to delete old stories?

The most explanatory step in the research is to remove the previous dialogue and only give the last round of the retrieval system. In this supplementary experiment, the authors used eleven configurations, two fields in total, rather than the forty sets of the main experiment. The final round and full history face the exact same candidates. According to all inquiries in various fields, the 22 groups have higher scores to keep history: just say "replaced with Tibetan blue", it is not enough to explain what style is appropriate. Appendix D.1.1 and Figure 10

However, just looking at the dialogue that the user has revised the conditions, there are ten groups in the twenty-two group, but the final round is better. These two results are not contradictory. Under normal circumstances, the previous article provides information that is not repeated in the last sentence; once the user changes his mouth, there is another value that should be invalidated in the previous article. Retriever may benefit both from the "Hot" and be biased by the "red". The "may" here cannot be saved: sorting can reveal the relationship between old conditions and mistakes, and we can't directly read out how to store or delete colors inside the model.

Next, let's see which dress is the fault

A more revealing question than another overall retrieval score is which mistakes become rarer as the new requirement becomes clearer. Fix the reference image and candidate pool. Include two easy-to-confuse garments: one meets every requirement except the new color, while the other lacks only the hood that was required all along. Rephrase “change it to navy” in natural, equivalent ways and track how often each wrong garment appears near the top. This is a proposed follow-up to the paper. If a change pushes the old color down the rankings but promotes garments without hoods, it’s hard to say the system understands the user’s revision any better.

The evidence so far comes from the authors’ evaluation on a fixed candidate pool. Clothing and cars lend themselves to clearly stated requirements. Real users often express vague preferences and may still be working out what they want to keep. I didn’t run the authors’ code or find an independent replication in this search. TRIAD gives us a precise question to pursue: when the final intent is clear, can the system change only what the user asked to change? Whether that distinction survives a different candidate pool and different wording tells us more than another place on a leaderboard.

References

The following entrances are all from the same preprint, which does not represent independent verification.

  1. arXiv Version Records and Summary
  2. TRIAD First Edition Full Text: Tasks and Scoring Agreement
  3. Main Experiment Table 4: Single and Multiple Round Scores of the Same Final Intent
  4. Appendix D.1.1: Final round of input, complete history and revision dialogue

Apps by the author

I also build small tools for everyday life. Here are two of my apps:

  • Daily Expense Journal: Track your income and spending, plan a budget and set savings goals. View on the App Store.
  • One Small Action: Pick a small action that fits the time and energy you have today, then record how it felt and what you noticed. View on the App Store.

Find more articles and projects on my website: zffjzz.top. I’d love to hear how the apps work for you and what I could improve.