AI Frontier Observation · Interpretation Date: November 17, 2026 · Data Deadline 20:28:22

Schematic of original concept. Rooms and footprints are used to illustrate the problem, not the actual trajectory of the video or model.
Walking into a study, I saw a blue cup on the table. Going around the corridor, I saw the same desk and lamp, but the blue cup was gone. Did someone take the cup, or did you walk into the next door? Submitted by Henry Zheng et al. in the early hours of November 17th, Beijing time preprint In this, this kind of confusion has become a video question and answer that can be scored. The new benchmark is called ReSpaceBench. It has seized the ability that is easy to miss the measurement: to clarify what has changed in a room, we must first identify which room this is.
Go through the threshold, it is only a visit
Previous quizzes already ask for routes. For example in Public examples of VSI-Bench In the middle, the answerer starts from the side of the door facing the bed, and needs to make up for the steering when going to the window. To do this question, you need to put the furniture seen at different times into the same space. That item early study It also reports, allowing the model to explicitly draw spatial relationships when answering questions, which helps to judge the distance. However, even if the beds and windows of the two study rooms are exactly the same, the owner will not regard them as the same place.
The practice of ReSpaceBench is to set "Enter an actual room" as a boundary for one visit. The camera turns around in the house, the picture may change greatly, but the room has not changed; go out from this door and then enter through the door next door, the picture may be very similar, but the location has been changed. The author puts a similar room in a controlled indoor walking video as interference, and also arranges the changes of objects between different visits. In this way, the answerer can neither rely solely on the room category to identify the way, nor can he determine that the furnishings will reach a new place as soon as the furnishings are changed. The key to the problem has changed from "what this picture looks like" to "between the two scenes, where people have gone".
People and models should be compared on the same paper
Press author's report, the benchmark covers a total of 590 scenarios with synthesis and real homes, including 3,244 questions. Nine tasks involve identifying revisited rooms, tracking the state before and after, and using this information to continue reasoning. The team tested 32 models, and the best performing model under zero-shot conditions is 50.5%. The ZERO-SHOT scores here are separately reported after training with dedicated data. 50% cannot be directly read as "equal to guessing": the probability of guessing depends on the question and which options are given.
The manual labeler did not make the full volume. The researchers compared 450 of the questions. On the same subset, the best performing Zero-Shot model was 52.2%, and the labor was 90.1%, with a difference of 37.9 percentage points. This difference answers "how much is the difference between the two in the face of these same questions". If you take 50.5% of the whole paper to reduce the manual score, the impact of the change of the question will also be included. Another denominator that is easy to ignore is the room: a wrong answer does not mean going through the wrong door once, and a few questions involving the same room may also be answered incorrectly due to the initial misunderstanding.
Looking back at the video, how many seconds should I look for?
If you only remember that the two desks are very similar, you can intuitively let the model go back and take a few more glances. The author also measured three frameworks that can actively find video evidence. According to its report, these frameworks have not improved overall compared to answering the same underlying model directly. When the author checked the execution record, he saw two types of errors: some missed the required evidence, and some found the screen, but the room that belonged to the two visits was wrong. Summary of preprints It is also mentioned that it is important to keep the access and transition in the middle when selecting a video frame with the help of annotation.
Back to the blue cup example. If you look back at the strategy that is most similar to the "desk", you may find two close-ups on the desktop that are very clear; however, the two close-ups just bypass the intersection of the decision to answer. For this problem, it may be more useful to go out, turn, and enter the door in a blurry manner than another high-definition desktop. This is an explanation of the error mechanism, and does not mean that the author has proved that the three frameworks are looking for pictures according to similarity. There is no gain in the whole, and it does not mean that there is no problem that benefits from looking back.
The score can be practiced, and the next road test should be changed.
The team also built a synthetic training set, RS-30K, and says its templates, scenes, routes, and videos are all distinct from those used for evaluation. The authors report that fine-tuning Qwen3-VL-8B on it raised overall accuracy from 35.5% to 60.4%, a gain of 24.9 percentage points. The largest gain was on room matching. Comparing the same model before and after training shows that these tasks contain learnable patterns. The score alone cannot tell us whether the model learned to track its location along a route or became better at spotting the visual cues common in the questions. I checked the preprint’s version history and the authors’ abstract. I could not verify the detailed results table in the full text or find an independent replication in this search.
Another difference is who decides where to go. When doing the video question, the photographer has already passed through the corridor for the answerer; instead, if he is a robot looking for a cup, if he still can't tell the room at the door, it can go back to the road and look at the position of the door again. Conversely, it may also be because the first choice in the wrong direction can no longer see the clues needed to answer the question. The former ability can be repeatedly tested with video, and the latter also involves active selection, which cannot be used instead of Accuracy here.
I’d like to see a pair of videos that keep the furnishings at both ends of the walk the same but change the route halfway down the corridor. One route would lead back to the original room, the other to the room next door. A model using evidence from the walk should give opposite answers and identify the doorway that made the difference. Then reverse the test: keep the route fixed and move only the cup. The model should still identify the same room. Passing both tests would tell us more than another few percentage points about whether it remembered the place or just the tabletop.
References
- Zheng et al.: Lost in the Hallway, arXiv summary and submission records. The numbers of this new study are all from the authors.
- Yang et al.: Thinking in Space, Early Space Understanding Research.
- VSI-Bench Author Project Page: Video Q&A Examples. The last two items provide a background and are not a reproduction of new results.
Apps by the author
I also build small tools for everyday life. Here are two of my apps:
- Daily Expense Journal: Track your income and spending, plan a budget and set savings goals. View on the App Store.
- One Small Action: Pick a small action that fits the time and energy you have today, then record how it felt and what you noticed. View on the App Store.
Find more articles and projects on my website: zffjzz.top. I’d love to hear how the apps work for you and what I could improve.