For graduate students, early-career researchers and research engineers; search and experiment date: 2026-09-14.

Abstract
An agent avoids every wall but never reaches the goal. Did it misperceive its surroundings, forget where it had been, or simply run out of steps? We study embodied navigation in a maze that can be audited one step at a time. We fix the local observations and action interface, vary only whether the agent remembers its exploration, and use planning with the full map as a diagnostic reference. Across 360 episodes, we saved each state transition and independently recomputed success rate, path efficiency, and reward. The experiment reproduces the interaction and evaluation procedures. It makes no claim about real robots or the original paper’s training results.
Contents
- Reproducing value: let failure be positioned
- Core ideas and formulas
- A guide to the official implementation
- A minimal experiment and its results
- Evaluation Protocols and Logs
- Troubleshooting and limits of the evidence
- Questions for further research
- Summary and References
Reproducing value: let failure be positioned
The body agent changes the environment through movements and gets feedback from subsequent observations. This reserves location, obstacles, goals and limited action budgets, eliminating the need for image recognition and mechanical control, and it is easy to trace a failure back to specific decisions. Partially visible means that the agent sees only a part of the environment: even if the current position is the same, whether or not to go through a certain fork in the past will change the reasonable action.
Paper report: Chevalier-Boisvert et al. in Minigrid & Miniworld in 2023 The paper introduces a customizable two-dimensional and three-dimensional target task environment, and uses some observable decision-making process to describe the state, observation and action. Its value is to make it easy for researchers to modify tasks, rather than representing all abilities in a small environment. Papers and Versions; The second section of the full text
This question is: What is the difference between the performance of the same local planning procedure, each step forgets the history and accumulated history? This control is easier to explain than directly comparing two complex models, but still only for the selected maze and handwritten strategy. There is no language model call, and there is no reinforcement learning training, and the reward will not update the parameters in reverse.
Core ideas and formulas

The environment in the figure enters the memory after partial observation, and then the strategy selects the action; the full-state log is sent to the evaluation independently. Grids are just conceptual representations, not experimental screenshots. The target coordinates are public task input, and the unknown is the wall and the path; the evaluator has a full map, and the ordinary strategy cannot read it.
The interaction can be written as
is the state of the complete map, location and target, is a local observation, is the cumulative map and the accessed collection, is an action. express the sensing rule, update memory, Execute the move. The shape of this map is nine by nine, and the observation contains three multiplications and three categories of windows and two coordinate vectors; the action is a four-choice integer. The window is read in rows and columns, and the coordinates are represented by horizontal and vertical. Mixing will cause a seemingly reasonable wrong movement.
This article also defines the reward for each step
Indicates that the function takes one when the event occurs, otherwise takes zero. Successful actions also deduct the cost of the steps; the position of the hit wall remains unchanged but consumes a budget. This reward is only used for log diagnostics, and it is different from the default success reward of MiniGrid, and the return value cannot be compared. DoorKey official task description
The success rate should also be combined with the path length. The navigation evaluation report of Anderson et al. proposes SPL, that is, the successfully weighted normalized inverse path length:
is the number of test rounds, as a sign of success, for the shortest distance of the whole picture, for the actual moving distance. This experiment is based on grid, and the displacement of the impact wall is zero; in addition, the efficiency of all actions is included in the denominator, which is clearly distinguished from SPL. Section 4 of the full text of the report
A guide to the official implementation
Read the fixed label first doorkey.py: Position _gen_grid, observe how the partition wall, lock door, key and target are generated, and then see how the task inherits the base class. This order can help determine whether the task is unsolvable, or the strategy has not completed the pre-operation. This maze does not contain keys and doors, and it cannot be used to test the long-distance operation chain.
Read again minigrid_env.py of gen_obs_grid、gen_obs and step: First check the rotation and visibility, and then check the action execution, reward and end signs. in the default observation dictionary image It is a local coding, and it cannot be regarded as a real camera image just because the field name; complete rendering is not equal to the input actually received by the policy.
The source code also has a specialized test boundary: reaching the target will be set terminated, and then set the upper limit of the number of steps and set it again truncated, so both may be true at the same time. Round control ends when any one is true, and the target condition is checked for successful statistics. The above reading is fixed on the V3.0.0 label, and the library is not actually installed and run; the complete framework compatibility is subject to manual verification.
A minimal experiment and its results
Entrance minimal_agent.py Only relying on the Python standard library, this time using Python 3.9.6 is executed in the CPU. Dependency, field and recalculation method see README. Running in the article directory:
python3 code/minimal_agent.py
python3 code/audit_trajectories.py
The program digs out the connected maze through depth-first search, and the starting point is fixed; the target is sampled from the shortest distance of at least eight grid candidate positions. The development set reserves five maps but is not used for parameter adjustment, 30 test sets, and deduplicated according to the map hash. The test seeds are scanned from 100 sequentially, and the map is skipped repeatedly; the original map, target and rejected seeds are all saved, and the map is not screened according to the strategy score.
Compare four strategies. The random strategy is evenly sampled four actions; the memory strategy is combined with local grids, and the target is known to reach the target, otherwise the breadth-first search is used to go to the nearest grid. Breadth-first searches for the shortest route by moving steps. Empty the memory policy resets the accumulated grid and access set at each step, and then executes the same selection program. Full map planning has privileged information, which is only used as a reference for reachability and budget diagnosis.
Each strategy runs the same test set on the budgets 16, 32, and 64, for a total of 360 rounds. The random strategy uses a fixed random source derived from the map seed, and keeps the action prefix consistent when the budget changes. The following are the actual results of the thirty-two-step condition, and all the mean values are denominator with thirty maps. operational output; Complete summary
| Tactics | Success rate | SPL | average number of actions | Round collision rate average |
|---|---|---|---|---|
| random action | 0.000 | 0.000 | 32.00 | 0.590 |
| Empty memory at each step | 0.067 | 0.067 | 30.40 | 0.000 |
| keep memory | 0.933 | 0.906 | 18.40 | 0.000 |
| Privilege planning | 1.000 | 1.000 | 16.80 | 0.000 |
Empty the memory has only been successful twice, and the remaining twenty-eight rounds have timed out, but there is no collision; the current local obstacle avoidance can be correct, and the exploration may still fall into a round-trip. The memory is preserved 28 times, and the SPL is lower than the success rate, indicating that the success trajectory also has a detour. The average number of actions includes failures and cannot be interpreted as "the number of steps required for success".
The budget has increased to 64 steps, and all the memory strategies are successful, but the SPL is still 0.946; the memory is still only successful twice. Extra time alleviates the inadequate exploration and does not fix the loop caused by forgetting. At sixteen steps, the whole map plan has only been successful 18 times, so some timeouts are firstly due to insufficient budget. These are the actual measurement results of the current synthetic task, and there is no meaning of the paper benchmark score.
Evaluation Protocols and Logs
Each step saves the current observation, action, next step observation, reward, collision and two ending signs; rounds record map hash, seed, budget, shortest path and strategy name. The full picture is saved in the independent specification file, and the ordinary policy function only receives observations. The object interface helps prevent accidental reading, but it is not a safe sandbox for malicious code.
Independent Audit Procedure If the main program is not imported, the shortest path is calculated by another distance relaxation algorithm, and then 9508 transfers are repeated according to the preservation action, 19,016 observations are checked, and 84 summary indicators are recalculated. The 240 sets of trajectory prefixes of different budgets also passed the consistency check. audit record
There are also blind spots in the path indicators: hit the wall twice before the adjacent target, and then successfully move once again. This controlled inspection states that the shortest path does not equal the lowest execution cost. If you increase the rotation, wait or tool action, you should record the distance, the number of actions and time-consuming respectively, and do not let the zero movement disappear into the cost statistics.
Status termination and external truncation should be handled separately. The Gymnasium document states that value learning usually still needs to be valued from the next state when the external time limit is stopped; the bootstrap is stopped when the task is actually terminated. This time, only one hypothetical counter-example is counted: the reward is negative 0.06, the discount is 0.99, the next state is estimated to be 0.5, and the cut-off target should be 0.435; if all the ends are terminated, the negative 0.06 will be obtained. This is not the value function learned. Time limit description
This experiment regards the budget as an external restriction; if the research problem itself is completed for a time limit, the limited time domain task should be redefined and the remaining time should be added to the observation, and the caliber cannot be mechanically copied and cut off. Implementation also checks that the dual signs, illegal actions, wall-to-wall counters, and refusal to continue after the last step is successful.
Troubleshooting and limits of the evidence
First look at whether the map is consistent with the action, and then check the local input, memory update and stop rules. The test map is on the 100th, and the memory strategy is the shortest way to 18 grids, but it fails after 32 steps, and there have been eight return visits; This is the price of the order of exploration and cannot be attributed to collisions. The successful trajectory after the increase in budget provides evidence that can be repeated.
There is also a protocol problem in the development: different seeds generate the same map, and the initial unique assertion failed; the fixed corner goal has made the first round of memory strategy at the shortest path. Then it is changed to deduplicate and sample the target according to the hash, and the complete rerun after freezing. The previous round scores were not mixed into the official form. This data design refers to the initial round of observation, so the so-called test set is still a teaching diagnostic set, not an independent generalization evidence that has not been affected by development. verification instructions
The author's inference: the income here comes from the historical map and the access set jointly reserved, and the contribution of the two cannot be distinguished. Perfect positioning, static walls, no sensed noise and fixed action cost make the task easy; the three-by-three windows are also calculated without sight occlusion. Results cannot be extrapolated to camera navigation, real robots, language planning or learning algorithms. A random policy uses only one set of fixed derived seeds, and zero success is not the overall probability of success.
Questions for further research
First, keep the map, access count and target history respectively, which part reduces the invalid round-trip? One-factor ablation should be done on the newly frozen atlas, and the number of return visits and success rate should be paired and compared according to the map.
Second, when the memory capacity is limited, is it more effective to keep the recent observation or the key fork in the road? Fixed the storage budget, increased the scale of the maze, and observed whether the forgotten position corresponds to the failure trajectory; different budgets on the same map are matching conditions and cannot be regarded as new independent samples.
Third, after adding positioning noise, do more memories accumulate wrong maps? Inject the known disturbance first, check the coordinate merge, and then connect to the real perception module. Real MiniGrid, DoorKey pre-operation chain, learning strategy, multi-random repetition and Confidence interval have not been implemented, and wait for manual verification.
Summary
A small environment lets us link each state to what the agent observed, what it did, and what reward it received. First check that the inputs stayed within the allowed bounds and that the trajectories can be reconstructed from the logs. Then the roles of memory and the step budget become concrete hypotheses for why the agent failed.
References
The following first-hand information is actually retrieved and opened on 2026-09-14; the paper is the listed version, the source code is fixed to V3.0.0, and the content of the online document may change.
- Maxime Chevalier-Boisvert、Bolun Dai、Mark Towers、Rodrigo de Lazcano、Lucas Willems、Salem Lahlou、Suman Pal、Pablo Samuel Castro、Jordan Terry,2023, Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks,arXiv v1. Version page; Full text.
- Eleven authors such as Peter Anderson, 2018, On Evaluation of Embodied Navigation Agents, arXiv v1, the navigation evaluation working group report. Author and version; Full text.
- Farama Foundation,MiniGrid v3.0.0: environment base; DoorKey generates code; Official Task Document.
- Farama Foundation,Gymnasium, Handling Time Limits, main document: Termination, truncation and self-study instructions.