Cover illustration

  • Date: 2026-10-30
  • For graduate students, early-career researchers and research engineers
  • Data deadline: 2026-10-30; search related research for nearly 12 months. This article has read three articles first-hand research, and has not run the agent experiment, and does not claim to be fully covered.

Summary: Remember a success, why do you still make mistakes?

Suppose the research assistant has used some sort of cleaning rule to process a data and get the correct result. It writes the rules to long-term notes, and the next time you encounter data with similar fields, you will copy them, but ignore that the unit has changed. What is even more troublesome is that the new results are summed up as "successful experience" and enter the next retrieval. This is a hypothetical scenario that illustrates the problem, not this experiment.

Long-term memory can preserve valuable information, and can also save inappropriate experience. Reliability depends on the writing basis, applicable conditions, calling method and repair ability, and cannot only be judged by the number of memorized articles or the successful judgment of one task. This article narrows the dispute to: performance changes come from knowledge accumulation, error reflow, or is experience summed up by mistakes? The three studies provide complementary evidence, rather than the two camps that support "memory useful" and "memory is harmful". Xiong et al、MemSecBench、MemForm.

Previously, this series has discussed reading and updating in long contexts, knowledge storage location and continuous learning. This article adds a cross-task feedback chain: how an output becomes a subsequent input, and how to confirm the error correction really cuts this chain.

First distinguish the behavior of memory content and memory

The long-term memory here refers to an external record that can be re-readable across tasks and can be read later, which is not equivalent to modifying the model parameters. Scenario memory saves "the problem and execution process at that time"; regular memory compresses several experiences into "how to do a certain condition". The original trajectory, abstract and rule are not the same object of evidence. See Xiong et al. §2.

This paper uses five links to organize analysis. Write determines what to save; retrieval determines what is currently seen; adopts the decision whether to do it; writes back to make this execution a future experience; fixes it to deal with existing errors and subsequent impacts. This is a framework for reading and design experiments, not a unified cognitive mechanism that has been proven.

Three differentiated interpretations can thus be proposed. First, the new record does cover the original missing information, thus helping the new task. Second, retrieval only improves the probability of imitation of old answers, both correct and wrong can be copied. Third, the original record was originally correct, but the conditions were lost during summary, screening or migration. These three situations can exist at the same time, and the average success rate cannot be judged separately.

For example, the assistant rewrites "this data set needs to be multiplied by a thousand" into "this type of fields multiplied by one thousand", the problem occurs in generalization; if the rule is written completely but is still used for mismatched data, the problem occurs in selection or adoption. Adding storage capacity to the two also does not necessarily solve any problem. Researchers need to record where the error first appeared, and cannot unify the final failure to "bad memory".

Evidence checking framework for memory lifecycle

Figure: The solid line shows the processing links that this article recommends to Check separately; the orange dotted line indicates the possible execution back-write path, which does not mean that all systems will write back, and do not prove that the write-back error must be transmitted. Check and Repair are inspection and Repair operations to be evaluated and are not guaranteed to be correct. This figure has no measurement data.

Natural error: save more execution, do not automatically accumulate reliable experience

The ACL 2026 paper of Xiong et al. starts from the same initial memory, and compares fixed memory, all writes and screening of different intensities. In its Table 1, the accuracy of the CIC-IoT Agent changed from 71.50 of fixed memory to 59.90 for all writes, and strictly screened at 85.40. This is a comparison of the same task for the same paper, and cannot be used to rank the attack rate of other papers. §3.1–3.2、Table 1.

However, there are key conditions for "strict screening": the paper is called Human or Oracle's evaluation, and the actual simulation is actually passed through the standard answer simulation in the experiment, and it is not obtained by real manual review every time it is executed. Most experiments use GPT-4o-mini; the subjects include controlled regression agents and three types of task agents. The results support the importance of quality control, and the same feedback cannot be promised to open tasks. §3.1.

More diagnostic significance is §3.4: keep the retrieved sample objects, replace the execution content with standard output, and then compare the subsequent performance. This is closer to the positioning content errors than just comparing the memory switch. However, the standard trajectory of AgentDriver may still be non-optimal; the paper does not include all abstracts and merge mechanisms into the experiment. §3.4 with Limitations.

The inference of this article is that if the replacement content can be improved after a fixed retrieval object can be improved, "just retrieved irrelevant tasks" is not enough to explain the entire gap. But this still does not measure the independent causal contribution of each memory. Several examples may interact, and a record that often comes with failure may just always be used for difficult tasks. It is valuable to treat the score after use as a quality clue, and it is too strong to treat it directly as an unbiased causal label.

This affects the interpretation of the deletion policy. The score rises after deleting a record, which may come from removing errors, reducing interference, or from changing the search candidate structure. If you want to assert that "hazardous knowledge has been identified", at least you need to keep the comparison between candidates and budgets, and check whether the deleted ones happen to be rare but important experiences. Remember some and remember more accurately, and should also report separately.

Preservation, adoption, and consequences are three different results

MemSecBench, a July 2026 preprint, uses a write, execute, and forget protocol. Execution and forgetting each start separately from the same state after a confirmed write. The study tests 24 combinations of agent framework, memory implementation, and model, with 310 cases per configuration. A case counts as a complete attack only if the malicious meaning persists, is retrieved and acted on, and produces the external consequence specified for that case. Selective repair must also preserve the designated benign memories. Methodology、Table 2.

Its counting caliber is particularly important: the write rate and end-to-end attack rate are denominator for all cases; the execution rate and repair rate are denominator of successful pollution. Successful pollution collections with different configurations can be different, and each configuration-case combination is only run once, and the result is mainly a descriptive comparison. The research comparison is a complete memory system containing adapter and service settings, not an isolated storage algorithm. Experimental Setup and Evaluation Metrics.

This inspires a method of ordinary reliability research: if there is an error in the database, it does not mean that the error will definitely affect the answer; the answer mentions the error, which does not mean that the wrong action has been taken. In turn, the model says "I have deleted" and cannot replace the check for the actual state. Evidence of each stage needs to be preserved in order to know which gate to improve.

However, this attack benchmark cannot estimate the frequency of natural errors used in daily use. It assumes that the attack content has reached the supported entry, and there is no cost to obtain the entry for the entry. The safety pressure test is different from the sampling mechanism of ordinary task flow. The value of putting the two together is a comparative measurement method. It is not that all memory systems are equally dangerous after erasing threat conditions.

The original experience is correct, and the formed rules may still be unreliable

The October 2026 MemForm preprint intervenes in the formation of memories. Two comparisons are central here. Selection changes which experiences are retained while keeping the candidate pool and retention budget fixed. Path holds the multiset of experiences and the updater fixed, changing only the order of integration. The authors observed corresponding changes in behavior, but note that sensitivity to order depends on the update process and what comes last. The main implementations share a rule-based memory protocol. §3–5.5.

Its attack success rate is also limited to the qualified questions of "can be answered correctly when there is no memory and clean memory"; the qualified sets of different models are different. Therefore, it measures the destruction of conditioned behavior, and cannot directly give the model a unified safe ranking, let alone estimate the real incidence rate. The alignment analysis of the memory text and the target behavior is also only related, not a causal media proof. §3.3、§5.2、§5.4–5.5.

For newcomers, this difference can be turned into a very specific question: do you keep samples, or the conclusions generated by samples? The former may allow rare conditions to disappear in the search, and the latter may also write local rules as general rules during compression. The same set of experiences is not guaranteed to get the same summary, which requires measurement and cannot be rolled out by "data are true".

For example, suppose that the two experimental records are respectively marked with normal temperature and low temperature conditions. If the temperature field is deleted when summarizing, any forged information may not be required for future failures. One sentence in the repair abstract is also not enough: if another derived note copy the old rules, subsequent retrievals may still be brought back. This is the path to be tested in this paper, and it is not a dissemination mechanism that has been jointly verified by three papers.

Which comparisons hold and which still lack evidence?

Research angles critical control or condition changes that can be observed cannot directly prove
Execute memory additions and deletions Same initial memory, different filter feedback Task performance changes with memory updates Open tasks have free and reliable gold labels
pollution life cycle Match the frame and model, track the same malicious semantics Write, Adoption, Consequences and Repair The probability of daily errors
memory formation intervention Fixed candidate pool or fixed experience content Selection and Integration Sequence Sensitivity All memory architectures share the same mechanism

The three types of research tasks, model starting points, feedback sources and evaluation units are different; they do not share training division or the same reasoning budget, and cannot form a performance ranking. The first category focuses on the quality of natural execution, and the latter two types of set controlled attacks; the evaluation calls paid, the update calls and retrieval costs cannot be covered up by "all a task".

At present, the more secure conclusion is that the value and reliability of long-term memory need to be measured separately. It is useful to help the system complete the tasks that were not originally done, which means that it can be used to locate, restrict the spread and restore after the error occurs, and then further explain the reliability. Existing studies provide different local evidence for the two, and it has not been demonstrated that a management strategy is optimal in any task flow for a long time.

Next Step: Let the Competitive Interpretation Make Different Forecasts

The following are research design recommendations, which have not been implemented yet. You can start with a small simulation task with a clear state and standard results, fixed models, hints, task flow, retrieval quantity and contextual budget, and save each read, write and derived relationships. Use natural errors first, and then add controlled pollution separately to avoid mixing the two sources into one total score.

The first set of comparisons does not save new records, saves all, verified, and retains random controls for budgets such as adding screening groups. If the validation screening is better than the random retention, there is more reason to put the benefits to quality, rather than just shrink the memory bank. The experiments with the validator standard results should be clearly marked as the ideal conditions, and then the degradation should be evaluated with noise feedback.

The second group of fixed retrieved record IDs only replaces the incorrect content; sets another fixed content and changes the group of applicable condition labels. If the content is replaced and the condition tag is invalid, the priority is to support the wrong interpretation of the content; if the condition tag is valid, the experience of using the model according to the scope should continue to be checked. Without improvement, it cannot prove that memory has no impact, and it may be that there is no coverage derived record.

The third group was split from the same pollution after the snapshot, one group only revised the original items, the other group was withdrawn from the derived abstract at the same time, and the other group reconstructed the clean memory. Errors are reported separately, the correct knowledge loss and the number of tasks required to recover, and repeat in different random task sequences. Only by using "deletion success" and "behavioral recovery" as two end points, we can recognize that only the interface is repaired.

The time change test should also be left in advance: the rules are legally updated at a certain point, and the old version only applies to the past. If the system is still mechanically executing old rules, this is an outdated information processing failure; if it just does not retrieve the new version, it is another responsibility. Different diagnosis of the two is more valuable than adding a mixed long-term memory score.

Summary

Long-term memory will not automatically turn experience into knowledge, and it will not necessarily allow mistakes to accumulate infinitely. When reading the paper, the most worthy of questioning is: what is the basis for recording, why the current task is used, how to return the execution result, and which dungeons will affect the future after correction. Only by separating these links can we move from the total score of "memory effective" to the evidence of reliable and rebuttable reliability.

References

  1. Zidi Xiong、Yuping Lin、Wenya Xie、Pengfei He、Zirui Liu、Jiliang Tang、Himabindu Lakkaraju、Zhen Xiang. How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior. ACL 2026, formal paper. formal page; Full text.
  2. Xuanze Chen、Xukang Xie、Wentao Fu、Jiajun Zhou、Shanqing Yu、Qi Xuan. MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair. 2026, arXiv V1 preprint, no official employment is confirmed. Version history; Full text.
  3. Haowen Pan、Pengyang Shao、Yunshan Ma、Yanzheng Jin、Naixin Zhai、Xiang Wang、Xun Yang. MemForm: Benchmarking Memory Formation Poisoning in LLM Agents. 2026, arXiv V1 preprint, no official employment is confirmed. Version history; Full text.

Apps by the author

This is the author's own product promotion. In addition to writing a technical blog, I also independently developed two daily gadgets, welcome to understand according to your needs:

  • Daily Expense Journal: Record the daily income and expenditure, arrange the budget and savings goals, so that the whereabouts of each money is more clear. Learn about it in the App Store.
  • One Small Action: Choose small actions according to the current time and energy, record the feelings after completion, and leave a little clue for the daily review. Learn about it in the App Store.

Find more articles and projects on my website: zffjzz.top. Your feedback helps me improve these tools.