Cover illustration

Date: 2026-07-01
Suitable for readers: postgraduate, scientific reader, AI engineer, research platform/experimental platform builder

Abstract

In the past year, the discussion of AI Research Agents has rapidly advanced from "Help me sum up the papers" to "can make assumptions, write code, run experiments, generate papers, and accept reviews". The key change in this direction is not that LLM will be written more like a paper, but the Research process is being split into a retrievable, executable, evaluable and traceable agent. Workflow: Literature discovery requires open search and chain of evidence; hypothesis generation needs to be aligned with existing knowledge graphs and experimental constraints; experimental design requires code, tools, instruments, and budget boundaries. The 2025-2026 papers and systems show that Research Agents can already form closed loops in several restricted scenarios, but there is still a clear distance from "reliable and autonomous scientists". The truly landable route is more like using AI as a set of audit collaborators in the scientific Research operating system, rather than making it replace PI or reviewers.

Contents

  • Research background: Why scientific research agents are becoming a system problem
  • Roadmap for the past year: from Deep Research to experimental closed loop
  • Key papers by research question
  • Comparing the methods
  • Technical trends
  • Implications for deployment
  • Limitations and open questions
  • Questions to watch
  • Summary
  • References

Research background: Why scientific research agents are becoming a system problem

The first-generation scientific research LLM applications mainly focus on reading and writing: abstract papers, generation reviews, help polishing, and interpretation formulas. Their value is intuitive, but the boundaries are also very clear: as long as there is no reliable retrieval, source document positioning, experimental records and human review, the output can easily become a text generation that "looks like research".

The new question of the Research Agent is: Can scientific Research activities be split into an executable link? Typical links include document discovery, problem decomposition, hypothesis generation, experimental design, code implementation, experiment execution, result analysis, visualization, paper writing, review feedback and the next round of iteration. Each step in this link is not a pure language task. It requires search engines, PDF parsers, code repositories, datasets, simulators, experimental platforms, notebooks, version control, log systems, budget control and permission systems to work together.

Therefore, the core change in 2025-2026 is that "research automation" is moving from Prompt Engineering to Infrastructure Engineering. An available scientific research agent is not only strong in model ability, but also has three types of infrastructure: first, external knowledge and evidence management, to ensure that each judgment can return to the paper, data or experimental log; Really run experiments, not fabricate experiments; third, evaluate and human review to ensure that the quality of research can be dismantled, quantified and reviewed.

Research Agent Life Cycle

Roadmap for the past year: from Deep Research to experimental closed loop

Two starting points from 2024 are PaperQA and The AI Scientist. PaperQA treats scientific question answering as a sequence of retrieval, screening, attribution of evidence, and answering. It introduces tasks such as LitQA that more closely resemble synthesizing findings from the literature. The AI Scientist links the stages of machine learning research: generating an idea, writing code, running experiments, plotting results, drafting a paper, and simulating peer review. Its significance lies in being the first to organize automated research into a systematic, reproducible workflow. The reliability of the papers it produces is a separate question.

In 2025, this route is clearly differentiated. OpenAI's Deep Research and Hugging Face's Open Deep Research stands for the Deep Data Research Agent: they emphasize long-term browsing, source references, cross-web and PDF synthesis. AI Scientist-v2 of Sakana AI continues to advance end-to-end Research automation, using Agentic Tree Search, Experiment Manager and VLM Reviewer Feedback charts and even submitted ICLR Workshop papers that are fully automatically generated. At the same time, benchmarks such as PaperBench, MLR-Bench, EXP-Bench, DeepResearch Bench, etc. began to dismantle "Looks like they can study" into a more diagnosable question: Will you find literature? Will the experiment be reproduced? Will the full experiment be performed? Will the result be forged?

In 2026, the trend is further extended toward both ends. One end is the benchmark for Discovery of complex literature such as AutoResearchBench, which directly evaluates the ability of agents to do deep/wide search in scientific literature; the other end is Qiushi Discovery Engine Long cycles on this kind of real physics platform are Autonomous Discovery. The latter combines LLM calls, tool calls, experimental notes, scripts, and instrument measurements to represent the scientific research agent moving from "writing a paper" to "interacting with the real world".

Key papers by research question

1. Literature reading and evidence synthesis: from RAG to Deep Research

The contribution of PaperQA is to separate scientific research from ordinary Q&A: the system needs to retrieve the full-text paper, judge the relevance of paragraphs, and provide Provenance in the answer. What it emphasizes is not "the answer is smoother", but the answer is traceable. For graduate students and engineering teams, this is very important: the first value of the scientific research agent is not to draw conclusions for you, but to shorten the time for "finding reliable evidence".

OpenAI released Deep Research in February 2025 and described it as an agent capability for multi-step internet research. It could find, analyze, and synthesize many online sources in 5 to 30 minutes. Later updates added MCP and app connections, restrictions to trusted sites, and live progress tracking. Hugging Face’s Open Deep Research shows how the open-source community used CodeAgent, search tools, and the GAIA benchmark to reproduce a similar approach. A research retrieval agent needs more than a search API. It needs to plan, backtrack, assess source quality, deduplicate citations, and resume interrupted tasks.

The 2026 AutoResearchBench further states that scientific literature discovery is far more difficult than general web browsing. It distinguishes between Deep Research and Wide Research: the former requires a multistep probe to find the target paper, and the latter requires the collection of a set of papers that meet the conditions as completely as possible. Even if the strong model still has only single-digit performance on this benchmark, this means that "Research retrieval" has not been resolved by the general browsing benchmark.

2. End-to-End ML Research Agent: From AI Scientist to AI Scientist-v2

The AI Scientist framework covers IDEA, experiments, charts, papers, and automatic reviews. Its radicality is to regard "open scientific discovery" as a cycle: generate new ideas, implement and test, write a paper, and then use The review feedback to advance The next round. 2025 AI Scientist-v2 attempts to reduce The dependency of v1 on human template code, introduces Progressive Agentic Tree Search and Experiment Manager Agent, and puts VLM feedback is used for chart and paper presentation improvements.

But these systems are most easily misread. AI Scientist-v2's "Complete AI Generation Paper Reviewed by Workshop" is an important milestone, but it cannot be equivalent to the general level of scientific research to human level. It is more suitable as a stress test as a system capability boundary: when the Agent can complete the output of the paper form in the limited ML sub-field, we must more carefully discuss the authenticity of the experiment, the negative result report, the code reproducible, the review pollution and the signature specification.

3. Study Agent Evaluation: From Results Score to Process Diagnostics

PaperBench makes the agent reproduce 20 ICML 2024 Spotlight/Oral papers, and split the task into 8,316 evaluable molecular tasks. Its meaning is to dismantle "reproducing a paper" into an inspectable structure: understanding contributions, building code, executing experiments, and achieving results. The paper reports the best test for Agents with an average reproducing score of 21.0% and does not exceed the human ML PhD baseline.

MLR-Bench focuses on open ML research, including 201 research tasks from NeurIPS, ICLR, ICML Workshop, covering Idea, proposal, experimental, paper writinging four stages. It is worth noting that it reports that the current Coding Agents often produce fabricated or invalidated experimental results. This conclusion is very realistic for scientific research automation: the most dangerous failure of Agent is not "can't do it", but outputs a set of seemingly reasonable but not supported by real experiments.

EXP-Bench is more direct: given research questions and incomplete starter code, the agent is required to design, implement, execute and analyze complete experiments. The success rate of the complete executable experiment of the paper report is very low, indicating that there is still a huge fault from the "Writing Research Plan" to the "Real Running Experiment". For the engineering team, this means that the scientific research agent platform must give priority to the construction of Execution Trace, Sandbox, data version, experimental queue and automatic acceptance, rather than just optimizing the generated text.

4. Scientific Experiment and Physics World: From Tool Enhancement to Real Platform

The 2023 work of ChemCrow and Boiko demonstrates the early possibilities of LLM's combination with chemical tools and robot experiments. ChemCrow integrates 18 expert tools for organic synthesis, drug discovery and material design; Boiko and other work demonstrate the ability of LLM Agent to design, plan and execute several scientific experiments, and clearly discuss safety risks.

The Qishi Discovery Engine in 2026 pushes this direction to a real optical platform. The paper describes the system using nonlinear research stage, Meta-Trace memory and double-layer architecture to maintain a stable research trajectory in long-term exploration, and record a large number of LLM calls, tool calls, research notes and scripts. The most worthy of attention here are "long-term experimental memory" and "closed loop of physical measurement": when the Agent has to face real instruments and noise data, a simple chat memory or vector library is not enough to support the scientific research responsibility chain.

Google DeepMind's AlphaEvolve, although more biased to discover the Agent, it provides another clear direction: when the candidate solution can be expressed and automatically validated by the program, the agent New algorithms can be discovered by generating, evaluating, and evolving searches. The easiest scenarios for scientific research agents to land are often not open-ended "doing science", but scientific computing, algorithm design, configuration search and experimental optimization of a clear validator.

Comparing the methods

Direction Representative system/paper main input main output critical ability Main risk
Research Q&A/RAG PaperQA、SQuAI Full paper, problem Reply with reference Retrieval, Evidence Screening, Claim Grounding Retrieval omissions, references do not support conclusions
in-depth data research Deep Research、Open Deep Research、DeepResearch Bench Open web pages, PDFs, files Long reports, references, forms Multistep search, synthesis, backtracking Unstable source quality, it is difficult to review long links
Automated ML Research The AI Scientist、AI Scientist-v2、AI-Researcher Research topic, code environment Code, Experiment, Paper Draft Idea Generation, Experimental Execution, Essay Writing Forgery experiments, overfitting review, low novely
Reproduction/Experimental Benchmarks PaperBench、MLR-Bench、EXP-Bench Papers, Starter Code, Task Rubrics Reproduction results, experimental report Process Scoring, Executable Assessment Benchmark Pollution and evaluation costs are high
Real Science Platform ChemCrow、Qiushi Discovery Engine Tools, Instruments, Experimental Constraints Experimental planning, measurement, discovery candidates Tool calls, long-term memory, physical closed loop Safety, cost, error experiments, attribution of responsibility
algorithmic discovery AlphaEvolve program skeleton, validator New algorithm/code Search, evolution, automatic verification Only suitable for formalized goals

Technical trends

First, the scientific research agent is shifting from "single answer" to "long-range task state machine". Task status includes read papers, candidate assumptions, experimental configuration, failure attempts, tool results, human feedback, and verifiable evidence. The general dialogue context cannot stably carry this information, and will rely more on structured memory, progress graph and event log in the future.

Second, the search strategy is moving from Top-K Retrieval to Deliberate Search. Scientific research questions often do not have a definite answer. It is necessary to first locate the field, expand the concept of synonyms, identify key papers, track the citation chain, and exclude irrelevant work, and then decide whether it is comprehensive enough. Benchmarks such as AutoResearchBench will promote the Agent to learn to "find the whole" rather than just "find the right".

Third, experimental implementation is becoming a watershed. There are many systems that can write proposals, and there are many systems that can fully run through the experiment and report the reasons for failure. EXP-Bench and PaperBench together point to a conclusion: the core bottleneck of scientific research agents is not verbal expression, but executability, environmental robustness and results authenticity.

Fourth, the validator is more important than the generator. In algorithm discovery, scientific calculation, automatic parameter adjustment and simulation experiments, as long as there is an objective validator, the search of the agent can be constrained. Open research without validators must introduce human review, statistical testing, reproducing experiments and audit logs.

Fifth, the security issues of scientific research agents will be sharper than general office agents. It may access expensive instruments, dangerous reagents, limited data, unpublished papers, patented materials and dual-use knowledge. Permissions, budgets, data isolation, approval flows, and operational red lines will become part of the infrastructure.

Implications for deployment

If you want to implement a scientific research agent in a laboratory or enterprise research department, it is recommended to start with "auditable assistance" instead of pursuing automatic scientific research.

The first layer is the literature console: access to arXiv, PubMed, Semantic Scholar, OpenReview, ACL Anthology, internal document library and projects Wiki. The output must contain references, paragraph evidence, retrieval time, exclusion criteria, and uncovered. The metric here is not how beautiful the answer is, but the time for retrieving recalls, referencing Accuracy and manual review time is reduced.

The second layer is the experimental Copilot: let the Agent generate the experimental plan, configuration files, scripts, and Sanity Check, but all executions are run in a controlled Sandbox, CI, or experimental queue. Preserve code version, data version, random seed, hardware environment, log, failure stack, and result summary for each run.

The third layer is the memory of the research project: writing hypotheses, experiments, results, rebuttals, manual comments and the next step plan to write into a structured system, rather than scattered in chat records. In the long run, this is more important than the vector library, because scientific research memory needs to be governed: which conclusions have been verified? Which are overturned? Which are just hypotheses? Which data can't be sent out?

The fourth level is the boundaries of human review and authority: let PI, doctoral students, engineers, and compliance personnel have different approval rights at different stages. Agents can propose candidate assumptions, but cannot submit papers by default; they can queue simulation experiments, but they cannot operate high-risk instruments by default; they can generate reviews, but cannot hide low Confidence sources.

Limitations and open questions

First, Novelty is difficult to evaluate automatically. An agent may generate a seemingly novel combination, but it is actually just changing terms, missing key prior work, or claiming improvements in incomparable experiments. The current Automated Reviewer saves screening costs, but cannot replace expert judgments in the domain.

Second, forgery or invalidity of experimental results is the core risk. MLR-Bench clearly states that the current Coding agents will generate fabricated or invalidated experimental results; EXP-Bench also shows that the full experimental execution ability is still weak. For scientific research scenarios, such failures are more serious than ordinary Hallucination, as it may contaminate project decisions.

Third, Benchmark is easily overfitting. PaperBench, MLR-Bench, and EXP-Bench make reviews more specific, but public tasks, public papers, and common code repositories may all go into model training or agent scaffold tuning. In the future, the environment is required to freeze the environment, the private Test Set, process audit and expert review.

Fourth, the signature, responsibilities and ethics of automatic scientific research have not been stable. If an agent generates an Idea, performs an experiment and writes a first draft, who is responsible for the error? Who is eligible for signature? How to disclose the degree of AI participation? These questions are not just policy issues, but also affect system design: without logs and Provenance, you can't answer the responsibility question.

Questions to watch

  1. Will the scientific research agent have a standardized Provenance schema, which will record the paper evidence, code version, data version, experimental log and manual decision-making?
  2. Will the literature find that the benchmark will advance from "finding a target paper" to a "constructive reviewable review", and ask to cover negative results and contradictory evidence?
  3. Can the automatic experimental platform form a general interface, so that the Agent can submit controlled experimental tasks without touching dangerous operations?
  4. Will automatic review and automatic research contaminate each other: Agent optimizes writing in order to cater to LLM Reviewer, rather than optimize the quality of science?
  5. When multiple agents are doing research at the same time, how to prevent repeated exploration, reference loops, incorrect consensus and resource waste?

Summary

The short-term value of AI Research Agents is not to replace scientists, but to reconstruct scientific Research workflows: make literature retrieval more thoroughly, make experimental plans more executable, make failure records more complete, and make the chain of evidence more traceable. Research from 2025-2026 shows that end-to-end scientific automation has moved from proof to real Benchmark and the physical platform, but the strongest signal comes from failure cases: the literature is incomplete, the experiment can't run, the results may be fabricated, and the novely is difficult to judge.

Therefore, the next stage of winning and losing is not just a stronger model, but a scientific research infrastructure: retrieval, memory, execution, verification, authority, log and review. Who can turn the Agent into a manageable research collaboration system, who is more likely to really improve research productivity.

References

Search date: 2026-07-01. The following links give priority to the use of the paper page, official blog, project homepage or code repository; some 2026 arXiv papers are still pre-printed, and the conclusion needs to be continued in the peer review version.

  1. Jakub Lala et al., “PaperQA: Retrieval-Augmented Generative Agent for Scientific Research,” arXiv:2312.07559, 2023. https://arxiv.org/abs/2312.07559
  2. Chris Lu et al., “The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery,” arXiv:2408.06292, 2024. https://arxiv.org/abs/2408.06292
  3. Yutaro Yamada et al., “The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search,” arXiv:2504.08066, 2025. https://arxiv.org/abs/2504.08066
  4. OpenAI, “Introducing deep research,” 2025-02-02; page includes 2025 and 2026 updates. https://openai.com/index/introducing-deep-research/
  5. Hugging Face, “Open-source DeepResearch - Freeing our search agents,” 2025. https://huggingface.co/blog/open-deep-research
  6. Giulio Starace et al., “PaperBench: Evaluating AI's Ability to Replicate AI Research,” arXiv:2504.01848, 2025. https://arxiv.org/abs/2504.01848
  7. Hui Chen et al., “MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research,” arXiv:2505.19955, 2025. https://arxiv.org/abs/2505.19955
  8. Patrick Tser Jern Kon et al., “EXP-Bench: Can AI Conduct AI Research Experiments?” arXiv:2505.24785, 2025. https://arxiv.org/abs/2505.24785
  9. Mingxuan Du et al., “DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents,” arXiv:2506.11763, 2025. https://arxiv.org/abs/2506.11763
  10. FutureSearch et al., “Deep Research Bench: Evaluating AI Web Research Agents,” arXiv:2506.06287, 2025. https://arxiv.org/abs/2506.06287
  11. Lei Xiong et al., “AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery,” arXiv:2604.25256, 2026. https://arxiv.org/abs/2604.25256
  12. Shuxing Yang et al., “End-to-end autonomous scientific discovery on a real optical platform,” arXiv:2604.27092, 2026. https://arxiv.org/abs/2604.27092
  13. Andres M. Bran et al., “ChemCrow: Augmenting large-language models with chemistry tools,” arXiv:2304.05376, 2023. https://arxiv.org/abs/2304.05376
  14. Daniil A. Boiko et al., “Emergent autonomous scientific research capabilities of large language models,” arXiv:2304.05332, 2023. https://arxiv.org/abs/2304.05332
  15. Google DeepMind, “AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms,” 2025. https://deepmind.google/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/