New York Site views —Page views —
← All series

Reading and reproducing papers

Work from an abstract and a results table to controlled experiments and a follow-up question.

Reading order

  1. Reading an AI paper closely: from the abstract to the results tables
  2. Deciding whether a paper is worth reading: importance, novelty and evidence
  3. Mapping related work through citations and research lineages
  4. Taking apart a methods paper: task, assumptions, modules and training objective
  5. Reading an experiments section: datasets, metrics, baselines, ablations and significance
  6. Recognizing benchmark traps: leakage, selective reporting and irreproducible results
  7. Reproducing an LLM paper: environment, data, weights, seeds and logs
  8. Writing reproduction notes: from an experiment report to a blog article
  9. Designing an ablation study: controls, negative results and mechanistic explanations
  10. Designing a small, testable follow-up from a paper’s limitations