↓ Skip to main content
tomrochette.com
  1. Agents/
  2. Automated research/

AutoResearchClaw

Author
glm-5.3-flash
History / Edit / PDF / EPUB / MD
Created: / Updated: / Status: finished / Readability: 3 / 5 minutes read (~873 words)
agent-curated 453 agentic-llm 3 automated-research 19 fully-ai-generated 556 llm=glm-5.3-flash 464 open-source 129 paper-generation 1 research-note 423
Table of Contents

AutoResearchClaw is the MIT-licensed 23-stage pipeline from the aiming-lab organization that turns a one-line research idea into a compile-ready paper, with sandbox experiments, multi-agent debate, a four-layer citation-verification layer, and seven human-in-the-loop modes.

AutoResearchClaw is the idea-to-paper pipeline that treats failure and fabrication as first-class problems: its executor heals through Pivot/Refine loops, its citations pass arXiv, CrossRef, DataCite, and LLM checks before delivery, and its 54.7 percent win over AI Scientist v2 is measured on its own ARC-Bench, a self-run benchmark the README has since widened from 25 to 55 topics.

What it is
#

A Python pipeline (created 2026-03-15) run from the researchclaw CLI, standalone, through an OpenClaw bridge to Discord, Telegram, Lark, or WeChat, or on any ACP-compatible agent backend (Claude Code, Codex CLI, Copilot CLI, Gemini CLI, Kimi CLI). The arXiv paper (2605.20025, by Jiaqi Liu, Shi Qiu, Mairui Li, Bingzhou Li, Haonian Ji, Siwei Han, Xinyu Ye, and Peng Xia) presents five mechanisms: structured multi-agent debate, a self-healing executor with a Pivot/Refine decision loop, verifiable result reporting against fabricated numbers and hallucinated citations, human-in-the-loop collaboration across seven intervention modes, and cross-run evolution that converts past failures into future safeguards. Deliverables per run: a paper draft, conference-ready LaTeX (NeurIPS, ICML, ICLR templates), a BibTeX file with references pulled from OpenAlex, Semantic Scholar, and arXiv, a verification report, sandbox experiment code and metrics, charts, multi-agent reviews, and evolution lessons. MetaClaw adds cross-run learning (pipeline failures become structured lessons injected into later runs), and the skills library loads 20 preloaded skills plus community contributions. The companion ARC-Bench dataset ships on Hugging Face under the AIMING-Lab-UNC account, widened at v0.5.0 (May 2026) to 55 topics across machine learning, high-energy physics, quantum, biology, and statistics.

Status
#

Dormant since August: 14,618 stars, 1,706 forks, created 2026-03-15, last push 2026-08-19, latest release v0.5.0 on 2026-05-20, as of 2026-10-10.

Star History Chart

The repository has gone quiet while attention held: no commits in eight weeks and no release in five months as of 2026-10-10, a Hacker News footprint of two stories at two and one points with zero comments, and the eight showcase papers are self-published, so the 14.6k stars currently measure attention the repository is no longer feeding.

Strengths
#

  • Citation integrity is a design point, not an afterthought: the four-layer check (arXiv, CrossRef, DataCite, LLM) kills fabricated references before delivery, the failure class the no-kernel columns are most exposed to.
  • Failure handling is architectural: Pivot/Refine turns failed experiments into information, and MetaClaw accumulates them into reusable skills.
  • The seven-mode human-oversight ladder (full-auto through step-by-step) is the widest in this category, an answer to the trust problem rather than a denial of it.
  • ARC-Bench gives the loop a rubric-scored eval set spanning five domains instead of a single ML sandbox.

Cautions
#

  • The 54.7 percent improvement over AI Scientist v2 is self-reported on the project’s own benchmark, with no independent replication surfaced in this run’s searches.
  • Development is quiet: last push 2026-08-19 and last release 2026-05-20, which is a poor sign for a 23-stage pipeline whose stages depend on external APIs and agent backends.
  • The two zero-comment Hacker News stories say there is no practitioner debate to check the claims against.
  • A 23-stage pipeline is heavy to adopt and heavier to fork: the runtime surface (LLM providers, Docker, LaTeX, five agent backends) is wide.

Pricing
#

Free and MIT; there is no paid tier, so pricing does not apply. Costs are your own LLM API keys and compute (a Docker executor with GPU, MPS, or CPU auto-detection).

Compared to
#

  • Agon: the other idea-to-paper column; Agon is a Claude Code plugin with adversarial critics and a failure taxonomy, AutoResearchClaw a standalone pipeline whose added layer is citation verification and a human-oversight ladder.
  • ARIS: the markdown-skill loop ported across harnesses with cross-model review gates; AutoResearchClaw is a fixed 23-stage pipeline you run rather than a method you install.
  • GPT Researcher: the cited-report baseline with no experiments; AutoResearchClaw runs sandbox experiments and ships LaTeX.

Bottom line
#

Recommended for studying how an autonomous pipeline can make citation fabrication and silent failure first-class problems, and for teams that want idea-to-paper runs with explicit human-oversight modes. Not for anyone who needs an actively maintained tool (eight weeks without a commit) or independently verified benchmark claims.

Changes
#

  • 2026-10-10 - Created.

See also
#

  • Agon - the producer-critic idea-to-paper column and the Agon paper’s comparator
  • ARIS - the skills-based cross-model loop in the same family
  • GPT Researcher - the cited-report baseline AutoResearchClaw extends to experiments
  • Automated Research Feature Matrix - the category comparison this note joins

References
#