renee-jia/scholar-loop

About

read papers → find a gap → run real experiments → reflect → write & self-review

ScholarLoop runs the loop a PhD actually runs: it reads the literature, forms a grounded hypothesis, runs real ML experiments, scores them against a frozen ground-truth metric, learns from its failures, and drafts a peer-reviewed write-up — autonomously, with a deterministic harness that keeps the agents honest and impossible to reward-hack.

The LLM does only the open-ended reasoning. Everything checkable — search-space pruning, dedup, calibration, number-grounding, promotion gates — is deterministic, unit-tested code, and the metric is the only optimization target (no LLM-as-judge in the optimization loop).…

Excerpted from github.com/renee-jia/scholar-loop