Project Overview

overview

The goal of this project is to build a fully autonomous, end-to-end system for generating research survey papers. This is with the ultimate goal of creating hundreds or thousands of surveys with varying subjects and granularity, giving up-to-date information for a researcher that wants to learn about any field.

Input: a research area or topic $A$ plus 3–5 papers $P_1$–$P_5$.
Output: a research survey paper $S$ for area $A$.

The generated survey should:

• conform to the structure, style, and formatting of published surveys
• help readers understand the field, select appropriate methods, identify open problems, and avoid duplicating existing work

The solution pipeline is:

1. collect papers from sources such as arXiv and Semantic Scholar
2. generate paper summaries
3. determine a paper outline
4. synthesize an area survey

core challenges

• Hallucinations and citations: LLMs still struggle to cite accurately and depict statistical figures, sometimes making them up. Both are particularly relevant to survey generation.
• Building a field model: writing a high quality survey requires writers to have a strong field model—possessing an accurate, organized mental representation of a field to understand not just what exists, but how the field fits together and behaves. Helping an LLM build this field model is a nontrivial task:
  • Small context window: human experts spend immeasurable amounts of time consuming written research, and time thinking about the field. This is the equivalent of an immeasurable number of reasoning and retrieval tokens. An LLM needs to construct this field model with far smaller information intake and reasoning budget within a comparatively tiny context window.
  • Imitating a field model: through observation, an LLM doesn’t build an actual strong field model, and instead is only able to imitate the impression of one. It’s the equivalent of a smart, experienced researcher trying to write a survey for a field that’s adjacent to their expertise, with very little time to research and fully grasp it.
• Evaluation: survey quality is subjective, and it is harder to assess whether output grasps the larger context without preexisting knowledge.
• Avoiding a human upper bound: avoid making humans the upper-bound of survey quality. This is a direct challenge for evaluation and also agent creativity, as an LLM can anchor onto existing human surveys for both tasks.
• Figures and tables: creating figures and tables is challenging because their non-textual nature makes them a non-native language for LLMs.

completed work

paper collection

The paper-collection subsystem takes a research area $A$ and an optional description $A_d$ and returns a set of papers $P$ where each paper’s main contribution is in scope for $A$ or a direct, first-order component $A_c$.

Paper collection is broadly a two-stage system:

1. Stage 1 is focused on recall: find a broad set of candidates that are plausibly in scope for the area.
2. Stage 2 is focused on precision: filter those candidates and identify papers whose main contribution is in scope.

More details about paper collection

outline generation

The outline subsystem takes the research area $A$ and the in-scope papers $P$ and returns a survey outline $O$.

Outline generation is broadly a two-part process:

1. Evidence gathering: collect grounded signals about the area’s vocabulary, mechanisms, relationships, tensions, and missing threads using condensed paper tags and web research.
2. Outline design: select an organizing lens, refine its structure, review and revise the outline, and finalize it.

More details about outline generation