Paper Collection Overview

problem def

input: an existing Paper Corpus $P$ plus an Area Brief $B = (A, A_{d})$ , containing the area’s canonical name $A$ and immutable positive definition $A_{d}$

output: an Area-specific paper set $P_{A} \subseteq P$ where each paper $p$ ‘s main contribution is in-scope to $A$ , or directly advances a first-order component $A_{c}$ in a way that transfers to its role within $A$

solution

paper collection is broadly a two-stage system:

stage 1: focused on recall; finding a broad set of paper candidates that are plausibly in-scope for the area.

stage 2: focused on precision; filtering candidates, directly judging the survivors, and publishing papers whose main contribution is in scope for the area or one of its first-order components.

how each stage works

candidate generation

Goal: cover the central literature and harder-to-find parts of the area well enough to support a high-recall final collection.

initial retrieval

Goal: generate a high-quality initial Candidate Set using complementary lexical and semantic queries.

Have an agent construct the initial Area Card $C_{1}$ from $B$ , containing:
- possible first-order components
- lexical and semantic retrieval queries
- exclusions and suspected coverage gaps
Run three Area Card revisions, $C_{1}, C_{2}, C_{3}$ .
- execute all lexical and semantic queries in $C_{i}$
- examine coverage, overlap, score distributions, relevance samples, and the retrieved literature
- inspect ambiguous papers when needed
- revise the components, queries, and semantic cutoffs based on the evidence
Recompute the Candidate Set $R_{i}$ after each revision. Retiring or changing a query can remove papers supported only by its earlier form. The third revision produces the initial Candidate Set $R_{3}$ .

citation expansion

Goal: use citation expansion to find relevant papers missed by lexical and semantic retrieval.

Build a citation-neighborhood graph over $R_{3}$ .
Select a structurally diverse set of expansion seeds across citation communities.
Follow both references and citing papers to produce two bounded expansion layers, $E_{1}$ and $E_{2}$ .
Choose the expansion budget per Area. The same budget can produce similarly large candidate pools for very different source-set sizes.
Combine the initial Candidate Set and both expansion layers into $R = R_{3} \cup E_{1} \cup E_{2}$ .

contribution filtering

Goal: identify and retain papers whose main scholarly contribution is in scope for the Area or one of its first-order components.

cheap filtering

Goal: reduce the number of papers that require direct contribution judgment while preserving relevant papers.

Sample contribution judgments from across the candidate population.
Use those judgments to train separate lexical and embedding-based relevance models.
Combine the models into a single relevance score.
Calibrate a conservative cutoff on a separate judgment sample.
Apply the cutoff to produce the filtered population $R_{F} \subseteq R$ .

contribution judgment

Goal: determine whether each remaining paper’s main contribution belongs in the Area-specific collection.

Judge every remaining paper $p \in R_{F}$ against $B$ .
Treat a paper as in scope when it directly advances the Area or a transferable first-order component.
Attach Contribution Tags that describe the in-scope contributions.
Remove papers judged out of scope and keep unresolved cases explicit, producing $P_{A} \subseteq R_{F}$ .

evaluation

we have a Silver Truth, being the generated topic / primary-topic for papers in the Paper Corpus.

let:

$R_{s}$ be the retrieved or retained population at a named stage $s$
$L$ be the papers with the evaluated Silver field populated
$S \subseteq L$ be the Silver-positive papers labeled with the area

then:

Silver Recall: $∣ R_{s} \cap S ∣/∣ S ∣$ , the fraction of Silver-positive papers in the declared evaluation population that were retrieved
Silver Precision: $∣ R_{s} \cap S ∣/∣ R_{s} \cap L ∣$ , the fraction of retrieved, labeled papers that are Silver-positive

papers in $R_{s} ∖ L$ are reported separately as unevaluated and are not included in the Silver-precision denominator. the metrics can be calculated using either primary_topic = area or area in topics as the definition of $S$ , with $L$ defined from the corresponding field.

however, the Silver Truth has flaws:

many papers do not have the evaluated field populated
labels can be wrong or use a broader notion of topical relevance than the final main-contribution judgment
papers discovered only in the external citation graph are outside the metric until they resolve to the Paper Corpus

future directions

the main open directions are:

Collection and taxonomy as a shared loop. Generate an initial collection and taxonomy together, use the taxonomy to reveal undercovered components, collect more papers for those components, and revise both until they stabilize. The hard part is deciding whether an apparent component reflects the field or only the structure of the current paper set.
Adaptive, gap-driven collection. Let evidence about coverage gaps determine what the system searches next, how deeply it expands citations, and when further collection is no longer worth the cost. This would replace the current fixed number of retrieval rounds and manually chosen expansion budgets.
Relevance, importance, and representation. Go beyond deciding whether a paper is in scope. Build collections that also represent the major parts of the field, include important older and emerging work, and avoid allowing citation count or one dense research community to dominate the result.

system diagram

Interactive HTML diagram

paper collection final system diagram

AI-Automated Surveys

Explorer