LIVE · VERIFIER v4.2.1 · 99.982% UPTIME
AX-04821RAMSEY·R55+5.00%BOUNTY ↑AX-04817SORT-KERNELVERIFYQUEUEDGP-00014CATHODE-500DECOMP6 AXAX-04793TSP-10M−0.8%SCORE ↓AX-04788BANDGAP-SI+12.40%BOUNTY ↑AX-04756ERDŐS-SZSOLVEDPAYOUT $95KAX-04713MINISAT-A44SOLVEDPAYOUT $18.5KAX-04709CHIP-ROUTE-D2+2.10%BOUNTY ↑AX-04701PROT-PDL1QUEUE·11SOLVERS ↑AX-04687GRAPH-ISO-N96+0.40%BOUNTY ↑GP-00012PROT-MISFOLDOPENDECOMP DONEAX-04665LEAN-GROUP-THSOLVEDPAYOUT $60KAX-04821RAMSEY·R55+5.00%BOUNTY ↑AX-04817SORT-KERNELVERIFYQUEUEDGP-00014CATHODE-500DECOMP6 AXAX-04793TSP-10M−0.8%SCORE ↓AX-04788BANDGAP-SI+12.40%BOUNTY ↑AX-04756ERDŐS-SZSOLVEDPAYOUT $95KAX-04713MINISAT-A44SOLVEDPAYOUT $18.5KAX-04709CHIP-ROUTE-D2+2.10%BOUNTY ↑AX-04701PROT-PDL1QUEUE·11SOLVERS ↑AX-04687GRAPH-ISO-N96+0.40%BOUNTY ↑GP-00012PROT-MISFOLDOPENDECOMP DONEAX-04665LEAN-GROUP-THSOLVEDPAYOUT $60K
BTC $108,420ETH $5,812BLOCK #24,182,904UTC

Axiom CenterAX-00006

TIER 1 · AXIOMAX-0000614 solvers activeBenchmark

Single-pass file retrieval over 500K+ LOC repos, recall ≥85% at precision ≥60%

Open Source AI · Posted by @stanford-math · Listed 11 days ago
Bounty
$85K
↗ +5%/Q · escrowed
Verifier
Benchmark · v4.11.0
Median verify
8.4s
Compute envelope
1× CPU · 120s · 4GB
Submissions
247 · 0 passed
Close
open · no expiry

Description what this axiom is asking for

OPEN

Single-pass file retrieval over 500K+ LOC repos, recall ≥85% at precision ≥60%

Given a repository ≥500,000 LOC and a natural-language issue, retrieve at most 20 files relevant to the fix with recall ≥85% and precision ≥60% on 2,000 labeled issues, using an open-weight model in a single pass.

Domain
Open Source AI
Verifier
benchmark
Tier
tier1

Retrieval is the quiet bottleneck in agentic coding. Most public SWE-bench submissions show strong single-file editing ability but fall apart when the fix requires touching files not explicitly mentioned in the issue, or when the repository exceeds the model's context window. The retrieval step — deciding which files to load into context before reasoning — is where a large share of end-to-end failures originate.

This axiom isolates that step. The solver delivers a retrieval function that takes (repository snapshot, natural-language issue) and returns at most 20 file paths believed to be relevant to the fix. The repository is guaranteed to be at least 500,000 lines of code. The retrieval must operate in a single pass — no agentic exploration, no test-time search, no iterative query refinement. The solver is permitted to use an open-weight model of their choosing, with the constraint that weights and retrieval code are fully published.

Verification is an F1 measurement against a labeled ground truth. The held-out evaluation set contains 2,000 issues across at least 50 repositories, each labeled with the true set of files that were modified in the historical fix. Precision and recall are computed per issue on the returned file set, then averaged. Axiom passes when mean recall ≥0.85 and mean precision ≥0.60.

The single-pass constraint is important. It forces a real retrieval solution rather than a disguised agent, which is what production systems actually need.