A record of
what it learns.
Find the function behind a bug report. Measured on 319 held-out Django fixes after learning from 1281 earlier fixes.
- Right file, first result
- 44.51%from 41.07% at step 0
- Right function, top 10
- 48.28%from 45.14% at step 0
One fix, remembered.
HydraDB feedback journalA reviewed fix for Django #37442 changed the confirmed function's rank, then a fresh process reconstructed the same learned result from HydraDB.
- Before feedback
- 02Deserializer.__init__
- After feedback
- 01one journal event written
- After restart
- 01one event replayed
This verifies a durable update on the same report, not accuracy on unseen bugs. The base runtime hashes and graph counts stayed unchanged. Inspect the raw receipt →
What changed in the graph
Amendments to the graph
Totals across the training pass| Operator | What changes | Count |
|---|---|---|
| Reweight | Strengthen useful paths; weaken misleading ones. | 3408 |
| Path compression | A direct road for a repeated detour. | 535 |
| Gap ingestion | Bring missing code into reach. | 1667 |
| Merge / split | Resolve shared names and separate meanings. | 33 |
What the benchmark measures
On 319 chronological fix rows, function Acc@10 is 41.69% for dense + BM25 without graph, 45.14% for the frozen graph, and 48.28% after learning. All gold functions must appear in the top ten.
This is a retrospective replay on one Django snapshot. Three test rows share tickets with training. Excluding them leaves 316 rows: 42.09%, 45.57%, and 48.73%, respectively. The snapshot includes later fixes; these are offline results, not live request measurements.
Read the evaluation audit · Download the reproduced comparison
Does learning work beyond Django?
With unchanged ranking settings, HTTPX improves from 14/33 successful held-out reports with the frozen graph to 15/33 after learning, following 51 training reports. Flask stays at 4/7 after 13 training reports.
Two further datasets were frozen before scoring. Click uses 44 training reports and 37 held out: text-only search finds 18/37, while frozen and learned graphs both find 21/37. Werkzeug uses 26 training reports and 16 held out: text-only finds 9/16, frozen finds 8/16, and learned finds 9/16.
Graph structure helps Click at top ten, but learning adds no further successes and lowers MRR. Werkzeug recovers one report through learning without beating text-only search. The learning-gain intervals include zero. These retrospective offline checks do not establish reliable generalization.
Prospective Starlette confirmation
Preregistered before scoringThe fixed study learned from 41 chronological reports, then evaluated the same 24 untouched holdout reports at every checkpoint.
On the training stream, flat fix memory found 31.7% and the learned graph found 34.1%—a directional secondary result. The registered holdout comparison tied at 54.2%, so this study demonstrated no learning advantage.
All baselines, including dense + BM25 without graph
| Method | File Acc@1 | Function Acc@10 |
|---|---|---|
| RepoCortex | 44.51% | 48.28% |
| Dense + BM25 (no graph) | 39.18% | 41.69% |
| Dense vectors | 35.74% | 37.3% |
| Frozen graph | 41.07% | 45.14% |
| BM25 | 34.48% | 32.92% |