RepoCortex
Plate iA field manual for one repository

A record of
what it learns.

Find the function behind a bug report. Measured on 319 held-out Django fixes after learning from 1281 earlier fixes.

Right file, first result
44.51%from 41.07% at step 0
Right function, top 10
48.28%from 45.14% at step 0
What changed in the graph

Amendments to the graph

Totals across the training pass
Measured operator counts
OperatorWhat changesCount
ReweightStrengthen useful paths; weaken misleading ones.3408
Path compressionA direct road for a repeated detour.535
Gap ingestionBring missing code into reach.1667
Merge / splitResolve shared names and separate meanings.33
What the benchmark measures

On 319 chronological fix rows, function Acc@10 is 41.69% for dense + BM25 without graph, 45.14% for the frozen graph, and 48.28% after learning. All gold functions must appear in the top ten.

This is a retrospective replay on one Django snapshot. Three test rows share tickets with training. Excluding them leaves 316 rows: 42.09%, 45.57%, and 48.73%, respectively. The snapshot includes later fixes; these are offline results, not live request measurements.

Read the evaluation audit · Download the reproduced comparison

Does learning work beyond Django?

With unchanged ranking settings, HTTPX improves from 14/33 successful held-out reports with the frozen graph to 15/33 after learning, following 51 training reports. Flask stays at 4/7 after 13 training reports.

Two further datasets were frozen before scoring. Click uses 44 training reports and 37 held out: text-only search finds 18/37, while frozen and learned graphs both find 21/37. Werkzeug uses 26 training reports and 16 held out: text-only finds 9/16, frozen finds 8/16, and learned finds 9/16.

Graph structure helps Click at top ten, but learning adds no further successes and lowers MRR. Werkzeug recovers one report through learning without beating text-only search. The learning-gain intervals include zero. These retrospective offline checks do not establish reliable generalization.

Read all four repository checks and memory controls →

Prospective Starlette confirmation

Preregistered before scoring

The fixed study learned from 41 chronological reports, then evaluated the same 24 untouched holdout reports at every checkpoint.

On the training stream, flat fix memory found 31.7% and the learned graph found 34.1%—a directional secondary result. The registered holdout comparison tied at 54.2%, so this study demonstrated no learning advantage.

Study summary → · Raw result →

All baselines, including dense + BM25 without graph
At the end of the pass / held-out accuracy
MethodFile Acc@1Function Acc@10
RepoCortex44.51%48.28%
Dense + BM25 (no graph)39.18%41.69%
Dense vectors35.74%37.3%
Frozen graph41.07%45.14%
BM2534.48%32.92%