University of Texas paper finds that handing small memory decisions to a fast scoring model let an AI memory system build its memory of long conversations 6.6 times faster

Many memory systems ask an LLM to make small choices, like what to link or when to stop searching.

Generating text for each choice is slow, and those calls pile up during memory building and search.

Jev-Mem gives those choices to a fast scorer, Jev, that returns probabilities in batches instead of text. The LLM is called only at the end, to read the chosen memories and answer.

On the LoCoMo long-conversation benchmark, it got a 0.777 answer score versus 0.700 for the best baseline. It built memory in 158 seconds versus 1,044 for the fastest rival and answered in 0.93 seconds versus 1.47.

If your memory pipeline calls an LLM for routing, scoring, or stopping, try a fast classifier there and save the LLM for answers.