TL;DR — MemER scene-mem v2 LoRA, held-out val: final-query 0.0% (0/2); true compressed-memory recall (chunked) 0.0%
0.0%
PRIMARY final-query
0/2
5.0%
memory-turn match
3/60
0.0%
CHUNKED (true mem)
0/2
0.0%
NOT-chunked floor
0/0
Eval set (honest split)
Held-out val only: 2 test samples across
1 scenarios (split-val total
57). Train leak:
NONE. MemER did NOT train on these scenarios.
base = gemini-2.5-flash
adapter = None
max_new_tokens = 2048
(2) Per-memory-turn narration match rate
| bucket | match rate | count |
|---|
| overall | 5.0% |
3/60 |
| fine segments | 5.17% |
3/58 |
| composite segments (verbatim) | 0.0% |
0/2 |
(3) Keyframe selection quality
keyframe-count distribution:
{0: 8, 1: 63, 2: 68, 3: 21, 4: 9, 5: 3, 6: 4, 8: 2}
mean keyframes/turn: 1.983
final-query emitted-a-keyframe rate:
100.0%
(2/2)
(4) Chunked-vs-not memory-recall decomposition
| bucket | final-query acc | count |
|---|
| CHUNKED (true compressed-memory recall) |
0.0% |
0/2 |
| NOT-chunked (learned-prior floor) |
0.0% |
0/0 |
chunked = the test's target task instruction matches the _PNP pick-and-place regex reused verbatim from scripts/analyze_tier3b_vs_tier3a.py (object genuinely entered the bounded FIFO). chunked = true compressed-memory recall; notchunked = learned-prior floor.
Run
parse failures: 2 ·
duration: 2128.2s