2026-05-19 — Analysis
Scene-Mem API Track — 2-ep 스모크 결과
SceneMem | Letsur gateway · claude-sonnet-4-6 · dev --limit 2
TL;DR — control/M1/M2/M3 4종 메모리 메커니즘의 next-subtask slot-match accuracy를 검증. 아래 집계 표 + 테스트별 '왜' inspector(예측 sub-task vs GT, 메모리 근거).
12
runs
0.34
best slot_acc
3600
tests
1. 집계 비교 (method × model)
| method | model | n | slot_acc | place_acc | action_acc | exact_acc | parse_fail |
|---|---|---|---|---|---|---|---|
| control | gemini-2.5-flash | 300 | 0.04 | 0.50 | 0.04 | 0.02 | 0 |
| m1 | gemini-2.5-flash | 300 | 0.01 | 0.42 | 0.10 | 0.00 | 0 |
| m2 | gemini-2.5-flash | 300 | 0.03 | 0.45 | 0.12 | 0.01 | 0 |
| m3 | gemini-2.5-flash | 300 | 0.09 | 0.59 | 0.24 | 0.02 | 0 |
| control | gemini-3-flash-preview | 300 | 0.06 | 0.57 | 0.07 | 0.00 | 0 |
| m1 | gemini-3-flash-preview | 300 | 0.27 | 0.67 | 0.39 | 0.03 | 0 |
| m2 | gemini-3-flash-preview | 300 | 0.34 | 0.73 | 0.41 | 0.06 | 0 |
| m3 | gemini-3-flash-preview | 300 | 0.10 | 0.59 | 0.10 | 0.00 | 0 |
| control | gemini-3.5-flash | 300 | 0.00 | 0.61 | 0.00 | 0.00 | 0 |
| m1 | gemini-3.5-flash | 300 | 0.24 | 0.62 | 0.32 | 0.04 | 0 |
| m2 | gemini-3.5-flash | 300 | 0.29 | 0.65 | 0.36 | 0.10 | 0 |
| m3 | gemini-3.5-flash | 300 | 0.00 | 0.59 | 0.00 | 0.00 | 0 |
slot_acc=action&object&place 슬롯 일치, place_acc=place 일치, action_acc=action 일치, exact_acc=문자열 완전 일치.
2. 테스트별 정성 inspector
control · gemini-2.5-flash (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Open the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✓ pred: Place the fish on the bottom rack of the oven gt: Place the fish on the bottom rack of the oven action=✓ place=✓ exact=✓ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the fridge door gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Pick up the red bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 99.2–111.4s (STATE 위치) |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✗ pred: Slide the oven tray back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.2–158.3s (STATE 위치) |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Open the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✗ pred: Pick up the fish from the counter gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Navigate to the freezer gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✗ pred: Pick up the red bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✗ pred: Slide the tray back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Navigate to the freezer gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Pick up the red bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✓ pred: Place the fish on the bottom rack of the oven gt: Place the fish on the bottom rack of the oven action=✓ place=✓ exact=✓ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
m1 · gemini-2.5-flash (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | uniform history · n_history=32 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Pick up the red bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | uniform history · n_history=64 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✗ pred: Place the fish on the oven rack gt: Place the fish on the bottom rack of the oven action=✓ place=✗ exact=✗ | ok | uniform history · n_history=56 | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Open the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | uniform history · n_history=32 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Pick up the bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | uniform history · n_history=64 | target window 99.2–111.4s (STATE 위치) |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✗ pred: Slide the tray back inside the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | uniform history · n_history=56 | target window 146.2–158.3s (STATE 위치) |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Open the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | uniform history · n_history=56 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✗ pred: Slide the oven rack back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | uniform history · n_history=48 | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✗ pred: Pick up the red bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | uniform history · n_history=56 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✗ pred: Slide the tray back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | uniform history · n_history=48 | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Open the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | uniform history · n_history=56 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✗ pred: Slide the tray back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | uniform history · n_history=48 | target window 146.1–158.2s (STATE 위치) |
m2 · gemini-2.5-flash (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | idx=[170]  | target window 9.1–18.1s (STATE 위치) | KF@ 8.5s |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Place the red bowl in the cabinet gt: Place a bowl in the cabinet action=✓ place=✓ exact=✗ | ok | idx=[1250, 1280, 1310, 1340, 1370, 1410, 1440, 1470]         | target window 99.0–111.2s (STATE 위치) | KF@ 62.5s, 64.0s, 65.5s, 67.0s, 68.5s, 70.5s, 72.0s, 73.5s |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✗ pred: Slide the tray back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | idx=[730, 970, 2290, 2900, 3080]      | target window 146.1–158.2s (STATE 위치) | KF@ 36.5s, 48.5s, 114.5s, 145.0s, 154.0s |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | idx=[140, 170]   | target window 9.1–18.1s (STATE 위치) | KF@ 7.0s, 8.5s |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Pick up the red bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | idx=[1190, 1220, 1960]    | target window 99.2–111.4s (STATE 위치) | KF@ 59.5s, 61.0s, 98.0s |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✗ pred: Slide the tray back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | idx=[910, 940, 970, 1010, 1190, 3110, 3140]       | target window 146.2–158.3s (STATE 위치) | KF@ 45.5s, 47.0s, 48.5s, 50.5s, 59.5s, 155.5s, 157.0s |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | idx=[20, 50, 70, 100, 120, 140]       | target window 9.1–18.1s (STATE 위치) | KF@ 1.0s, 2.5s, 3.5s, 5.0s, 6.0s, 7.0s |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Open the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | idx=[460, 550, 640, 970, 1470]     | target window 99.0–111.2s (STATE 위치) | KF@ 23.0s, 27.5s, 32.0s, 48.5s, 73.5s |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✗ pred: Slide the oven rack back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | idx=[550, 640, 910, 970, 2900, 3140]    | target window 146.1–158.2s (STATE 위치) | KF@ 27.5s, 32.0s, 45.5s, 48.5s, 145.0s, 157.0s |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | idx=[230, 260]   | target window 9.1–18.1s (STATE 위치) | KF@ 11.5s, 13.0s |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✗ pred: Open the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | idx=[360, 1710, 1740]    | target window 99.0–111.2s (STATE 위치) | KF@ 18.0s, 85.5s, 87.0s |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✗ pred: Slide the tray back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | idx=[870, 910, 940, 2570, 2900, 3080, 3110, 3140]         | target window 146.1–158.2s (STATE 위치) | KF@ 43.5s, 45.5s, 47.0s, 128.5s, 145.0s, 154.0s, 155.5s, 157.0s |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | idx=[230]  | target window 9.1–18.1s (STATE 위치) | KF@ 11.5s |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Open the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | idx=[1130, 1160, 1190, 1220, 1250, 1370, 2170]      | target window 99.0–111.2s (STATE 위치) | KF@ 56.5s, 58.0s, 59.5s, 61.0s, 62.5s, 68.5s, 108.5s |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✗ pred: Slide the tray back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | idx=[640, 870, 2290, 2550, 2570, 2900, 3110, 3140]        | target window 146.1–158.2s (STATE 위치) | KF@ 32.0s, 43.5s, 114.5s, 127.5s, 128.5s, 145.0s, 155.5s, 157.0s |
m3 · gemini-2.5-flash (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Navigate to the freezer gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | The robot has picked up the steak from the fridge. Now, close the fridge door. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Open the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | The robot needs to open the cabinet, then pick up the bowls from the counter and place them inside, and finally close the cabinet. The cabinet door is currently closed, so the next step is to open it. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✗ pred: Slide the rack back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | The robot has placed the fish on the bottom rack of the oven. The oven door needs to be closed. | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Navigate to the freezer gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | The steak has been picked up from the fridge and is in the robot's gripper. The fridge door is almost closed, or in the process of closing. The next step is to move the steak to the freezer. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Place the red bowl in the cabinet gt: Place a bowl in the cabinet action=✓ place=✓ exact=✗ | ok | The cabinet has been opened. The red bowl has been picked from the counter. The robot is placing the red bowl in the cabinet. | target window 99.2–111.4s (STATE 위치) |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✗ pred: Place the fish on the rack gt: Place the fish on the bottom rack of the oven action=✓ place=✗ exact=✗ | ok | The robot is at the oven, which is open. It has placed the fish on the bottom rack of the oven. Now it needs to slide the tray back in. | target window 146.2–158.3s (STATE 위치) |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Navigate to the freezer gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | The goal is to move the steak from the fridge to the freezer. The steak has been picked up from the fridge. Now, navigate to the freezer to place the steak inside. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Place the red bowl in the cabinet gt: Place a bowl in the cabinet action=✓ place=✓ exact=✗ | ok | The cabinet is open. I am placing the red bowl in the cabinet. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✗ pred: Place the fish on the oven rack gt: Place the fish on the bottom rack of the oven action=✓ place=✗ exact=✗ | ok | The cabinet is open. One bowl has been picked from the counter and placed in the cabinet. The robot has navigated to the oven and is holding the fish. The oven door is open and a rack is extended. The robot is now placing the fish on the oven rack. | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | The steak has been picked up from the fridge and is now being placed in the freezer. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✗ pred: Place the red bowl in the cabinet gt: Place a bowl in the cabinet action=✓ place=✓ exact=✗ | ok | The cabinet has been opened. I have picked up the red bowl. Now I will place it in the cabinet. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✗ pred: Place the fish on the oven rack gt: Place the fish on the bottom rack of the oven action=✓ place=✗ exact=✗ | ok | The robot is holding the fish. The oven door is open and the bottom rack is pulled out. The fish needs to be placed on the oven rack. | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Navigate to the freezer gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | The steak has been picked up from the fridge. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Pick up the red bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | The cabinet door is now open. I need to pick up the bowls from the counter and place them in the cabinet. I am now picking the red bowl. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✗ pred: Slide the tray back inside the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | The robot has placed the fish on the bottom rack of the oven. Now it needs to slide the tray back inside the oven. | target window 146.1–158.2s (STATE 위치) |
control · gemini-3-flash-preview (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✗ pred: Slide the oven tray back inside gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Pick up the steak from the fridge gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.2–111.4s (STATE 위치) |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✗ pred: Slide the tray back inside the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.2–158.3s (STATE 위치) |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✗ pred: Slide the bottom rack into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✗ pred: Slide the tray into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✗ pred: Slide the oven rack back inside gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
m1 · gemini-3-flash-preview (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Pick up the bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | uniform history · n_history=56 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | uniform history · n_history=48 | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Place the red bowl in the cabinet gt: Place a bowl in the cabinet action=✓ place=✓ exact=✗ | ok | uniform history · n_history=56 | target window 99.2–111.4s (STATE 위치) |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | uniform history · n_history=48 | target window 146.2–158.3s (STATE 위치) |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | uniform history · n_history=56 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | uniform history · n_history=48 | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✗ pred: Pick up the white bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | uniform history · n_history=56 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | uniform history · n_history=48 | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Place the white bowl in the cabinet gt: Place a bowl in the cabinet action=✓ place=✓ exact=✗ | ok | uniform history · n_history=56 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | uniform history · n_history=48 | target window 146.1–158.2s (STATE 위치) |
m2 · gemini-3-flash-preview (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | idx=[170, 350]  | target window 9.1–18.1s (STATE 위치) | KF@ 8.5s, 17.5s |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Navigate to the cabinet gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | idx=[640, 730, 870, 1220, 1470, 1990]    | target window 99.0–111.2s (STATE 위치) | KF@ 32.0s, 36.5s, 43.5s, 61.0s, 73.5s, 99.5s |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | idx=[180, 270, 640, 730, 1190, 1740, 2110, 2290]    | target window 146.1–158.2s (STATE 위치) | KF@ 9.0s, 13.5s, 32.0s, 36.5s, 59.5s, 87.0s, 105.5s, 114.5s |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | idx=[100, 140, 170, 230]   | target window 9.1–18.1s (STATE 위치) | KF@ 5.0s, 7.0s, 8.5s, 11.5s |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | idx=[270, 550, 1010, 1220, 1990, 2110]    | target window 99.2–111.4s (STATE 위치) | KF@ 13.5s, 27.5s, 50.5s, 61.0s, 99.5s, 105.5s |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✓ pred: Place the fish on the bottom rack of the oven gt: Place the fish on the bottom rack of the oven action=✓ place=✓ exact=✓ | ok | idx=[90, 270, 550, 1380, 1930, 2300, 2720, 2870]       | target window 146.2–158.3s (STATE 위치) | KF@ 4.5s, 13.5s, 27.5s, 69.0s, 96.5s, 115.0s, 136.0s, 143.5s |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | idx=[170, 230]   | target window 9.1–18.1s (STATE 위치) | KF@ 8.5s, 11.5s |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | idx=[270, 640, 970, 1220, 1990]  | target window 99.0–111.2s (STATE 위치) | KF@ 13.5s, 32.0s, 48.5s, 61.0s, 99.5s |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✓ pred: Place the fish on the oven gt: Place the fish on the bottom rack of the oven action=✓ place=✓ exact=✗ | ok | idx=[180, 640, 2110, 2290, 2710, 2900]   | target window 146.1–158.2s (STATE 위치) | KF@ 9.0s, 32.0s, 105.5s, 114.5s, 135.5s, 145.0s |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | idx=[100, 170] | target window 9.1–18.1s (STATE 위치) | KF@ 5.0s, 8.5s |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✓ pred: Place the bowl in the cabinet gt: Place a bowl in the cabinet action=✓ place=✓ exact=✗ | ok | idx=[270, 640, 870, 1220, 1710, 1990, 2020, 2050]   | target window 99.0–111.2s (STATE 위치) | KF@ 13.5s, 32.0s, 43.5s, 61.0s, 85.5s, 99.5s, 101.0s, 102.5s |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✓ pred: Place the fish on the bottom rack of the oven gt: Place the fish on the bottom rack of the oven action=✓ place=✓ exact=✓ | ok | idx=[270, 640, 1190, 1560, 1920, 2290, 2840, 2900]    | target window 146.1–158.2s (STATE 위치) | KF@ 13.5s, 32.0s, 59.5s, 78.0s, 96.0s, 114.5s, 142.0s, 145.0s |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | idx=[100, 170, 190]   | target window 9.1–18.1s (STATE 위치) | KF@ 5.0s, 8.5s, 9.5s |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Pick up the red bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | idx=[90, 640, 1220, 1470, 1500]  | target window 99.0–111.2s (STATE 위치) | KF@ 4.5s, 32.0s, 61.0s, 73.5s, 75.0s |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✗ pred: Place the fish on the oven rack. gt: Place the fish on the bottom rack of the oven action=✓ place=✗ exact=✗ | ok | idx=[90, 640, 1190, 1740, 2110, 2740, 2900]  | target window 146.1–158.2s (STATE 위치) | KF@ 4.5s, 32.0s, 59.5s, 87.0s, 105.5s, 137.0s, 145.0s |
m3 · gemini-3-flash-preview (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | I have successfully picked up the steak from the fridge and placed it on a shelf in the freezer. Now I will close the freezer door and then the fridge door to complete the goal. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | The cabinet is open and both bowls (red and white) have been placed inside. I am now closing the cabinet door to complete the task. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✗ pred: Slide the oven rack back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | Navigated to the counter and picked up the fish. Moved to the oven and placed the fish on the extended bottom rack. Next step is to push the rack back into the oven and then close the door. | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | I have successfully moved the steak from the fridge to a shelf in the freezer. Both the fridge and freezer doors are currently open. I am now proceeding to close the freezer door, after which I will close the fridge door to complete the task. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | I have successfully opened the cabinet and placed both bowls (red and white) inside. I am currently closing the cabinet door to finish the task. | target window 99.2–111.4s (STATE 위치) |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✗ pred: Push the oven rack back into the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | The red and white bowls are in the closed cabinet. I have successfully placed the fish on the bottom rack of the oven. The next step is to push the rack back inside before closing the door. | target window 146.2–158.3s (STATE 위치) |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the fridge door gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | Opened fridge and freezer. Moved steak from fridge to freezer shelf. Now closing the fridge door. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | The cabinet is open. Both the red bowl and the white bowl have been placed inside the cabinet. I am now closing the cabinet door to complete the task. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✗ pred: Slide the oven rack back inside the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | Successfully placed the fish on the extended bottom rack of the oven. Next, I need to push the rack back inside before closing the door. | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Open the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | I have picked up the steak from the fridge and am now opening the freezer door to place it inside. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | The cabinet was opened, and both bowls were placed inside. I am now closing the cabinet door to complete the goal. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✗ pred: Slide the oven rack back inside gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | I have placed the fish on the bottom rack of the oven. Now I need to slide the rack back inside before closing the door. | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | I have successfully placed the steak on the freezer shelf. Now I need to close the freezer door and the fridge door to finish the task. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | I have successfully opened the cabinet and placed the red and white bowls inside. I am now closing the cabinet door to finish the task. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✗ pred: Slide the oven rack back inside the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | I have placed the fish on the bottom rack of the oven. Now I must slide the rack back inside and then close the oven door. | target window 146.1–158.2s (STATE 위치) |
control · gemini-3.5-flash (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✗ pred: Slide the oven tray back inside gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.2–111.4s (STATE 위치) |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✗ pred: Push the oven rack inside gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.2–158.3s (STATE 위치) |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the fridge door gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✗ pred: Slide the bottom rack back inside the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✗ pred: Slide the tray back inside the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | (no long-term memory — control) | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✗ pred: Slide the oven rack back inside gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | (no long-term memory — control) | target window 146.1–158.2s (STATE 위치) |
m1 · gemini-3.5-flash (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak on the shelf in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Pick up the bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | uniform history · n_history=56 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | uniform history · n_history=48 | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Pick up the bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | uniform history · n_history=56 | target window 99.2–111.4s (STATE 위치) |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | uniform history · n_history=48 | target window 146.2–158.3s (STATE 위치) |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Pick up the bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | uniform history · n_history=56 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | uniform history · n_history=48 | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✗ pred: Pick up the bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | uniform history · n_history=56 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | uniform history · n_history=48 | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | uniform history · n_history=16 | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Pick up the bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | uniform history · n_history=56 | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | uniform history · n_history=48 | target window 146.1–158.2s (STATE 위치) |
m2 · gemini-3.5-flash (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the fridge door gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | idx=[140, 260]   | target window 9.1–18.1s (STATE 위치) | KF@ 7.0s, 13.0s |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | idx=[640, 970, 1220, 1990, 2200]  | target window 99.0–111.2s (STATE 위치) | KF@ 32.0s, 48.5s, 61.0s, 99.5s, 110.0s |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | idx=[270, 550, 640, 1370, 3080, 3140]  | target window 146.1–158.2s (STATE 위치) | KF@ 13.5s, 27.5s, 32.0s, 68.5s, 154.0s, 157.0s |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | idx=[100, 170] | target window 9.1–18.1s (STATE 위치) | KF@ 5.0s, 8.5s |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Open the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | idx=[460, 640, 1220]  | target window 99.2–111.4s (STATE 위치) | KF@ 23.0s, 32.0s, 61.0s |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✗ pred: Navigate to the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✓ exact=✗ | ok | idx=[180, 270, 550, 2300, 3140]  | target window 146.2–158.3s (STATE 위치) | KF@ 9.0s, 13.5s, 27.5s, 115.0s, 157.0s |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the fridge door gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | idx=[140, 260]  | target window 9.1–18.1s (STATE 위치) | KF@ 7.0s, 13.0s |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Open the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | idx=[270, 640, 1160] | target window 99.0–111.2s (STATE 위치) | KF@ 13.5s, 32.0s, 58.0s |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✗ pred: Place the fish on the oven rack gt: Place the fish on the bottom rack of the oven action=✓ place=✗ exact=✗ | ok | idx=[270, 460, 2290, 2900] | target window 146.1–158.2s (STATE 위치) | KF@ 13.5s, 23.0s, 114.5s, 145.0s |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✓ pred: Place the steak in the freezer gt: Place the steak on a shelf in the freezer action=✓ place=✓ exact=✗ | ok | idx=[170] | target window 9.1–18.1s (STATE 위치) | KF@ 8.5s |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✗ pred: Pick up the bowl from the counter gt: Place a bowl in the cabinet action=✗ place=✗ exact=✗ | ok | idx=[640, 1160] | target window 99.0–111.2s (STATE 위치) | KF@ 32.0s, 58.0s |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✓ pred: Place the fish on the bottom rack of the oven gt: Place the fish on the bottom rack of the oven action=✓ place=✓ exact=✓ | ok | idx=[640, 2290, 2900] | target window 146.1–158.2s (STATE 위치) | KF@ 32.0s, 114.5s, 145.0s |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the fridge door gt: Place the steak on a shelf in the freezer action=✗ place=✗ exact=✗ | ok | idx=[100, 260] | target window 9.1–18.1s (STATE 위치) | KF@ 5.0s, 13.0s |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Open the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | idx=[640, 1160, 1220] | target window 99.0–111.2s (STATE 위치) | KF@ 32.0s, 58.0s, 61.0s |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✗ pred: Place the fish on the oven rack gt: Place the fish on the bottom rack of the oven action=✓ place=✗ exact=✗ | ok | idx=[550, 640, 2900] | target window 146.1–158.2s (STATE 위치) | KF@ 27.5s, 32.0s, 145.0s |
m3 · gemini-3.5-flash (inspector: first 15/300 rows; full metrics in §1 table)
| scenario / test | 예측 vs GT | status | 메모리 근거 | 영상(타깃 구간) |
|---|---|---|---|---|
| combo_002_L29_S48_0006 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | Placed the steak in the freezer. Now closing the freezer door. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0006 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | Both bowls have been placed inside the cabinet. Closing the cabinet door to complete the task. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0006 OvenBroilFish_obj | ✗ pred: Push the bottom rack back inside the oven gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | Placed the fish on the bottom rack of the oven. Now pushing the bottom rack back inside the oven. | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0016 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | Placed the steak in the freezer. Now closing the freezer door. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0016 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | Both bowls are placed in the cabinet. Currently closing the cabinet door. | target window 99.2–111.4s (STATE 위치) |
| combo_002_L29_S48_0016 OvenBroilFish_obj | ✗ pred: Slide the oven tray back inside gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | Placed the fish on the bottom rack of the oven. Now sliding the oven tray back inside. | target window 146.2–158.3s (STATE 위치) |
| combo_002_L29_S48_0018 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | Placed the steak in the freezer. Now closing the freezer door. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0018 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | The cabinet door is open. Both the red bowl and the white-and-gold bowl are now placed in the cabinet. Currently closing the cabinet door. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0018 OvenBroilFish_obj | ✗ pred: Slide the oven rack back inside gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | Placed the fish on the bottom rack of the oven. Now sliding the oven rack back inside. | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0020 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | Placed the steak in the freezer. Now closing the freezer door. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0020 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | The red bowl and beige bowl are inside the cabinet, and the cabinet door is now fully closed. The task is complete. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0020 OvenBroilFish_obj | ✗ pred: Slide the tray back inside gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | Placed the fish on the bottom rack of the oven. Now sliding the tray back inside. | target window 146.1–158.2s (STATE 위치) |
| combo_002_L29_S48_0023 MoveFridgeToFreezer_refrigerated_item | ✗ pred: Close the freezer door gt: Place the steak on a shelf in the freezer action=✗ place=✓ exact=✗ | ok | Successfully placed the steak on a shelf in the freezer. Now I need to close the freezer door. | target window 9.1–18.1s (STATE 위치) |
| combo_002_L29_S48_0023 RestockBowls_obj1 | ✗ pred: Close the cabinet door gt: Place a bowl in the cabinet action=✗ place=✓ exact=✗ | ok | Finished placing the bowl in the cabinet and successfully closed the cabinet door. Task complete. | target window 99.0–111.2s (STATE 위치) |
| combo_002_L29_S48_0023 OvenBroilFish_obj | ✗ pred: Slide the oven rack back inside gt: Place the fish on the bottom rack of the oven action=✗ place=✗ exact=✗ | ok | Placed the fish on the bottom oven rack. Now need to slide the rack back inside the oven. | target window 146.1–158.2s (STATE 위치) |