전체 프롬프트 덤프 — combo_002_L29_S48_0006 / MoveFridgeToFreezer_refrigerated_item

0. 후보 집합 — 현재(누출) vs 제안(scene-wide)

현재 fixture_candidates()는 이 ep의 test fixture(=정답)들만 줍니다 → 후보 집합이 정답공간을 누출.
현재 후보 (recall 프롬프트에 그대로 주입):
- fridge_main_group_freezer (freezer)
- cab_6_main_group (cabinet)
- oven_main_group (oven)
제안: scene-wide (74개, ep_meta.fixtures, 벽/바닥 제외) — distractor 포함:
- bottom_main_group_1  [Drawer]
- cab_1_left_group  [HingeCabinet]
- cab_1_main_group  [HingeCabinet]
- cab_2_left_group  [HingeCabinet]
- cab_2_main_group  [HingeCabinet]
- cab_3_left_group  [HingeCabinet]
- cab_3_main_group  [HingeCabinet]
- cab_4_left_group  [HingeCabinet]
- cab_4_main_group  [HingeCabinet]
- cab_5_main_group  [HingeCabinet]
- cab_6_main_group  [HingeCabinet]
- coffee_machine_left_group  [CoffeeMachine]
- counter_1_left_group  [Counter]
- counter_1_main_group  [Counter]
- dishwasher_main_group  [Dishwasher]
- fridge_housing_main_group  [HousingCabinet]
- fridge_main_group  [FridgeSideBySide]
- hood_left_group  [Hood]
- island_island_group  [Counter]
- light_switch_1_room  [WallAccessory]
- light_switch_2_room  [WallAccessory]
- microwave_housing_main_group  [HousingCabinet]
- microwave_main_group  [Microwave]
- middle_main_group  [HingeCabinet]
- oven_housing_main_group  [HousingCabinet]
- oven_main_group  [Oven]
- panel_box_1_island_group_1  [PanelCabinet]
- panel_cab_2_island_group_1  [PanelCabinet]
- panel_stack_main_group_1  [PanelCabinet]
- paper_towel_main_group  [Accessory]
- sink_island_group  [Sink]
- stack_1_island_group_1  [Drawer]
- stack_1_island_group_2  [Drawer]
- stack_1_island_group_3  [Drawer]
- stack_1_left_group_1  [Drawer]
- stack_1_left_group_2  [Drawer]
- stack_1_left_group_3  [Drawer]
- stack_1_main_group_1  [Drawer]
- stack_1_main_group_2  [Drawer]
- stack_1_main_group_3  [Drawer]
- stack_2_island_group_1  [SingleCabinet]
- stack_2_left_group_1  [Drawer]
- stack_2_left_group_2  [Drawer]
- stack_2_left_group_3  [Drawer]
- stack_2_main_group_1  [Drawer]
- stack_2_main_group_2  [Drawer]
- stack_2_main_group_3  [Drawer]
- stack_3_island_group_1  [HingeCabinet]
- stack_3_left_group_1  [Drawer]
- stack_3_left_group_2  [Drawer]
- stack_3_left_group_3  [PanelCabinet]
- stack_3_main_group_1  [Drawer]
- stack_3_main_group_2  [Drawer]
- stack_3_main_group_3  [Drawer]
- stack_4_island_group_1  [SingleCabinet]
- stack_4_left_group_1  [Drawer]
- stack_4_left_group_2  [Drawer]
- stack_4_left_group_3  [Drawer]
- stack_5_island_group_1  [Drawer]
- stack_5_island_group_2  [Drawer]
- stack_5_island_group_3  [Drawer]
- stack_5_left_group_1  [Drawer]
- stack_5_left_group_2  [Drawer]
- stack_5_left_group_3  [Drawer]
- stool_1_room  [Stool]
- stool_2_room  [Stool]
- stool_3_room  [Stool]
- stool_4_room  [Stool]
- stovetop_left_group  [Stovetop]
- toaster_left_group  [Toaster]
- toaster_oven_main_group  [ToasterOven]
- top_main_group  [HingeCabinet]
- wall_accessory_1_room  [WallAccessory]
- wall_accessory_2_room  [WallAccessory]
+ 이 ep test fixtures (GT 인스턴스 매칭용, scene-wide와 합집합):
- cab_6_main_group (cabinet)
- fridge_main_group_freezer (freezer)
- oven_main_group (oven)

1. M2 (MemER) memory-build 프롬프트 — 세그먼트마다 호출

SYSTEM (MemER Appendix F verbatim):
You are a robot program that predicts actions. The video input from the egocentric camera shows the most recent actions the robot has executed. The images are selected frames of particular importance from all the actions the robot has executed so far. Based on these, output the current subtask the robot should execute and nothing else.
Return a JSON with:
- current_subtask: the action that should be executed at the current timestep
- keyframe_positions: list of frame positions (1-indexed) from the video input where actions change
USER (예: 첫 task 세그먼트, cap 후 n프레임 가정 n=48):
Task: infer the current subtask and which recent frames are keyframes.
[MEMORY] selected frames from the entirety of the full video (not selectable):
[RECENT] video of the most recent actions — Frame 1, Frame 2, Frame 3, Frame 4, Frame 5, Frame 6, Frame 7, Frame 8, Frame 9, Frame 10, Frame 11, Frame 12, Frame 13, Frame 14, Frame 15, Frame 16, Frame 17, Frame 18, Frame 19, Frame 20, Frame 21, Frame 22, Frame 23, Frame 24, Frame 25, Frame 26, Frame 27, Frame 28, Frame 29, Frame 30, Frame 31, Frame 32, Frame 33, Frame 34, Frame 35, Frame 36, Frame 37, Frame 38, Frame 39, Frame 40, Frame 41, Frame 42, Frame 43, Frame 44, Frame 45, Frame 46, Frame 47, Frame 48. keyframe_positions must be 1-indexed into these RECENT frames only.
+ 이미지 파트: 해당 세그먼트 프레임 ≤48장(cap). 출력 JSON {current_subtask, keyframe_positions(1-indexed)}.

2. M3 (MEM) memory-build 프롬프트 — 세그먼트마다 호출, m carry-forward

SYSTEM (MEM Sec III-B 내용충실):
You maintain a running memory of what a robot has done so far. Given the previous memory and the most recent segment of activity, output an updated memory. Summarize all information from previous and current activity that is still relevant for future task execution — especially where objects were placed and which steps are done. Remove or compress information whenever appropriate; keep only the minimal set of relevant information (e.g. 'I placed three bowls in the top right cabinet' rather than enumerating each bowl). Do not record failed or repeated attempts.
USER (첫 세그먼트 'MoveFridgeToFreezer', 이전 메모리 빈 상태):
You maintain a running memory of what a robot has done so far. Given the previous memory and the most recent segment of activity, output an updated memory. Summarize all information from previous and current activity that is still relevant for future task execution — especially where objects were placed and which steps are done. Remove or compress information whenever appropriate; keep only the minimal set of relevant information (e.g. 'I placed three bowls in the top right cabinet' rather than enumerating each bowl). Do not record failed or repeated attempts.

GOAL: Track the robot's activity across the whole video; remember where each object ends up and which steps are completed.

PREVIOUS MEMORY:
(empty)

Most recent segment frames follow.
+ 이미지 파트: 세그먼트 프레임 ≤48장. 출력 {memory}. 다음 세그먼트 USER의 PREVIOUS MEMORY 자리에 이 출력이 그대로 들어가 누적.

3. 공유 recall 프롬프트 (전 메소드 동일) — 최종 1회

SYSTEM:
You answer where an object ended up using ONLY the provided memory. Output must conform to the schema and pick a target_fixture_instance that is exactly one of the offered instances.
USER (test=MoveFridgeToFreezer_refrigerated_item, lang='Show me where the steak is placed.'):
Below is your memory of the robot's prior activity (images and/or text).
The robot must now satisfy this request:
  "Show me where the steak is placed."
Decide where the relevant object ended up, based ONLY on the memory. Choose exactly ONE target fixture from this list:
- fridge_main_group_freezer (freezer)
- cab_6_main_group (cabinet)
- oven_main_group (oven)
Return JSON: object_category, target_fixture_instance (must be one of the instances above, verbatim), instruction (a short imperative the robot should execute, e.g. 'go to the freezer and open it'), rationale.
+ 메모리 파트: M1=균일 8프레임 이미지 / M2=선택 keyframe 이미지 / M3=아래 누적 텍스트(이미지 없음). 출력 {object_category, target_fixture_instance (후보 중 verbatim), instruction, rationale}.

4. M3 이 시나리오 최종 누적 메모리 전문 (recall 때 위 USER 앞에 'MEMORY:\n…'로 첨부)

Completed steps: (1) Fish-design plate placed inside small countertop microwave (door open, on right counter). (2) Oven rack removed from wall oven and placed on counter; wall oven door remains open. (3) Small black pan placed on counter near sink area (now empty of items). (4) Several small mushroom-like items placed on counter near stovetop. (5) Orange/carrot-like item placed in lower drawer (drawer closed). (6) Red round item (tomato/apple) removed from pan and placed on counter to the left of sink. (7) Orange round item also placed on counter to the left of sink (both red and orange items now on counter left of sink). (8) Robot picked up a white bowl (previously on right counter area) and placed it in the black sink. Current state: fish-design plate in countertop microwave (door open); oven rack on counter; black pan on counter near sink (empty); mushroom-like items on counter near stovetop; red and orange round items on counter left of sink; white bowl placed in the sink; robot is idle near sink area.