Routine read-track 3편 합본 — 쌍둥이 pair 채점

← 리포트 목록

TL;DR

3편(3000 D①+D②, 2000/2001 D① 쌍둥이) × flash × 5전략 = 75행, 239 call, $1.78. 파싱 실패 0. 헤드라인(누출 행 제외, n=8): textlog .875 · rf-v3 .875 · frames .625 · vis .25 · blind 0.
쌍둥이 pair(A6, 5짝): textlog 1.0 · rf-v3 0.6 · frames 0.2 · vis 0.2 · blind 0 — textlog·rf-v3는 두 편에서 답이 정확히 뒤집혔다(cloth→바구니 ↔ 책상). blind는 항상 '조리대'라 한 편에서 필패.
데이터 결함 실증: strip 안 된 Pen_3가 거실 소파 위에 보여 frames·rf-v3·vis가 control(pen)에서 '소파'라고 답함(rf-v3 메모리에 'sofa (with a pen)'). 미끼·홀드아웃 클래스 소품 strip이 필수. 2000편 마지막 프레임은 바구니 속 천이 보여 leak 플래그 정확.

contexts

rows

239

API calls

1.7774

cost $

1세 편 — 규칙·유닛·누출·쌍둥이

편	규칙 (출처, 증거 k)	유닛 ok	길이	누출 행	쌍둥이
`train_1652__2000`	FOOD→Desk (self, k=1) CUTLERY→DiningTable (self, k=2) CLOTH→Basket (self, k=2)	5/6	13.7분	direct:cloth, multi	train_1652__2001
`train_1652__2001`	CLOTH→Desk (self, k=1) FOOD→DiningTable (self, k=2) CUTLERY→Basket (self, k=1)	4/6	13.4분	—	train_1652__2000
`train_1652__3000`	FOOD→Desk (self, k=2) CLOTH→Basket (self, k=2) CUTLERY→Sofa (observed, k=4)	4/4	11.8분	direct:cloth, multi	—

2000/2001 = 같은 집·같은 홀드아웃(tomato·hand towel·fork·pen)·규칙 값 순환(고정점 없음). 3000 = D① 2 + D② 1(관찰). 홀드아웃은 세 편 모두 컨텍스트 미등장 하위 카테고리.

train_1652__2000 t≈235 s (exo_camera_1): 거실 소파 위 흰 펜 — strip되지 않은 씬 소품 Pen_3. frames·rf-v3·vis가 control 행에서 이 위치를 답했다.

train_1652__2000 마지막 프레임: 바구니 속 분홍 천이 보인다 → cloth·multi 행 leak_last_frame=True(헤드라인 제외).

2쌍둥이 짝 채점 (A6) — 같은 홀드아웃, 두 편 모두 place_set이어야 짝 정답

행	GT	전략	2000	2001	pair
food	2000: bedroom/desk · 2001: living room/dining table	blind	❌	❌	❌
food	2000: bedroom/desk · 2001: living room/dining table	textlog	✅	✅	✅
food	2000: bedroom/desk · 2001: living room/dining table	frames-uniform	✅	△	❌
food	2000: bedroom/desk · 2001: living room/dining table	rollforward-v3	✅	✅	✅
food	2000: bedroom/desk · 2001: living room/dining table	rollforward-vis-pure	❌	❌	❌
cutlery	2000: living room/dining table · 2001: kitchen/laundry basket	blind	❌	❌	❌
cutlery	2000: living room/dining table · 2001: kitchen/laundry basket	textlog	✅	✅	✅
cutlery	2000: living room/dining table · 2001: kitchen/laundry basket	frames-uniform	✅	△	❌
cutlery	2000: living room/dining table · 2001: kitchen/laundry basket	rollforward-v3	✅	✅	✅
cutlery	2000: living room/dining table · 2001: kitchen/laundry basket	rollforward-vis-pure	❌	❌	❌
cloth	2000: kitchen/laundry basket · 2001: bedroom/desk	blind	❌	❌	❌
cloth	2000: kitchen/laundry basket · 2001: bedroom/desk	textlog	✅	✅	✅
cloth	2000: kitchen/laundry basket · 2001: bedroom/desk	frames-uniform	✅	✅	✅
cloth	2000: kitchen/laundry basket · 2001: bedroom/desk	rollforward-v3	✅	✅	✅
cloth	2000: kitchen/laundry basket · 2001: bedroom/desk	rollforward-vis-pure	✅	✅	✅
multi	2000: bedroom/desk · 2001: bedroom/desk	blind	❌ ctl❌	❌ ctl✅	❌
multi	2000: bedroom/desk · 2001: bedroom/desk	textlog	✅ ctl✅	✅ ctl✅	✅
multi	2000: bedroom/desk · 2001: bedroom/desk	frames-uniform	✅ ctl✅	△ ctl✅	❌
multi	2000: bedroom/desk · 2001: bedroom/desk	rollforward-v3	✅ ctl✅	❌ ctl❌	❌
multi	2000: bedroom/desk · 2001: bedroom/desk	rollforward-vis-pure	❌ ctl❌	❌ ctl✅	❌
control	기권이 정답	blind	기권❌	기권✅	❌
control	기권이 정답	textlog	기권✅	기권✅	✅
control	기권이 정답	frames-uniform	기권❌	기권✅	❌
control	기권이 정답	rollforward-v3	기권❌	기권❌	❌
control	기권이 정답	rollforward-vis-pure	기권❌	기권✅	❌

전략	n_pairs (direct·multi, 누출 행 포함)	pair_ok
blind	5	0.00
textlog	5	1.00
frames-uniform	5	0.20
rollforward-v3	5	0.60
rollforward-vis-pure	5	0.20

✅ place_set · ◐ place_any(다른 인스턴스) · △ class_ok · ❌ 오답/기권 · ctl = 미끼 제외. 상식으로 찍으면 한 편에서 필패 → pair가 규칙 사용의 증거.

3전 편 집계 (15행 × 5전략) — 헤드라인은 누출 행(⚠️) 제외

행	blind	textlog	frames-uniform	rollforward-v3	rollforward-vis-pure
`2000:control`	기권❌	기권✅	기권❌	기권❌	기권❌
`2000:direct:cloth` ⚠️	❌	✅	✅	✅	✅
`2000:direct:cutlery`	❌	✅	✅	✅	❌
`2000:direct:food`	❌	✅	✅	✅	❌
`2000:multi` ⚠️	❌ ctl❌	✅ ctl✅	✅ ctl✅	✅ ctl✅	❌ ctl❌
`2001:control`	기권✅	기권✅	기권✅	기권❌	기권✅
`2001:direct:cloth`	❌	✅	✅	✅	✅
`2001:direct:cutlery`	❌	✅	△	✅	❌
`2001:direct:food`	❌	✅	△	✅	❌
`2001:multi`	❌ ctl✅	✅ ctl✅	△ ctl✅	❌ ctl❌	❌ ctl✅
`3000:control`	기권❌	기권✅	기권✅	기권✅	기권❌
`3000:direct:cloth` ⚠️	❌	✅	✅	✅	✅
`3000:direct:cutlery`	❌	❌	✅	✅	✅
`3000:direct:food`	❌	✅	✅	✅	❌
`3000:multi` ⚠️	❌ ctl✅	❌ ctl✅	✅ ctl✅	✅ ctl✅	❌ ctl✅

전략	n	place_set	loc_set	place_any	class_ok	D① self	D② observed	multi	전체(누출 포함)	control(multi)	control(abstain)	abstain	parse
blind	8	0.00	0.12	0.00	0.00	0.00 n=6	0.00	0.00	0.00 n=12	1.00	0.33	0.125	1.0
textlog	8	0.88	0.88	0.88	0.88	1.00 n=6	0.00	1.00	0.83 n=12	1.00	1.00	0.125	1.0
frames-uniform	8	0.62	0.62	0.62	1.00	0.67 n=6	1.00	0.00	0.75 n=12	1.00	0.67	0.0	1.0
rollforward-v3	8	0.88	0.88	0.88	0.88	1.00 n=6	1.00	0.00	0.92 n=12	0.00	0.33	0.0	1.0
rollforward-vis-pure	8	0.25	0.50	0.25	0.25	0.17 n=6	1.00	0.00	0.33 n=12	1.00	0.33	0.0	1.0

A6 작동: textlog·rf-v3는 2000(음식→침실 책상·수저→거실 식탁·천→부엌 바구니)과 2001(천→책상·음식→식탁·수저→바구니)에서 모두 정답을 뒤집어 맞혔다(direct 3클래스 6/6). 상식 대조군 blind는 모든 행에서 '부엌 조리대'(0/6). 규칙 사용의 증거가 짝 채점으로 드러난다.

textlog(방 포함 로그)는 D① 6/6 place, D②(3000 수저) 기권 — 오라클 본래 자리. rf-v3도 D① 6/6·D② 1/1로 텍스트 요약이 방까지 보존했다.

frames-uniform은 수납처 클래스는 8/8 맞지만 방을 틀린다(2001: '식탁은 부엌에', '바구니는 욕실에'): 8장 균등 샘플로는 수납처의 방을 특정하기 어렵다 → class_ok 1.0 vs place .625. 3000편 4/4는 운이 좋은 샘플이었다.

vis-pure(모델이 고른 8장)는 마지막 유닛에 치우친 보관(2000: 17999~20348 프레임 4장)으로 마지막 규칙(천)만 맞히고 나머지는 질의 시점 조리대 위 여분 사과·도구에 끌려 '조리대'. 2001은 식탁을 '사이드테이블'로 오인.

control(pen) 기권률: textlog 3/3, frames 2/3, blind·rf-v3·vis 1/3. 단 2000/2001의 실패는 대부분 '소파 위 펜을 봤다'는 사실 보고(데이터 결함)이고, 순수 confabulation은 3000 vis('거실 흰 책상')·blind(상식)뿐. control 지표는 소품 strip 후 다시 재야 한다.

pair_ok 채점기 수정: control 행 짝은 place_set이 아니라 기권(control_ok)으로 재도록 고침(textlog 0.8→1.0). 누출 행(2000 cloth·multi)은 pair에 포함돼 있음 — 표에서 ⚠️로 구분.

4쌍둥이 메모리 실물

train_1652__2000

textlog (방 포함)

[ 0s] The user said: "Put the food from the countertop on the desk." [ 0s] The user said: "Put the kitchen utensils from the countertop on the dining table." [ 0s] The user said: "Put the clothes from the countertop into the laundry basket." [ 0s– 212s] I picked up the potato from the countertop in the kitchen and placed it on the desk in the bedroom. [ 212s– 380s] I picked up the knife from the countertop in the kitchen and placed it on the dining table in the living room. [ 380s– 498s] I picked up the cloth from the countertop in the kitchen and put it into the laundry basket in the kitchen. [ 498s– 582s] I tried to move the apple from the countertop in the kitchen to the desk in the bedroom, but the attempt failed. [ 582s– 725s] I picked up the knife from the countertop in the kitchen and placed it on the dining table in the living room. [ 725s– 821s] I picked up the cloth from the countertop in the kitchen and put it into the laundry basket in the kitchen.

rf-v3 최종 요약

I received instructions to: 1. Put food from the countertop on the desk. 2. Put kitchen utensils from the countertop on the dining table. 3. Put clothes from the countertop into the laundry basket. I have observed: - In the living/dining area: a desk (with a pencil), a dining table (with two knives I placed), a sofa (with a pen and fabric), a house plant on the floor, and a tv stand (with a candle and a bowl). An apple is on the floor near the doorway. - In the bedroom: a bed (with a pillow), a chair, a desk (with a CD, an irish potato, and a tomato I placed), and a dresser (with an alarm clock and a vase). - In the hallway/other room: a shelf (with a bottle and a CD). - In the kitchen: a laundry basket on the floor (with three fabrics I placed), a refrigerator, and a countertop (with lettuce, a spatula, an egg, and a tomato). I placed the pink fabric into the laundry basket.

vis-pure 보관 8장

train_1652__2001

textlog (방 포함)

[ 0s] The user said: "Put the clothes from the countertop on the desk." [ 0s] The user said: "Put the food from the countertop on the dining table." [ 0s] The user said: "Put the kitchen utensils from the countertop into the laundry basket." [ 0s– 212s] I picked up the cloth from the countertop in the kitchen and placed it on the desk in the bedroom. [ 212s– 392s] I picked up the apple from the countertop in the kitchen and placed it on the dining table in the living room. [ 392s– 510s] I picked up the spatula from the countertop in the kitchen and put it into the laundry basket in the kitchen. [ 511s– 550s] I tried to move the cloth from the countertop in the kitchen to the desk in the bedroom, but the attempt failed. [ 550s– 725s] I picked up the egg from the countertop in the kitchen and placed it on the dining table in the living room. [ 725s– 801s] I tried to move the knife from the countertop in the kitchen to the laundry basket in the kitchen, but the attempt failed.

rf-v3 최종 요약

The user instructed me to move clothes from the countertop to the desk, food from the countertop to the dining table, and kitchen utensils from the countertop into the laundry basket. In the bedroom/living area, I saw a pillow on the bed, a pen on the sofa, a bowl and a candle on the tv stand, a house plant on the floor, an alarm clock and a vase on the dresser, an apple on a side table, and a bottle on a shelf. On the desk, there is a CD and a fabric. On the dining table, there is lettuce, an apple, and an egg. In the kitchen, on the countertop there is an irish potato, a spoon, an apple, and two fabrics. On the kitchen floor, there is a laundry basket containing a spatula. I dropped the knife near the kitchen countertop / floor.

vis-pure 보관 8장

5원문 답 (2000/2001 — 3000은 전 과정 리포트 참조)

2000:control — “Put this pen away.”

blind 기권❌ attempts=1 $0.004

```json {"reasoning": "A yellow pen/pencil is stored in the wooden shelf located in the kitchen.", "selections": [{"category": "pen", "room": "kitchen", "surface": "shelf"}], "abstain": false} ```

textlog 기권✅ attempts=1 $0.003

```json {"reasoning": "The user's instructions covered food, kitchen utensils, and clothes, but gave no rule or location for pens or office supplies.", "selections": [], "abstain": true} ```

frames-uniform 기권❌ attempts=1 $0.010

```json {"reasoning": "In the living room snapshot (t=234.8s), a pen was observed lying on the sofa.", "selections": [{"category": "pen", "room": "living room", "surface": "sofa"}], "abstain": false} ```

rollforward-v3 기권❌ attempts=1 $0.003

```json {"reasoning": "A pen was observed on the sofa in the living room.", "selections": [{"category": "pen", "room": "living room", "surface": "sofa"}], "abstain": false} ```

rollforward-vis-pure 기권❌ attempts=1 $0.012

```json {"reasoning": "In the memory frames (e.g., at t=460.0s and t=813.9s), stationery/pens are kept on the low shelf unit located in the kitchen.", "selections": [{"category": "pen", "room": "kitchen", "surface": "shelf"}], "abstain": false} ```

2000:direct:cloth — “Put this hand towel away.”

blind ❌ attempts=1 $0.004

```json {"reasoning": "Hand towels belong in the bathroom and are typically kept on the countertop or shelf.", "selections": [{"category": "hand towel", "room": "bathroom", "surface": "countertop"}], "abstain": false} ```

textlog ✅ attempts=1 $0.002

```json {"reasoning": "A hand towel is a cloth/linen item, which the user instructed to place into the laundry basket in the kitchen.", "selections": [{"category": "hand towel", "room": "kitchen", "surface": "laundry basket"}], "abstain": false} ```

frames-uniform ✅ attempts=1 $0.010

```json {"reasoning": "The user previously instructed to put clothes (including linens/towels) into the laundry basket located in the kitchen.", "selections": [{"category": "hand towel", "room": "kitchen", "surface": "laundry basket"}], "abstain": false} ```

rollforward-v3 ✅ attempts=1 $0.002

```json {"reasoning": "Based on the instructions and memory, clothes/fabrics like a hand towel are placed in the laundry basket in the kitchen.", "selections": [{"category": "hand towel", "room": "kitchen", "surface": "laundry basket"}], "abstain": false} ```

rollforward-vis-pure ✅ attempts=1 $0.009

```json {"reasoning": "In the memory frames (t=770.0s to t=813.9s), a pink hand towel/cloth from the kitchen counter was placed into the laundry basket in the kitchen.", "selections": [{"category": "hand towel", "room": "kitchen", "surface": "laundry basket"}], "abstain": false} ```

2000:direct:cutlery — “Put this fork away.”

blind ❌ attempts=1 $0.002

```json { "reasoning": "Forks and other cutlery belong on the kitchen countertop.", "selections": [ { "category": "fork", "room": "kitchen", "surface": "countertop" } ], "abstain": false } ```

textlog ✅ attempts=1 $0.002

```json {"reasoning": "Kitchen utensils (such as knives and forks) are placed on the dining table in the living room based on the user's instructions.", "selections": [{"category": "fork", "room": "living room", "surface": "dining table"}], "abstain": false} ```

frames-uniform ✅ attempts=1 $0.011

```json {"reasoning": "The user instructed to put kitchen utensils on the dining table, which is located in the living room.", "selections": [{"category": "fork", "room": "living room", "surface": "dining table"}], "abstain": false} ```

rollforward-v3 ✅ attempts=1 $0.003

```json {"reasoning": "The user instructed to put kitchen utensils on the dining table in the living room. A fork is a kitchen utensil, so it belongs on the dining table.", "selections": [{"category": "fork", "room": "living room", "surface": "dining table"}], "abstain": false} ```

rollforward-vis-pure ❌ attempts=1 $0.011

```json {"reasoning": "Kitchen utensils and cutlery belong on the countertop in the kitchen, where items like the spatula and food preparation items are kept.", "selections": [{"category": "fork", "room": "kitchen", "surface": "countertop"}], "abstain": false} ```

2000:direct:food — “Put this tomato away.”

blind ❌ attempts=1 $0.003

```json {"reasoning": "Tomatoes are food/produce items and belong on the countertop in the kitchen.", "selections": [{"category": "tomato", "room": "kitchen", "surface": "countertop"}], "abstain": false} ```

textlog ✅ attempts=1 $0.002

```json { "reasoning": "The user previously instructed to put food on the desk, and food items like the potato were placed on the desk in the bedroom.", "selections": [ { "category": "tomato", "room": "bedroom", "surface": "desk" } ], "abstain": false } ```

frames-uniform ✅ attempts=1 $0.009

```json {"reasoning": "A tomato is food. The user instructed to put food on the desk, which is located in the bedroom.", "selections": [{"category": "tomato", "room": "bedroom", "surface": "desk"}], "abstain": false} ```

rollforward-v3 ✅ attempts=1 $0.002

```json {"reasoning": "The user instructed to put food on the desk, and food items like the irish potato and tomato were placed on the desk in the bedroom.", "selections": [{"category": "tomato", "room": "bedroom", "surface": "desk"}], "abstain": false} ```

rollforward-vis-pure ❌ attempts=1 $0.010

```json {"reasoning": "In the memory frames (e.g. at t=770.0s), food items including a tomato are kept on the countertop in the kitchen.", "selections": [{"category": "tomato", "room": "kitchen", "surface": "countertop"}], "abstain": false} ```

2000:multi — “Clear the countertop: a tomato, a fork, a hand towel and a pen are on it.”

blind ❌ ctl❌ attempts=1 $0.007

```json {"reasoning": "A pen is seen stored on the shelf in the kitchen, and a laundry basket is located in the kitchen for laundry items like the hand towel. There are no visual cues or memories for the tomato or fork.", "selections": [ {"category": "pen", "room": "kitchen", "surface": "shelf"}, {"category": "hand towel", "room": "kitchen", "surface": "laundry basket"} ], "abstain": false} ```

textlog ✅ ctl✅ attempts=1 $0.003

```json {"reasoning": "The user instructed that food items go on the desk in the bedroom, kitchen utensils go on the dining table in the living room, and clothes/linens go into the laundry basket in the kitchen. There is no rule for pens.", "selections": [{"category": "tomato", "room": "bedroom", "surface": "desk"}, {"category": "fork", "room": "living room", "surface": "dining table"}, {"category": "hand towel", "room": "kitchen", "surface": "laundry basket"}], "abstain": false} ```

frames-uniform ✅ ctl✅ attempts=1 $0.012

```json {"reasoning": "Based on the instructions heard during the episode, food goes on the desk in the bedroom, kitchen utensils go on the dining table in the living room, and clothes go into the laundry basket in the kitchen. There was no instruction for the pen.", "selections": [ {"category": "tomato", "room": "bedroom", "surface": "desk"}, {"category": "fork", "room": "living room", "surface": "dining table"}, {"category": "hand towel", "room": "kitchen", "surface": "laundry basket"} ], "abstain": false} ```

rollforward-v3 ✅ ctl✅ attempts=1 $0.004

```json {"reasoning": "Based on instructions and past actions: food (tomato) is placed on the desk in the bedroom; kitchen utensils (fork) go on the dining table in the living room; and clothes/textiles (hand towel) go into the laundry basket in the kitchen.", "selections": [ {"category": "a tomato", "room": "bedroom", "surface": "desk"}, {"category": "a fork", "room": "living room", "surface": "dining table"}, {"category": "a hand towel", "room": "kitchen", "surface": "laundry basket"} ], "abstain": false} ```

rollforward-vis-pure ❌ ctl❌ attempts=1 $0.014

```json {"reasoning": "In the memory frames, the robot transported a cloth/towel from the kitchen countertop to the laundry basket in the kitchen, and pens are kept on the low shelf in the kitchen.", "selections": [ {"category": "hand towel", "room": "kitchen", "surface": "laundry basket"}, {"category": "pen", "room": "kitchen", "surface": "shelf"} ], "abstain": false} ```

2001:control — “Put this pen away.”

blind 기권✅ attempts=1 $0.002

```json {"reasoning": "I have no prior memory or observed placement rules for pens in this house.", "selections": [], "abstain": true} ```

textlog 기권✅ attempts=1 $0.002

```json {"reasoning": "There is no record or user instruction in memory regarding pens or office supplies.", "selections": [], "abstain": true} ```

frames-uniform 기권✅ attempts=1 $0.010

```json {"reasoning": "I have not received any instructions or observed any prior placements for pens, so I must abstain.", "selections": [], "abstain": true} ```

rollforward-v3 기권❌ attempts=2 $0.004

```json {"reasoning": "In the memory summary, a pen was observed on the sofa in the living room.", "selections": [{"category": "pen", "room": "living room", "surface": "sofa"}], "abstain": false} ```

rollforward-vis-pure 기권✅ attempts=1 $0.011

```json {"reasoning": "There is no memory or record of pens or stationery items being placed or stored in this household.", "selections": [], "abstain": true} ```

2001:direct:cloth — “Put this hand towel away.”

blind ❌ attempts=1 $0.002

```json {"reasoning": "A hand towel is currently seen resting on the kitchen countertop, indicating that hand towels are kept on the countertop in the kitchen.", "selections": [{"category": "hand towel", "room": "kitchen", "surface": "countertop"}], "abstain": false} ```

textlog ✅ attempts=1 $0.002

```json {"reasoning": "A hand towel falls under the category of clothes/cloth items, which the user instructed to place on the desk in the bedroom.", "selections": [{"category": "hand towel", "room": "bedroom", "surface": "desk"}], "abstain": false} ```

frames-uniform ✅ attempts=1 $0.009

```json {"reasoning": "The user instructed to put clothes/fabrics from the countertop onto the desk, which is located in the bedroom.", "selections": [{"category": "hand towel", "room": "bedroom", "surface": "desk"}], "abstain": false} ```

rollforward-v3 ✅ attempts=1 $0.003

```json {"reasoning": "The user previously instructed clothes/fabrics to be moved to the desk, which is in the bedroom.", "selections": [{"category": "hand towel", "room": "bedroom", "surface": "desk"}], "abstain": false} ```

rollforward-vis-pure ✅ attempts=1 $0.010

```json {"reasoning": "In earlier frames (t=160.0s to t=210.0s), a hand towel/cloth was picked up from the floor and placed onto the desk in the bedroom.", "selections": [{"category": "hand towel", "room": "bedroom", "surface": "desk"}], "abstain": false} ```

2001:direct:cutlery — “Put this fork away.”

blind ❌ attempts=1 $0.002

```json {"reasoning": "Forks are kitchen utensils and belong on the kitchen countertop.", "selections": [{"category": "fork", "room": "kitchen", "surface": "countertop"}], "abstain": false} ```

textlog ✅ attempts=1 $0.002

```json { "reasoning": "The user instructed to put kitchen utensils into the laundry basket in the kitchen. A fork is a kitchen utensil, so it belongs in the laundry basket in the kitchen.", "selections": [ { "category": "fork", "room": "kitchen", "surface": "laundry basket" } ], "abstain": false } ```

frames-uniform △ attempts=1 $0.019

```json { "reasoning": "The user previously instructed to put kitchen utensils into the laundry basket. A fork is a kitchen utensil, and the laundry basket is in the bathroom.", "selections": [ { "category": "fork", "room": "bathroom", "surface": "laundry basket" } ], "abstain": false } ```

rollforward-v3 ✅ attempts=1 $0.002

```json {"reasoning": "Kitchen utensils such as a fork are instructed to be placed into the laundry basket in the kitchen.", "selections": [{"category": "fork", "room": "kitchen", "surface": "laundry basket"}], "abstain": false} ```

rollforward-vis-pure ❌ attempts=1 $0.009

```json {"reasoning": "Utensils and cutlery are kept on the kitchen countertop.", "selections": [{"category": "fork", "room": "kitchen", "surface": "countertop"}], "abstain": false} ```

2001:direct:food — “Put this tomato away.”

blind ❌ attempts=1 $0.002

```json {"reasoning": "Tomatoes and food ingredients belong in the kitchen on the countertop.", "selections": [{"category": "tomato", "room": "kitchen", "surface": "countertop"}], "abstain": false} ```

textlog ✅ attempts=1 $0.002

```json {"reasoning": "The user instructed to put food on the dining table in the living room, which was demonstrated with items like an apple and an egg.", "selections": [{"category": "tomato", "room": "living room", "surface": "dining table"}], "abstain": false} ```

frames-uniform △ attempts=1 $0.011

```json {"reasoning": "The user previously instructed to put food on the dining table, which is located in the kitchen.", "selections": [{"category": "tomato", "room": "kitchen", "surface": "dining table"}], "abstain": false} ```

rollforward-v3 ✅ attempts=1 $0.003

```json {"reasoning": "The user previously instructed to move food items to the dining table, which is located in the living room.", "selections": [{"category": "tomato", "room": "living room", "surface": "dining table"}], "abstain": false} ```

rollforward-vis-pure ❌ attempts=1 $0.011

```json {"reasoning": "In the episode, a tomato was picked up from the kitchen countertop and placed on the side table in the living room.", "selections": [{"category": "tomato", "room": "living room", "surface": "side table"}], "abstain": false} ```

2001:multi — “Clear the countertop: a hand towel, a tomato, a fork and a pen are on it.”

blind ❌ ctl✅ attempts=1 $0.002

```json { "reasoning": "There is no prior memory or demonstration recorded for where hand towels, tomatoes, forks, or pens belong in this house.", "selections": [], "abstain": true } ```

textlog ✅ ctl✅ attempts=1 $0.003

```json {"reasoning": "Based on prior instructions, clothes (such as the hand towel) go on the desk in the bedroom, food (such as the tomato) goes on the dining table in the living room, and kitchen utensils (such as the fork) go into the laundry basket in the kitchen. There is no recorded rule for office supplies like a pen.", "selections": [ {"category": "hand towel", "room": "bedroom", "surface": "desk"}, {"category": "tomato", "room": "living room", "surface": "dining table"}, {"category": "fork", "room": "kitchen", "surface": "laundry basket"} ], "abstain": false} ```

frames-uniform △ ctl✅ attempts=1 $0.011

```json {"reasoning": "Based on the earlier instructions, clothes go on the desk in the bedroom, food goes on the dining table in the kitchen, and kitchen utensils go into the laundry basket in the bathroom. There are no rules for the pen.", "selections": [ {"category": "hand towel", "room": "bedroom", "surface": "desk"}, {"category": "tomato", "room": "kitchen", "surface": "dining table"}, {"category": "fork", "room": "bathroom", "surface": "laundry basket"} ], "abstain": false} ```

rollforward-v3 ❌ ctl❌ attempts=1 $0.006

```json {"reasoning": "The user instructed to move clothes (hand towel) from the countertop to the desk (bedroom), food (tomato) to the dining table (living room), and kitchen utensils (fork) into the laundry basket (kitchen). A pen was previously seen on the sofa (living room).", "selections": [ {"category": "hand towel", "room": "bedroom", "surface": "desk"}, {"category": "tomato", "room": "living room", "surface": "dining table"}, {"category": "fork", "room": "kitchen", "surface": "laundry basket"}, {"category": "pen", "room": "living room", "surface": "sofa"} ], "abstain": false} ```

rollforward-vis-pure ❌ ctl✅ attempts=1 $0.015

```json {"reasoning": "In previous episodes, the hand towel (green cloth) was placed on the desk in the bedroom, and the tomato was moved from the countertop to the dining table in the living room. There is no prior memory of where forks or pens belong.", "selections": [ {"category": "hand towel", "room": "bedroom", "surface": "desk"}, {"category": "tomato", "room": "living room", "surface": "dining table"} ], "abstain": false} ```

6실패 사례

2000:control × blind

```json {"reasoning": "A yellow pen/pencil is stored in the wooden shelf located in the kitchen.", "selections": [{"category": "pen", "room": "kitchen", "surface": "shelf"}], "abstain": false} ```

원인·교훈: 마지막 프레임 배경 선반 위 노란 물체를 보고 '부엌 선반'. 메모리 없는 대조군이 현재 관측에서 답을 만든 것 — control은 blind에서 실패하는 게 정상이지만, 관측 이미지 속 소품이 답을 유도한 점은 여분/소품 정리 필요를 재확인.

2000:control × rollforward-v3

```json {"reasoning": "A pen was observed on the sofa in the living room.", "selections": [{"category": "pen", "room": "living room", "surface": "sofa"}], "abstain": false} ```

원인·교훈: 'A pen was observed on the sofa in the living room.' — strip되지 않은 씬 소품 Pen_3가 실행 동선(거실)에서 보였고 rf-v3 요약에 'sofa (with a pen and fabric)'로 남았다. 규칙이 아닌 목격 위치를 답함. 교훈: 미끼·홀드아웃 클래스 소품을 strip하지 않으면 control 행이 find 질의로 변질된다(데이터 쪽 전달).

2000:direct:food × rollforward-vis-pure

원인·교훈: 보관 8장 중 4장이 마지막 유닛(천→바구니)에 몰려 음식 규칙 장면이 없고, 마지막 프레임 조리대 위 여분 사과를 보고 '조리대'. 교훈: vis-pure의 최근성 편향 + 출발 상면 여분 배우가 질의 시점 미끼.

2001:direct:cutlery × frames-uniform

원인·교훈: 'laundry basket is in the bathroom' — 8장 균등 프레임엔 바구니가 부엌 바닥에 놓인 장면이 있어도 방을 식별할 단서(냉장고 등)가 프레임 안에 없었다. 클래스는 맞고 방만 틀림(△). 교훈: 균등 샘플의 방 특정 실패는 frames의 구조적 약점 — place와 class를 병기해야 한다.

2001:direct:food × frames-uniform

원인·교훈: 'dining table, which is located in the kitchen' — 실제 식탁은 거실. 위와 같은 방 특정 실패(△).

2001:direct:food × rollforward-vis-pure

원인·교훈: 'tomato … placed on the side table in the living room' — 거실 식탁을 사이드테이블로 오인(클래스 오답). 보관 프레임의 저해상도 원거리 장면에서 가구 종류 오인.

2001:multi × rollforward-v3

원인·교훈: 3개 규칙은 전부 정답인데 pen까지 '소파'로 답해 4개 → control ✗·다중집합 place 0. 원인은 위와 같은 펜 목격. 규칙 3개만 보면 정답(loc_f1 0.857).

7다음

데이터 쪽 전달(우선): 미끼·홀드아웃 클래스 씬 소품 strip(Pen_3 소파 위 실증) · 출발 상면 여분 배우 치우기 · pack_routine에 meta.rooms · L2 낱말 결함.
소품 strip된 편이 오면 control·pair 재측정. 그 전엔 control 수치를 본표에 쓰지 않는다.
다른 모델(sonnet·luna·8B)로 3편 격자 — 편당 ≈$0.6(flash 기준), 8B는 자체 서빙 $0.
frames-uniform의 방 특정 실패는 cap 16이나 seg-end 샘플로 완화되는지 ablation 후보(측정 조건 변경이라 사용자 결정).
rule_stmt 2단 귀속: rf-v3 요약은 규칙 문장('instructions to: 1. …')과 배치 사실을 둘 다 남긴다 — 유도 판정기는 지시문 보존 vs 관찰 배치 유도를 구분해야.

Routine read-track — 3편 합본, 쌍둥이 pair 채점

TL;DR

1세 편 — 규칙·유닛·누출·쌍둥이

2쌍둥이 짝 채점 (A6) — 같은 홀드아웃, 두 편 모두 place_set이어야 짝 정답

3전 편 집계 (15행 × 5전략) — 헤드라인은 누출 행(⚠️) 제외

4쌍둥이 메모리 실물

train_1652__2000

train_1652__2001

5원문 답 (2000/2001 — 3000은 전 과정 리포트 참조)

6실패 사례

2000:control × blind

2000:control × rollforward-v3

2000:direct:food × rollforward-vis-pure

2001:direct:cutlery × frames-uniform

2001:direct:food × frames-uniform

2001:direct:food × rollforward-vis-pure

2001:multi × rollforward-v3

7다음