2026-09-07 · scene_bench · routine read-track
Routine read-track 파일럿 — 전 과정 한 편
contexts/type3/train_1652__3000 × gemini-3.7-flash × 5전략 · 답 형식 find와 동일 · 발화는 들은 시점 주입 · prompt v3+vocab3+routine-v1+textlog-rooms
TL;DR
- 파이프라인 관통: 3000편 × flash × 5전략 × 5행 = 25행, 파싱 실패 0, 78 call, $0.56 (textlog 5행은 방 포함 로그로 재질의). 게이트 G1·A2·시각 D②>blind 통과.
- frames-uniform·rf-v3 = 4/4 place_set (D② 관찰 규칙 '수저→거실 소파' 포함), textlog D① 2/2·D② 기권(정상), vis-pure 2/4, blind 0/4. 헤드라인(누출 행 제외 n=2: food·cutlery): frames 1.0 · rf-v3 1.0 · textlog 0.5 · vis 0.5 · blind 0.
- 발견 ①: 첫 실행에서 textlog가 방을 몰랐던 건 하네스 비대칭(type2 유닛 문장에 방 없음) — find/restore textlog처럼 로봇이 간 방을 붙이자(meta.rooms + tasks uid) D① place 0→1.0. ② 출발 조리대에 남은 여분 사과가 질의 시점 미끼(blind·vis '조리대'). ③ 씬 소품 Pen_3·Alarm_Clock_7(미끼·홀드아웃 클래스) strip 안 됨 — 데이터 쪽 전달.
25
rows
5
strategies
78
API calls
0.5613
cost $
8.4
wall min
1컨텍스트 — contexts/type3/train_1652__3000 (17701 스텝 ≈ 11.8분, 유닛 4)
규칙 3개: 실행 2(지시→로봇이 옮김, D①) + 관찰 1(무언급·무접촉, 실행 동선에서 지나가며 봄, D②). 발화는 프레임 1에 두 건.
| 규칙 | 클래스 | 수납처 | 출처 | k | 증거 |
|---|---|---|---|---|---|
| R1 | FOOD | Desk | self | 2 | [{"unit": "t1u0", "obj": "t1a1_Potato_6", "frames": [0, 4594]}, {"unit": "t1u1", "obj": "t1a3_Egg_29", "frames": [8091, |
| R2 | CLOTH | Basket | self | 2 | [{"unit": "t2u0", "obj": "t2a0_Cloth_11", "frames": [4595, 8090]}, {"unit": "t2u1", "obj": "t2a1_Cloth_10", "frames": [1 |
| R3 | CUTLERY | Sofa | observed | 4 | [{"kind": "pass_by", "obj": "o1e0_Knife_4", "units": ["t1u0", "t1u1", "t2u0", "t2u1"], "max_px": 760, "segments&quo |
들은 발화 (메모리 입력으로 주입되는 것)
- [t=0s] “Put the food from the countertop on the desk.”
- [t=0s] “Put the clothes from the countertop into the laundry basket.”
유닛 타임라인
| 유닛 | ok | 물체 | 구간 | L2 문장 |
|---|---|---|---|---|
| t1u0 | ✅ | t1a1_Potato_6 | 0–184s | Pick up the apple from the countertop and place it in or on the desk |
| t2u0 | ✅ | t2a0_Cloth_11 | 184–324s | Pick up the cloth from the countertop and place it into the basket |
| t1u1 | ✅ | t1a3_Egg_29 | 324–557s | Pick up the apple from the countertop and place it in or on the desk |
| t2u1 | ✅ | t2a1_Cloth_10 | 557–708s | Pick up the cloth from the countertop and place it into the basket |
영상 (16×)
exo_camera_1 스트립
관찰 증거 — 거실 소파 위 칼·국자·뒤집개 (실행 동선에서 보임, 4/4 유닛)
질의 시점 관측 (마지막 프레임) + 누출 검사 (마지막 5 s 픽셀 ≥200)
control: 안 보임direct:cloth: ⚠️ 증거 보임 (헤드라인 제외)direct:cutlery: 안 보임direct:food: 안 보임multi: ⚠️ 증거 보임 (헤드라인 제외)
2질의 행 (5) — 홀드아웃은 컨텍스트에 없던 하위 카테고리
| id | kind | query | GT (증거 인스턴스) | 출처 | k | leak |
|---|---|---|---|---|---|---|
control | routine_control | “Put this pen away.” | pen → 그대로 두기 | none | 0 | |
direct:cloth | routine_direct | “Put this hand towel away.” | hand towel → kitchen / laundry basket (인스턴스 1) | self | 2 | ⚠️ |
direct:cutlery | routine_direct | “Put this fork away.” | fork → living room / sofa (인스턴스 2) | observed | 4 | |
direct:food | routine_direct | “Put this tomato away.” | tomato → bedroom / desk (인스턴스 1) | self | 2 | |
multi | routine_multi | “Clear the countertop: a tomato, a hand towel, a fork and a pen are on it.” | tomato → bedroom / desk (인스턴스 1) hand towel → kitchen / laundry basket (인스턴스 1) fork → living room / sofa (인스턴스 2) pen → 그대로 두기 | mixed | 2 | ⚠️ |
채점: place_set = (방, 수납처)가 증거 인스턴스와 일치(find와 동일) · place_any = 그 클래스의 어느 인스턴스든 · class_ok = 수납처 클래스만 · control = 미끼(pen) 제외/기권. chance: loc 1/3 · place 1/54.
3전략별 메모리 실물 (cap 8, 30 s 청크)
textlog (오라클 — 발화 + 유닛 문장, 코드 생성)
[ 0s] The user said: "Put the food from the countertop on the desk."
[ 0s] The user said: "Put the clothes from the countertop into the laundry basket."
[ 0s– 184s] I picked up the potato from the countertop in the kitchen and placed it on the desk in the bedroom.
[ 184s– 324s] I picked up the cloth from the countertop in the kitchen and put it into the laundry basket in the kitchen.
[ 324s– 557s] I picked up the egg from the countertop in the kitchen and placed it on the desk in the bedroom.
[ 557s– 708s] I picked up the cloth from the countertop in the kitchen and put it into the laundry basket in the kitchen.
frames-uniform (8장 + heard 캡션)
rollforward-v3 (재작성 요약 — 최종, 24 calls / 24 청크)
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow and fabric, a sofa, and a desk holding a CD, an irish potato, and an egg. In the living room, I saw a tv stand holding a TV, a bowl, and a candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf holding a bottle. In the kitchen, I saw a countertop holding an apple, fabric, and a tomato. I picked up the fabric from the kitchen floor and put it into the laundry basket.
청크별 요약 이력
[chunk 0]
I was instructed to put the food from the countertop on the desk, and put the clothes from the countertop into the laundry basket. In the living room, I saw a tv stand holding a TV, bowl, and candle. In an adjacent room, I saw a shelf with a bottle. In the kitchen, I saw a laundry basket on the floor by the refrigerator. On the kitchen countertop, I observed an irish potato, lettuce, an apple, an egg, a tomato, and fabric (clothes). I moved toward the kitchen countertop.
[chunk 1]
I was instructed to put the food from the countertop on the desk, and put the clothes from the countertop into the laundry basket. In the living room, I saw a tv stand holding a TV, bowl, and candle. In an adjacent room, I saw a shelf with a bottle. In the kitchen, I saw a laundry basket on the floor by the refrigerator. On the kitchen countertop, I observed lettuce, an apple, an egg, a tomato, and fabric. I picked up the irish potato from the kitchen countertop.
[chunk 2]
I was instructed to put the food from the countertop on the desk, and put the clothes from the countertop into the laundry basket. In the living room, I saw a tv stand holding a TV, bowl, and candle. In an adjacent room, I saw a shelf with a bottle. In the kitchen, I saw a laundry basket on the floor by the refrigerator. On the kitchen countertop, I observed lettuce, an apple, an egg, a tomato, and fabric. In the bedroom with orange flooring, I observed a dresser and a desk. I am holding the irish potato and navigating toward the desk.
[chunk 3]
I was instructed to put the food from the countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room with orange flooring, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser and a desk. In the kitchen, I saw a laundry basket on the floor by the refrigerator, and a countertop holding lettuce, an apple, an egg, a tomato, and fabric. I am currently holding the irish potato and moving through the doorway.
[chunk 4]
I was instructed to put the food from the countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room with orange flooring, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the kitchen, I saw a laundry basket on the floor by the refrigerator, and a countertop holding lettuce, an apple, an egg, a tomato, and fabric. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD. I am currently carrying the irish potato towards the desk in the bedroom.
[chunk 5]
I was instructed to put the food from the countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room with orange flooring, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the kitchen, I saw a laundry basket on the floor by the refrigerator, and a countertop holding lettuce, an apple, an egg, a tomato, and fabric. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD. I am currently placing the irish potato onto the desk in the bedroom.
[chunk 6]
I was instructed to put the food from the countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room with orange flooring, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the kitchen, I saw a laundry basket on the floor by the refrigerator, and a countertop holding lettuce, an apple, an egg, a tomato, and fabric. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD. I placed the irish potato onto the desk in the bedroom and am now returning to the kitchen.
[chunk 7]
I was instructed to put the food from the countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room with orange flooring, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD and the irish potato that I placed there. In the kitchen, I saw a laundry basket on the floor and a countertop holding lettuce, an apple, an egg, a tomato, and fabric. I have returned to the kitchen countertop to pick up another item.
[chunk 8]
I was instructed to put the food from the countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room with orange flooring, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD and the irish potato that I placed there. In the kitchen, I saw a laundry basket on the floor and a countertop holding lettuce, an apple, an egg, a tomato, and other fabric. I picked up a piece of fabric from the kitchen countertop and moved toward the laundry basket.
[chunk 9]
I was instructed to put the food from the countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room with orange flooring, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD and the irish potato that I placed there. In the kitchen, I saw a refrigerator, a laundry basket on the floor, and a countertop holding lettuce, an apple, an egg, a tomato, and fabric. I took a piece of fabric from the kitchen countertop and placed it on the floor next to the laundry basket.
[chunk 10]
I was instructed to put the food from the countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room with orange flooring, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD and the irish potato that I placed there. In the kitchen, I saw a refrigerator, a countertop holding lettuce, an apple, an egg, and a tomato, and a laundry basket into which I placed a piece of fabric taken from the countertop.
[chunk 11]
I was instructed to put the food from the countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room with orange flooring, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD and the irish potato that I placed there. In the kitchen, I saw a refrigerator, a laundry basket containing fabric, and a countertop holding an apple, lettuce, an egg, fabric, and a tomato. I moved back to the kitchen and approached the apple on the countertop.
[chunk 12]
I was instructed to put the food from the countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room with orange flooring, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD and an irish potato. In the kitchen, I saw a refrigerator, a laundry basket containing fabric, and a countertop holding an apple, lettuce, an egg, fabric, and a tomato. I attempted to pick up the lettuce from the kitchen countertop.
[chunk 13]
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD and an irish potato. In the kitchen, I saw a refrigerator, a laundry basket containing fabric, and a countertop holding an apple, lettuce, fabric, and a tomato. I picked up the egg from the kitchen countertop.
[chunk 14]
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD and an irish potato. In the kitchen, I saw a refrigerator, a laundry basket containing fabric, and a countertop holding an apple, lettuce, fabric, and a tomato. I picked up the egg from the kitchen countertop and carried it into the bedroom towards the desk.
[chunk 15]
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD and an irish potato. In the kitchen, I saw a refrigerator, a laundry basket containing fabric, and a countertop holding an apple, lettuce, fabric, and a tomato. I picked up the egg from the kitchen countertop and am carrying it through the living room toward the bedroom desk.
[chunk 16]
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD and an irish potato. In the kitchen, I saw a refrigerator, a laundry basket containing fabric, and a countertop holding an apple, lettuce, fabric, and a tomato. I am carrying the egg into the bedroom toward the desk.
[chunk 17]
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow, a sofa, and a desk holding a CD, an irish potato, and an egg that I just placed on it. In the kitchen, I saw a refrigerator, a laundry basket containing fabric, and a countertop holding an apple, lettuce, fabric, and a tomato.
[chunk 18]
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the living room, I saw a tv stand holding a TV, bowl, and candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf with a bottle. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow and fabric, a sofa, and a desk holding a CD, an irish potato, and an egg. In the kitchen, I saw a refrigerator, a laundry basket containing fabric, and a countertop holding an apple, lettuce, fabric, and a tomato.
[chunk 19]
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow and fabric, a sofa, and a desk holding a CD, an irish potato, and an egg. In the living room, I saw a tv stand holding a TV, a bowl, and a candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf holding a bottle. In the kitchen, I saw a refrigerator, a laundry basket containing fabric, and a countertop holding an apple, lettuce, fabric, and a tomato. I have now navigated from the bedroom through the living room toward the kitchen.
[chunk 20]
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow and fabric, a sofa, and a desk holding a CD, an irish potato, and an egg. In the living room, I saw a tv stand holding a TV, a bowl, and a candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf holding a bottle. In the kitchen, I saw a laundry basket containing fabric, fabric on the floor, and a countertop holding an apple, lettuce, fabric, and a tomato. I have approached the kitchen countertop and grasped the lettuce.
[chunk 21]
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow and fabric, a sofa, and a desk holding a CD, an irish potato, and an egg. In the living room, I saw a tv stand holding a TV, a bowl, and a candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf holding a bottle. In the kitchen, I saw a laundry basket containing fabric, fabric on the floor, and a countertop holding an apple, fabric, and a tomato. I picked up the lettuce from the kitchen countertop and am carrying it toward the bedroom.
[chunk 22]
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow and fabric, a sofa, and a desk holding a CD, an irish potato, and an egg. In the living room, I saw a tv stand holding a TV, a bowl, and a candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf holding a bottle. In the kitchen, I saw a laundry basket containing fabric, fabric on the floor, and a countertop holding an apple, fabric, and a tomato. I am holding lettuce and moving through the rooms.
[chunk 23]
I was instructed to put the food from the kitchen countertop onto the desk, and put the clothes from the countertop into the laundry basket. In the bedroom, I saw a dresser holding an alarm clock and a vase, a bed holding a pillow and fabric, a sofa, and a desk holding a CD, an irish potato, and an egg. In the living room, I saw a tv stand holding a TV, a bowl, and a candle, a house plant on the floor, and a sofa holding fabric, a knife, a spoon, and a spatula. In another room, I saw a shelf holding a bottle. In the kitchen, I saw a countertop holding an apple, fabric, and a tomato. I picked up the fabric from the kitchen floor and put it into the laundry basket.
rollforward-vis-pure (모델이 고른 8장, 24 calls)
4질의 프롬프트와 원문 답
system prompt (routine v1) 전문
You are the memory module of a household robot. The earlier turns of this
conversation are your memory of one episode in this house — what the robot did,
what it saw, and what the user told it. The final image is the robot's CURRENT
view at the end of the episode.
The user now hands the robot an item to put away ("Put this ... away.") or asks
it to clear a surface. Answer WHERE that kind of item goes in THIS house: the
receptacle (surface) and the room it is in. Decide from your memory — where the
robot was told to put such things, where it put them, or where you saw such
things kept. The exact item may be one you have never seen; use what you
remember about items of the same kind. Nobody will tell you what is in this
house: recall it.
Output a single JSON object, nothing else:
{"reasoning": "...",
"selections": [{"category": "...", "room": "...", "surface": "..."}],
"abstain": false}
Rules:
- One entry in "selections" per item you are putting away, "category" being the
item as named in the query. If the query lists several items, answer each one
you have a rule for and leave it out if you have no memory of where such
things go. If you have no rule for any of them, set "abstain": true.
- "room": one of bathroom | bedroom | kitchen | living room.
- "surface": the receptacle the item should go on or in, from the list below.
- Use the room and surface names exactly as spelled in the lists.
Surfaces:
armchair | bed | cart | chair | clothes dryer | coffee table | countertop | desk
dining table | dresser | laundry basket | shelf | side table | sink | sofa | toilet
tv stand | washing machine | floor
질의 턴 = 마지막 프레임 1장 + Query: "…" + Reply with the JSON object only. 메모리 턴은 §3의 실물이 그대로 앞에 붙는다. 답 형식은 find와 동일 {category, room, surface}.
control — “Put this pen away.” · GT pen → 그대로 두기
blind place=0 any=0 cls=0 loc=0 ctl=0 abstain=0 attempts=1 $0.005
```json
{"reasoning": "Pens are stationery items that belong on a desk, typically located in the bedroom.",
"selections": [{"category": "pen", "room": "bedroom", "surface": "desk"}],
"abstain": false}
```
textlog place=0 any=0 cls=0 loc=0 ctl=1 abstain=1 attempts=1 $0.002
```json
{"reasoning": "The memory only contains instructions and actions regarding food items (going to the desk in the bedroom) and clothes (going to the laundry basket in the kitchen). There is no information or history about where pens go.",
"selections": [],
"abstain": true}
```
frames-uniform place=0 any=0 cls=0 loc=0 ctl=1 abstain=1 attempts=1 $0.010
```json
{"reasoning": "I have no memory or instructions regarding where pens belong in this house.",
"selections": [],
"abstain": true}
```
rollforward-v3 place=0 any=0 cls=0 loc=0 ctl=1 abstain=1 attempts=1 $0.003
```json
{"reasoning": "I have no memory or instructions regarding where pens or general writing/office supplies belong in this house.",
"selections": [],
"abstain": true}
```
rollforward-vis-pure place=0 any=0 cls=0 loc=0 ctl=0 abstain=0 attempts=1 $0.010
```json
{"reasoning": "In the living room (with the orange carpet), pens are kept on the white desk.",
"selections": [{"category": "pen", "room": "living room", "surface": "desk"}],
"abstain": false}
```
direct:cloth — “Put this hand towel away.” · GT hand towel → kitchen / laundry basket (인스턴스 1)
blind place=0 any=0 cls=0 loc=0 ctl=None abstain=1 attempts=1 $0.003
```json
{"reasoning": "I have no prior memory or demonstrated preference for where a hand towel belongs in this house.",
"selections": [],
"abstain": true}
```
textlog place=1 any=1 cls=1 loc=1 ctl=None abstain=0 attempts=1 $0.002
```json
{
"reasoning": "A hand towel is a cloth/fabric item. Earlier, clothes/cloth were placed into the laundry basket in the kitchen.",
"selections": [
{
"category": "hand towel",
"room": "kitchen",
"surface": "laundry basket"
}
],
"abstain": false
}
```
frames-uniform place=1 any=1 cls=1 loc=1 ctl=None abstain=0 attempts=1 $0.010
```json
{"reasoning": "The user previously instructed to put clothes/cloth items from the countertop into the laundry basket, which is located in the kitchen next to the refrigerator.",
"selections": [{"category": "hand towel", "room": "kitchen", "surface": "laundry basket"}],
"abstain": false}
```
rollforward-v3 place=1 any=1 cls=1 loc=1 ctl=None abstain=0 attempts=1 $0.004
```json
{
"reasoning": "A hand towel is considered a clothing/fabric item. According to the instructions, clothes should be put into the laundry basket located in the kitchen.",
"selections": [
{
"category": "hand towel",
"room": "kitchen",
"surface": "laundry basket"
}
],
"abstain": false
}
```
rollforward-vis-pure place=1 any=1 cls=1 loc=1 ctl=None abstain=0 attempts=1 $0.010
{"reasoning": "The laundry basket is located in the kitchen next to the refrigerator, where fabric items and towels are collected.", "selections": [{"category": "hand towel", "room": "kitchen", "surface": "laundry basket"}], "abstain": false}
direct:cutlery — “Put this fork away.” · GT fork → living room / sofa (인스턴스 2)
blind place=0 any=0 cls=0 loc=0 ctl=None abstain=0 attempts=1 $0.002
```json
{"reasoning": "A fork is kitchen cutlery and belongs on the countertop in the kitchen.",
"selections": [{"category": "fork", "room": "kitchen", "surface": "countertop"}],
"abstain": false}
```
textlog place=0 any=0 cls=0 loc=0 ctl=None abstain=1 attempts=1 $0.002
```json
{"reasoning": "The memory contains rules for food (desk in bedroom) and clothes (laundry basket in kitchen), but there is no instruction or past record for cutlery/forks.",
"selections": [],
"abstain": true}
```
frames-uniform place=1 any=1 cls=1 loc=1 ctl=None abstain=0 attempts=1 $0.010
```json
{"reasoning": "Utensils including a fork, spoon, and knife were seen placed on the sofa in the living room.",
"selections": [{"category": "fork", "room": "living room", "surface": "sofa"}],
"abstain": false}
```
rollforward-v3 place=1 any=1 cls=1 loc=1 ctl=None abstain=0 attempts=1 $0.002
```json
{"reasoning": "Utensils like a knife, spoon, and spatula were observed on the sofa in the living room, so a fork should go there as well.",
"selections": [{"category": "fork", "room": "living room", "surface": "sofa"}],
"abstain": false}
```
rollforward-vis-pure place=1 any=1 cls=1 loc=1 ctl=None abstain=0 attempts=1 $0.012
```json
{"reasoning": "In the memory frames, kitchen utensils and cutlery (such as the knife, spoon, and spatula) were organized onto the sofa in the living room.",
"selections": [{"category": "fork", "room": "living room", "surface": "sofa"}],
"abstain": false}
```
direct:food — “Put this tomato away.” · GT tomato → bedroom / desk (인스턴스 1)
blind place=0 any=0 cls=0 loc=0 ctl=None abstain=0 attempts=1 $0.002
```json
{"reasoning": "Tomatoes are placed on the kitchen countertop, where similar red tomatoes are already located.",
"selections": [{"category": "tomato", "room": "kitchen", "surface": "countertop"}],
"abstain": false}
```
textlog place=1 any=1 cls=1 loc=1 ctl=None abstain=0 attempts=1 $0.002
```json
{"reasoning": "The user instructed to put food items on the desk in the bedroom, as shown when potatoes and eggs were moved there. A tomato is a food item, so it belongs on the desk in the bedroom.",
"selections": [{"category": "tomato", "room": "bedroom", "surface": "desk"}],
"abstain": false}
```
frames-uniform place=1 any=1 cls=1 loc=1 ctl=None abstain=0 attempts=1 $0.014
```json
{"reasoning": "The user gave an instruction to put food on the desk. The desk is located in the bedroom.",
"selections": [{"category": "tomato", "room": "bedroom", "surface": "desk"}],
"abstain": false}
```
rollforward-v3 place=1 any=1 cls=1 loc=1 ctl=None abstain=0 attempts=1 $0.002
```json
{"reasoning": "The robot was instructed to put food onto the desk in the bedroom, where other food items like an irish potato and an egg were also located.",
"selections": [{"category": "tomato", "room": "bedroom", "surface": "desk"}],
"abstain": false}
```
rollforward-vis-pure place=0 any=0 cls=0 loc=0 ctl=None abstain=0 attempts=1 $0.009
```json
{"reasoning": "In the kitchen, food items such as tomatoes and other produce were seen and handled on the countertop.",
"selections": [{"category": "tomato", "room": "kitchen", "surface": "countertop"}],
"abstain": false}
```
multi — “Clear the countertop: a tomato, a hand towel, a fork and a pen are on it.” · GT tomato → bedroom / desk (인스턴스 1)
hand towel → kitchen / laundry basket (인스턴스 1)
fork → living room / sofa (인스턴스 2)
pen → 그대로 두기
blind place=0 any=0 cls=0 loc=0 ctl=1 abstain=1 attempts=1 $0.003
```json
{"reasoning": "There is no prior interaction or memory indicating where tomatoes, hand towels, forks, or pens belong in this household.",
"selections": [],
"abstain": true}
```
textlog place=0 any=0 cls=0 loc=0 ctl=1 abstain=0 attempts=1 $0.003
```json
{
"reasoning": "The user instructed to put food on the desk in the bedroom (as seen with potato and egg) and clothes/cloths into the laundry basket in the kitchen (as seen with cloth). A tomato is food and a hand towel is cloth. There are no rules for the fork or pen.",
"selections": [
{
"category": "tomato",
"room": "bedroom",
"surface": "desk"
},
{
"category": "hand towel",
"room": "kitchen",
"surface": "laundry basket"
}
],
"abstain": false
}
```
frames-uniform place=1 any=1 cls=1 loc=1 ctl=1 abstain=0 attempts=1 $0.019
```json
{"reasoning": "Food is placed on the desk in the bedroom based on previous instructions and placement. Clothes and towels go into the laundry basket located in the kitchen. Cutlery is kept on the sofa in the living room. There is no prior memory for stationary items like pens.",
"selections": [
{"category": "tomato", "room": "bedroom", "surface": "desk"},
{"category": "hand towel", "room": "kitchen", "surface": "laundry basket"},
{"category": "fork", "room": "living room", "surface": "sofa"}
],
"abstain": false}
```
rollforward-v3 place=1 any=1 cls=1 loc=1 ctl=1 abstain=0 attempts=1 $0.005
```json
{
"reasoning": "Food items (like the tomato) are instructed to be placed on the desk in the bedroom, where other food items are stored. Fabrics/clothes (like the hand towel) go into the laundry basket in the kitchen. Eating utensils (like the fork) are stored on the sofa in the living room alongside the knife, spoon, and spatula. There is no specific rule or observation for pens, so it is omitted.",
"selections": [
{
"category": "tomato",
"room": "bedroom",
"surface": "desk"
},
{
"category": "hand towel",
"room": "kitchen",
"surface": "laundry basket"
},
{
"category": "fork",
"room": "living room",
"surface": "sofa"
}
],
"abstain": false
}
```
rollforward-vis-pure place=0 any=0 cls=0 loc=0 ctl=1 abstain=0 attempts=1 $0.012
```json
{"reasoning": "Based on the observed placements in the house, cutlery such as spoons, knives, and spatulas are kept on the sofa in the living room.",
"selections": [{"category": "fork", "room": "living room", "surface": "sofa"}],
"abstain": false}
```
5채점표 · 게이트
✅ place_set · ◐ place_any(다른 인스턴스) · △ class_ok(수납처 클래스만) · ❌ 오답/기권 · ctl = 미끼 제외 여부. 헤드라인 집계는 누출 행(§1 ⚠️) 제외, "전체(누출 포함)" 열에 전 행 값.
| 행 | blind | textlog | frames-uniform | rollforward-v3 | rollforward-vis-pure |
|---|---|---|---|---|---|
control | 기권 ❌ | 기권 ✅ | 기권 ✅ | 기권 ✅ | 기권 ❌ |
direct:cloth | ❌ | ✅ | ✅ | ✅ | ✅ |
direct:cutlery | ❌ | ❌ | ✅ | ✅ | ✅ |
direct:food | ❌ | ✅ | ✅ | ✅ | ❌ |
multi | ❌ ctl✅ | ❌ ctl✅ | ✅ ctl✅ | ✅ ctl✅ | ❌ ctl✅ |
| 전략 | place_set | loc_set | place_any | class_ok | D① self | D② observed | place_set 전체(누출 포함) | control(multi) | control(abstain) | abstain | parse |
|---|---|---|---|---|---|---|---|---|---|---|---|
| blind | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 1.00 | 0.00 | 0.0 | 1.0 |
| textlog | 0.50 | 0.50 | 0.50 | 0.50 | 1.00 | 0.00 | 0.50 | 1.00 | 1.00 | 0.5 | 1.0 |
| frames-uniform | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.0 | 1.0 |
| rollforward-v3 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.0 | 1.0 |
| rollforward-vis-pure | 0.50 | 0.50 | 0.50 | 0.50 | 0.00 | 1.00 | 0.50 | 1.00 | 0.00 | 0.0 | 1.0 |
| 게이트 | 판정 | 근거 |
|---|---|---|
| G1 blind ≤ chance | ✅ | blind place_set 0.0 (chance 1/54) |
| A2 textlog: D①≈1 / D②≈blind | ✅ | self 1.0 · observed 0.0 |
| 시각 메모리 D② > blind | ✅ | frames/vis observed place_any vs blind |
| A5 control 기권 (textlog) | ✅ | control_abstain_ok 1.0 |
A2 구조 확인: textlog는 D② 관찰 규칙(수저)에서 '기록에 없다'며 기권했고, 시각 메모리 3종은 모두 거실 소파를 맞혔다 — 관찰 규칙은 영상으로만 풀린다는 설계 의도가 한 편에서 그대로 나타남.
rf-v3 최종 메모리에 '지시문 + 침실 책상 위 감자·계란 + 거실 소파 위 칼·스푼·뒤집개'가 모두 남아 D①·D② 규칙을 둘 다 유도했다(규칙 문장 자체는 없고 배치 사실로 답함).
textlog 정합: type1 textlog_v2는 slots의 source_room/place_room을 적는데 type2 유닛 문장엔 방이 없었다(첫 실행 food→'kitchen desk'). routine textlog를 'I picked up the potato from the countertop in the kitchen and placed it on the desk in the bedroom.'로 바꾸자(방 이름은 meta.rooms, 정답 행 아님) D① place 1.0 — 오라클 본래 자리. resume(type2)은 tasks를 안 채워 바이트 불변.
누출 플래그: cloth·multi 행은 마지막 5 s에 천이 바구니로 들어가는 장면이 픽셀 로그에 잡혀 제외됐다. 마지막 프레임 자체는 바구니 내부가 안 보이지만 5 s 창은 보수적으로 둔다.
A5(미끼 pen): textlog·frames·rf-v3는 기권했으나 blind는 상식(침실 책상), vis-pure는 '거실 흰 책상'을 지어냈다(거실에 책상 없음). 시각 메모리는 규칙 없는 클래스에서 confabulation 경향.
6실패 사례 (원문 + 원인 + 교훈)
control × blind
GT: pen → 그대로 두기
```json
{"reasoning": "Pens are stationery items that belong on a desk, typically located in the bedroom.",
"selections": [{"category": "pen", "room": "bedroom", "surface": "desk"}],
"abstain": false}
```
원인·교훈: 상식으로 '침실 책상'. 메모리 없는 대조군이 규칙 없는 클래스에서 기권하지 않음 — 미끼 control은 blind에서 실패하는 게 정상(사전확률 측정). 참고: 씬에 Pen_3(침실)이 strip되지 않고 남아 있어 데이터 판에서는 미끼 클래스 소품도 strip해야 한다.
control × rollforward-vis-pure
GT: pen → 그대로 두기
```json
{"reasoning": "In the living room (with the orange carpet), pens are kept on the white desk.",
"selections": [{"category": "pen", "room": "living room", "surface": "desk"}],
"abstain": false}
```
원인·교훈: 규칙 없는 pen을 '거실 흰 책상'에 두겠다고 답함 — 거실에 책상이 없다(책상은 침실). 원인: 8장 보관 프레임에 근거가 없는데도 기권하지 않고 장면을 지어냄. 교훈: 시각 메모리 전략의 A5 취약점; control 행을 본 측정에 유지해 전략별 confabulation률로 보고.
direct:cutlery × textlog
GT: fork → living room / sofa (인스턴스 2)
```json
{"reasoning": "The memory contains rules for food (desk in bedroom) and clothes (laundry basket in kitchen), but there is no instruction or past record for cutlery/forks.",
"selections": [],
"abstain": true}
```
원인·교훈: 정상 기권(A2). 텍스트 로그에 수저 관련 기록이 없어 '규칙 없음'으로 기권 — D② 관찰 규칙은 텍스트로 풀리지 않는다는 설계 의도 그대로. 실패가 아니라 오라클의 상한을 보여주는 사례.
direct:food × rollforward-vis-pure
GT: tomato → bedroom / desk (인스턴스 1)
```json
{"reasoning": "In the kitchen, food items such as tomatoes and other produce were seen and handled on the countertop.",
"selections": [{"category": "tomato", "room": "kitchen", "surface": "countertop"}],
"abstain": false}
```
원인·교훈: 'kitchen countertop'으로 답함. 원인: 질의 시점 마지막 프레임과 보관 프레임에 조리대 위 남은 여분 사과(t1a0/t1a2)가 붉게 보이고, 모델이 이를 tomato로 오인('similar red tomatoes are already located'). 교훈: 출발 상면에 남은 여분 배우가 질의 시점 미끼가 된다 — 데이터 판에서 여분을 숨김 상자로 치우거나(계획서 §4-3 증거 제거와 같은 편집) 라벨로 병기.
multi × rollforward-vis-pure
GT: tomato → bedroom / desk (인스턴스 1)
hand towel → kitchen / laundry basket (인스턴스 1)
fork → living room / sofa (인스턴스 2)
pen → 그대로 두기
```json
{"reasoning": "Based on the observed placements in the house, cutlery such as spoons, knives, and spatulas are kept on the sofa in the living room.",
"selections": [{"category": "fork", "room": "living room", "surface": "sofa"}],
"abstain": false}
```
원인·교훈: fork→거실 소파만 답하고 tomato·hand towel을 빠뜨림(count 1/3). 원인: 보관 8장 중 책상·바구니 배치 장면(#2749 이후 #8999·#13999)이 있었음에도 '관찰된 배치'로 수저만 인식. 교훈: vis-pure는 지시문 텍스트를 프레임 캡션으로 받지 못하고(heard는 ingest 청크에만) 보관 프레임에서 규칙을 재구성해야 해서 다중 규칙에서 약하다.
multi × textlog
GT: tomato → bedroom / desk (인스턴스 1)
hand towel → kitchen / laundry basket (인스턴스 1)
fork → living room / sofa (인스턴스 2)
pen → 그대로 두기
```json
{
"reasoning": "The user instructed to put food on the desk in the bedroom (as seen with potato and egg) and clothes/cloths into the laundry basket in the kitchen (as seen with cloth). A tomato is food and a hand towel is cloth. There are no rules for the fork or pen.",
"selections": [
{
"category": "tomato",
"room": "bedroom",
"surface": "desk"
},
{
"category": "hand towel",
"room": "kitchen",
"surface": "laundry basket"
}
],
"abstain": false
}
```
원인·교훈: tomato→침실 책상·hand towel→부엌 바구니는 정답, fork는 '규칙 없음'으로 생략(A2 정상) → 3개 중 2개라 다중집합 place_set 0, control ✓. 실패가 아니라 텍스트 오라클이 관찰 규칙(수저)을 구조적으로 못 푸는 사례. 진단 지표로 store 답별 부분 점수(loc_f1 0.8)가 이를 드러낸다.
7다음
- 2000/2001 쌍둥이 추가 → pair_ok(A6) 검증, 그다음 sonnet·luna·8B 3종으로 격자.
- 정식 pack_routine에 meta.rooms(방 번호→이름) 포함 요청 — textlog 방 표기의 근거 파일.
- 데이터 쪽 전달: 미끼·홀드아웃 클래스 씬 소품 strip(Pen_3·Alarm_Clock_7 잔존), 출발 상면 여분 배우가 질의 시점 미끼가 됨(숨김 상자로 치우기), 유닛 L2 문장 낱말 고정(apple) 결함.
- 정식 eval/routine·contexts/type3 도착 시 pack_routine_sample·build_routine_rows 폐기, 러너 그대로.
- rule_stmt 추출(2단 귀속)은 rf-v3 메모리가 규칙 문장 대신 배치 사실로 답하는 것이 확인돼 '사실→규칙 유도' 판정기 설계가 필요.