goal 이미지 후보 전수
2026-09-17 · restore:train_2853__2:q03 · soap dispenser → sofa
tagmem이 실제로 가진 이미지와 그 태그 · place 당시 세팅
한 줄 — 물체가 원위치에 있던 5,359프레임 동안 tagmem이 저장한 이미지는 20장이고,
그중 물체가 보이는 건 1장(351px = 프레임의 0.11%)이다. 그 1장의 캡션·태그에도
soap dispenser는 없다. tagmem이 이 물체를 이름으로 적은 이미지 3장은
전부 origin 구간 밖 — 즉 옮겨진 뒤의 모습이다.
20
origin 창 내 tagmem 이미지
1
물체가 보이는 장수
0.11%
그 1장의 물체 크기
0
물체를 이름으로 적은 장수
1 place 당시 세팅
질의: “Pick up the soap bottle and place it in or on where it first was” ·
원위치 (11.680, 9.945, 0.379) — 거실 소파 ·
현재 위치 (12.019, 3.833) — 다른 방 stand
navigate(sofa) 직후 — 물체를 든 채 소파 옆
preplace 완료 — 하네스가 고른
(11.614, 10.635) 위 15cm. 원위치와 0.69 m 차이하네스가 팔을 옮기는 구간
VLA place — 12스텝. 그 자리에서 놓는다
2 실제로 모델에 먹인 goal (전 프레임 중 최선)
frame 704 · 1,507 px (0.49%) — 5,359프레임 전수에서 물체 픽셀이 최대인 프레임.
로봇 팔이 화면을 채우고 soap dispenser는 식별되지 않는다. 이건 tagmem 저장분이 아니다(내 규칙이 전수에서 고른 것)
전 프레임을 다 뒤진 최선이 이것이다. 데이터셋
exo_camera_1은 로봇을 따라다니므로
“물체가 보인다” = “로봇이 코앞” = “팔이 시야를 막는다”가 된다.
3 tagmem이 origin 구간에 저장한 이미지 20장 — 태그 전부
물체 픽셀 내림차순. 첫 장을 뺀 19장은 전부 0 또는 미측정이다.
frame 0 · 351 px (0.11%)
obj remote control, tissue box, newspaper, knickknack
surf sofa room living room
“Objects resting on the sofa before pickup”
obj remote control, tissue box, newspaper, knickknack
surf sofa room living room
“Objects resting on the sofa before pickup”
frame 749 · 3 px (0.00%)
obj house plant, table lamp, vase, remote control
surf floor room living room
“Living room overview with plant on floor and desk in background”
obj house plant, table lamp, vase, remote control
surf floor room living room
“Living room overview with plant on floor and desk in background”
frame 749 · 3 px (0.00%)
obj table lamp, vase, bowl
surf desk room living room
“Desk with lamp, bowl, and vase in background”
obj table lamp, vase, bowl
surf desk room living room
“Desk with lamp, bowl, and vase in background”
frame 1249 · 0 px (0.00%)
obj wine bottle, bottle, plate, mug, dining table
surf dining table room kitchen
“Kitchen view through doorway showing dining table with items”
obj wine bottle, bottle, plate, mug, dining table
surf dining table room kitchen
“Kitchen view through doorway showing dining table with items”
frame 1499 · 0 px (0.00%)
obj wine bottle, bottle, plate, dining table
surf dining table room kitchen
“Dining table with wine bottle, atomizer, plate, and items; trash bag on floor”
obj wine bottle, bottle, plate, dining table
surf dining table room kitchen
“Dining table with wine bottle, atomizer, plate, and items; trash bag on floor”
frame 1499 · 0 px (0.00%)
obj (없음)
surf floor room kitchen
“Bag on kitchen floor near dining table”
obj (없음)
surf floor room kitchen
“Bag on kitchen floor near dining table”
frame 1999 · 0 px (0.00%)
obj toaster, egg, irish potato, tomato
surf countertop room kitchen
“Kitchen countertop with toaster and small round food items”
obj toaster, egg, irish potato, tomato
surf countertop room kitchen
“Kitchen countertop with toaster and small round food items”
frame 2999 · 0 px (0.00%)
obj wine bottle, atomizer, plate, cup, lettuce
surf dining table room kitchen
“Dining table with wine bottle, atomizer, plate, and cup, looking towards doorway to living room”
obj wine bottle, atomizer, plate, cup, lettuce
surf dining table room kitchen
“Dining table with wine bottle, atomizer, plate, and cup, looking towards doorway to living room”
frame 3499 · 0 px (0.00%)
obj vase, tissue box, box, laptop, knickknack
surf dresser room living room
“White dresser with blue vase, tissue box, and blue box”
obj vase, tissue box, box, laptop, knickknack
surf dresser room living room
“White dresser with blue vase, tissue box, and blue box”
frame 4249 · 0 px (0.00%)
obj pillow, fabric
surf bed room bedroom
“Bed with spotted cover/pillow near open door to kitchen”
obj pillow, fabric
surf bed room bedroom
“Bed with spotted cover/pillow near open door to kitchen”
frame 4499 · 0 px (0.00%)
obj toaster, tomato, cup
surf countertop room kitchen
“Kitchen with white cart and countertop holding toaster and food items”
obj toaster, tomato, cup
surf countertop room kitchen
“Kitchen with white cart and countertop holding toaster and food items”
frame 4499 · 0 px (0.00%)
obj bottle, vase
surf cart room kitchen
“White cart in kitchen with bottle/vase”
obj bottle, vase
surf cart room kitchen
“White cart in kitchen with bottle/vase”
frame 5249 · 0 px (0.00%)
obj dining table, chair, wine bottle, bottle, plate, lettuce
surf dining table room kitchen
“Robot holding egg/potato, showing dining table with bottles, plate, and food in background”
obj dining table, chair, wine bottle, bottle, plate, lettuce
surf dining table room kitchen
“Robot holding egg/potato, showing dining table with bottles, plate, and food in background”
frame 1500 · 미측정
obj wine bottle, atomizer, plate
surf dining table room kitchen
“Dining table with wine bottle, atomizer, plate, and items; bag on floor”
obj wine bottle, atomizer, plate
surf dining table room kitchen
“Dining table with wine bottle, atomizer, plate, and items; bag on floor”
frame 1750 · 미측정
obj wine bottle, atomizer
surf dining table room kitchen
“Dining table with wine bottle, atomizer, and small yellow item”
obj wine bottle, atomizer
surf dining table room kitchen
“Dining table with wine bottle, atomizer, and small yellow item”
frame 1750 · 미측정
obj (없음)
surf cart room kitchen
“White cart in kitchen”
obj (없음)
surf cart room kitchen
“White cart in kitchen”
frame 2500 · 미측정
obj toaster, egg, irish potato, tomato, knife, butter knife, cup
surf countertop room kitchen
“Kitchen countertop with toaster, round food items, knives, and cup”
obj toaster, egg, irish potato, tomato, knife, butter knife, cup
surf countertop room kitchen
“Kitchen countertop with toaster, round food items, knives, and cup”
frame 3250 · 미측정
obj (없음)
surf sofa room living room
“Living room view with blue sofa and white dresser”
obj (없음)
surf sofa room living room
“Living room view with blue sofa and white dresser”
frame 3750 · 미측정
obj vase, tissue box, box, laptop
surf dresser room bedroom
“White dresser with blue vase, tissue box, and book/box”
obj vase, tissue box, box, laptop
surf dresser room bedroom
“White dresser with blue vase, tissue box, and book/box”
frame 4750 · 미측정
obj cup, knife, fork, toaster, egg, irish potato, tomato
surf countertop room kitchen
“Kitchen countertop with cup, knife, fork, toaster, and two brown items (egg/potato)”
obj cup, knife, fork, toaster, egg, irish potato, tomato
surf countertop room kitchen
“Kitchen countertop with cup, knife, fork, toaster, and two brown items (egg/potato)”
4 tagmem이 이 물체를 이름으로 적은 이미지 — 전부 origin 밖
origin 구간은 [0, 5359]. 아래 셋은 그 뒤라 이미 옮겨진 뒤의 모습이다.
frame 7999 · origin 창 밖
obj=[bottle, wine bottle, plate, soapdispenser]
“Dining table with chairs, bottle, wine bottle, and plate”
obj=[bottle, wine bottle, plate, soapdispenser]
“Dining table with chairs, bottle, wine bottle, and plate”
frame 11999 · origin 창 밖
obj=[bottle, soapdispenser]
“Bathroom sink with small white bottle”
obj=[bottle, soapdispenser]
“Bathroom sink with small white bottle”
frame 12000 · origin 창 밖
obj=[bottle, soapdispenser]
“Bathroom sink with white bottle”
obj=[bottle, soapdispenser]
“Bathroom sink with white bottle”
검색의 문제가 아니다. tagmem 태그로
obj=soap bottle ∧ surf=sofa를 조회하면
아무것도 안 나온다 — 원위치 구간에 그렇게 태깅된 항목이 존재하지 않기 때문이다.
검색 규칙을 개선해도 없는 것을 꺼낼 수는 없다.
5 후보를 잘못 골랐나 — 20칸 전수 재확인
첫 칸이 운 나쁜 표본이었는지 확인하려고 20칸 각각의 “가능한 최선”(두 exo 카메라, origin 구간 전 프레임 중 물체 픽셀 최대)을 전부 렌더했다. 아래가 그 20장이다.
20칸의 최선 프레임. 20장 전부에서 로봇 팔이 화면을 지배한다 — 데이터셋 exo 카메라가 로봇에 실려 있기 때문
| 편 | 물체 | 캠 | 최대 픽셀 | 프레임 대비 |
|---|---|---|---|---|
| train_6242__1 | bottle | cam2 | 10,119 | 3.29% |
| train_9026__1 | bowl | cam1 | 4,966 | 1.62% |
| train_4584__0 | cell phone | cam2 | 2,639 | 0.86% |
| train_7152__1 | remote control | cam1 | 1,902 | 0.62% |
| train_6878__2 | cell phone | cam1 | 1,720 | 0.56% |
| train_6278__2 | remote control | cam1 | 1,698 | 0.55% |
| train_2853__2 | soap bottle | cam1 | 1,507 | 0.49% |
| train_2853__0 | paper towel roll | cam1 | 1,480 | 0.48% |
| train_2159__2 | potato | cam1 | 1,282 | 0.42% |
| train_6242__1 | apple | cam2 | 1,221 | 0.40% |
| train_7661__1 | candle | cam1 | 1,182 | 0.38% |
| train_7385__0 | remote control | cam1 | 913 | 0.30% |
| train_2788__2 | spray bottle | cam1 | 800 | 0.26% |
| train_188__2 | paper towel roll | cam1 | 787 | 0.26% |
| train_9909__2 | tomato | cam2 | 751 | 0.24% |
| train_4304__0 | knife | cam2 | 699 | 0.23% |
| train_3994__0 | pan | cam2 | 639 | 0.21% |
| train_9778__1 | spray bottle | cam1 | 368 | 0.12% |
| train_1847__1 | ladle | cam2 | 323 | 0.11% |
| train_9797__1 | candle | cam1 | 291 | 0.09% |
첫 칸은 전형적이었다. 중앙값 0.34%, 20칸 중 프레임의 1%를 넘는 건 2칸뿐이다
(train_6242__1 3.29%, train_9026__1 1.62%).
6 예외 한 칸 — 여기선 검색이 문제였다
train_6242__1:q05 (bottle → countertop). 소스에는 쓸 만한 프레임이 있다:
소스의 최선 — frame 6119, 8,280px. 병이 왼쪽 아래 카운터탑에 크고 선명하게, 팔에 가리지 않음. 쓸 만하다
tagmem이 저장한 최선 — frame 5749, 414px. 캡션은 “Robot placing cell phone onto dining table” — 다른 물체 이야기다
이 칸에서 tagmem은 20배 나은 프레임을 두고 414px짜리를 저장했다. 게다가 이 물체는
tagmem이 이름으로 적기도 했다(origin 창 안 9항목,
obj=[…, bottle], surf=countertop).
즉 일부 칸에서는 소스가 아니라 저장·선택이 병목이다 — 앞 절의 “검색 문제가 아니다”는 과한 주장이었고 여기서 정정한다.
train_7152__1:q00 (remote control → dining table)은 반대다: tagmem 저장분의 최선이 1,383px인데
그 캡션은 “Desk with watch”(obj=[watch])이고, tagmem 전체에서 remote control을 이름으로 적은 항목은 0개다.
소스의 최선도 1,902px(0.62%)에 불과하다.