← 목록

온라인 1칸 — place 사슬 전 구간 (소금통 → 부엌 조리대)

2026-09-17 · job 133509 (core-own, 14분 57초) · 본 런과 같은 설정을 셀 하나로
한 줄 요약 — place 사슬이 온라인에서 처음으로 끝까지 탔고, 마지막 한 칸(VLA 가 실제로 놓기)에서 실패했습니다.
소금통을 3번째 시도에 집고 → 든 채 부엌까지 이동(물체 동반 확인) → 에이전트가 pick_and_place 를 냄 → 하네스가 팔을 조리대 위 hand-off 자세로 옮김(preplace=true) → VLA 가 상판에 내려놓지 않고 팔을 끌어내려 바닥 쪽에서 그리퍼를 열었습니다. 소금통 최종 높이 z = 0.027(바닥), 조리대 상판은 0.938 입니다.
어디를 보시면 되나 — 턴 5
① preplace 접근(하네스) 영상: 팔이 든 물체와 함께 조리대 위로 가는 구간. 그리퍼는 건드리지 않습니다.
② 정책 실행(exo/손목) 영상: 여기부터가 VLA 입니다. 정책 프롬프트는 학습 형식 Place the salt shaker on the countertop.
③ 시작 프레임에서는 팔이 상판 위에 있고, 끝 프레임에서는 팔이 아래로 내려와 로봇 앞 바닥을 향합니다.
마지막 턴이 abort 인 이유
에이전트는 떨어진 소금통을 다시 집으려 했는데, 플래너 가드가 두 번 거부했습니다 — you already picked up the salt shaker here — move on. 한 번 들어 올린 GT 는 pick 후보에서 빼는 find 용 가드가 restore 에도 걸려 있습니다. 다만 바닥에 떨어진 물체는 구조적으로 다시 집기 어려워서, 가드를 풀어도 복구될 가능성은 낮습니다.

설정

셀restore · v1_train_2277__2 · textlog · gemini-3.7-flash
질의Pick up the salt shaker and place it in or on where it first was
GTsaltshaker_ae8db92b68885e8a5331b49cf1534319_1_0_6 → place_on
place_modevla (VLA 가 놓는다)
navigate_modedrive
VLApi05_mlspaces_finetune · molmobot
스텝 예산 / 사용5000 / 1038
턴 상한 / 사용50 / 7
비용$0.0432 (VLM 호출 8회)

초기 계획 (읽기 답 → 계획)

읽기 트랙 채점: place_set=1 (방·가구 모두 정답). 즉 기억은 맞았고 실행만 남은 칸입니다.
전략 textlog 는 계획 갱신 수단이 없어 plan_mode=fixed — 끝까지 이 계획 그대로입니다(재작성 0회).

결과

최종 판정실패
GT 물체를 들었나예
GT 수용체 위에 놓였나아니오
에피소드 종료 시 손에 든 것None
첫 navigate 가 GT 적중예
목표 지점까지 남은 거리1.529 m (xy)
종료 사유abort

턴 전수

턴 0 navigate도착  경로 11.506392756325228 m · 439 스텝
에이전트의 이유
The salt shaker is currently on the dining table in the living room. I need to navigate there to pick it up and return it to the countertop in the kitchen.
skillnavigate
category / surface / roomNone / None / living room
접지된 object_idsaltshaker_ae8db92b68885e8a5331b49cf1534319_1_0_6
접지된 receptacle_idNone
현재 방room_4
들고 있는 물체None
정책 프롬프트—
VLM 원문 응답
```json
{"note": null, "reasoning": "The salt shaker is currently on the dining table in the living room. I need to navigate there to pick it up and return it to the countertop in the kitchen.", "next": {"skill": "navigate", "category": "salt shaker", "surface": "dining table", "room": "living room"}}
```
이 턴에 VLM 이 받은 프롬프트 전문
{'texts': ['Your current view:', 'Your notes so far:\nHere are my running notes from the episode, written by me while acting (subgoal by subgoal):\n\n[context] [    0s–   94s] I picked up remote control #1 from the dresser in the bedroom and placed it on the clothes dryer in the bathroom.\n[context] [   94s–  256s] I tried to move the egg from the countertop in the kitchen to the drawer of the fridge in the kitchen, but the attempt failed.\n[context] [  256s–  387s] I picked up remote control #1 from the clothes dryer in the bathroom and placed it on the dresser in the bedroom.\n[context] [  387s–  491s] I picked up the salt shaker from the countertop in the kitchen and placed it on the dining table in the living room.\n[context] [  491s–  572s] I picked up the tissue box from the tv stand in the living room and placed it on the dining table in the living room.\n[context] [  572s–  690s] I picked up remote control #2 from the tv stand in the living room and placed it on the dresser in the bedroom.', 'Object names used in this house:\napple, basketball, bed, book, bottle, bowl, box, bread, cell phone, chair, clothes dryer, countertop, credit card, dining table, dresser, egg, faucet, fridge, garbage bag, garbage can, house plant, kettle, knife, ladle, laptop, mug, newspaper, painting, pan, paper towel roll, pencil, pillow, plunger, remote control, salt shaker, sink, soap bottle, sofa, spatula, spoon, spray bottle, statue, teddy bear, television, tissue box, toilet, toilet paper, tomato, tv stand, vase, watch', 'Room names used in this house:\nbedroom, bathroom, kitchen, living room', 'Your plan:\n1. [ ] put the salt shaker back — it is now on the dining table in the living room; return it to the countertop in the kitchen', 'Goal: Pick up the salt shaker and place it in or on where it first was', 'Reply with the JSON object only.'], 'n_images': 1}
관측 0
관측 1
이동 전
이동(A* 주행)
이동 후
턴 1 pick실패  128 스텝 · 그리퍼 전환 4
에이전트의 이유
The salt shaker is on the dining table in front of me. I will pick it up.
skillpick
category / surface / roomsalt shaker / None / None
접지된 object_idsaltshaker_ae8db92b68885e8a5331b49cf1534319_1_0_6
접지된 receptacle_idNone
현재 방room_7
들고 있는 물체None
정책 프롬프트Pick up the salt shaker
VLM 원문 응답
```json
{"note": null, "reasoning": "The salt shaker is on the dining table in front of me. I will pick it up.", "next": {"skill": "pick", "category": "salt shaker", "surface": null, "room": null}}
```
이 턴에 VLM 이 받은 프롬프트 전문
{'texts': ['Frames captured while your previous subgoal was executing:', 'Your current view:', 'Your notes so far:\nHere are my running notes from the episode, written by me while acting (subgoal by subgoal):\n\n[context] [    0s–   94s] I picked up remote control #1 from the dresser in the bedroom and placed it on the clothes dryer in the bathroom.\n[context] [   94s–  256s] I tried to move the egg from the countertop in the kitchen to the drawer of the fridge in the kitchen, but the attempt failed.\n[context] [  256s–  387s] I picked up remote control #1 from the clothes dryer in the bathroom and placed it on the dresser in the bedroom.\n[context] [  387s–  491s] I picked up the salt shaker from the countertop in the kitchen and placed it on the dining table in the living room.\n[context] [  491s–  572s] I picked up the tissue box from the tv stand in the living room and placed it on the dining table in the living room.\n[context] [  572s–  690s] I picked up remote control #2 from the tv stand in the living room and placed it on the dresser in the bedroom.', 'Object names used in this house:\napple, basketball, bed, book, bottle, bowl, box, bread, cell phone, chair, clothes dryer, countertop, credit card, dining table, dresser, egg, faucet, fridge, garbage bag, garbage can, house plant, kettle, knife, ladle, laptop, mug, newspaper, painting, pan, paper towel roll, pencil, pillow, plunger, remote control, salt shaker, sink, soap bottle, sofa, spatula, spoon, spray bottle, statue, teddy bear, television, tissue box, toilet, toilet paper, tomato, tv stand, vase, watch', 'Room names used in this house:\nbedroom, bathroom, kitchen, living room', 'Your plan:\n1. [ ] put the salt shaker back — it is now on the dining table in the living room; return it to the countertop in the kitchen', 'Goal: Pick up the salt shaker and place it in or on where it first was', 'Reply with the JSON object only.'], 'n_images': 4}
관측 0
관측 1
정책 실행(exo)
정책 실행(손목)
턴 2 pick실패  128 스텝 · 그리퍼 전환 8
에이전트의 이유
The gripper attempted to grasp the salt shaker but did not secure or lift it. The salt shaker remains on the table. Retrying the pick.
skillpick
category / surface / roomsalt shaker / None / None
접지된 object_idsaltshaker_ae8db92b68885e8a5331b49cf1534319_1_0_6
접지된 receptacle_idNone
현재 방room_7
들고 있는 물체None
정책 프롬프트Pick up the salt shaker
VLM 원문 응답
```json
{"note": null, "reasoning": "The gripper attempted to grasp the salt shaker but did not secure or lift it. The salt shaker remains on the table. Retrying the pick.", "next": {"skill": "pick", "category": "salt shaker", "surface": null, "room": null}}
```
이 턴에 VLM 이 받은 프롬프트 전문
{'texts': ['Frames captured while your previous subgoal was executing:', 'Your current view:', 'Your notes so far:\nHere are my running notes from the episode, written by me while acting (subgoal by subgoal):\n\n[context] [    0s–   94s] I picked up remote control #1 from the dresser in the bedroom and placed it on the clothes dryer in the bathroom.\n[context] [   94s–  256s] I tried to move the egg from the countertop in the kitchen to the drawer of the fridge in the kitchen, but the attempt failed.\n[context] [  256s–  387s] I picked up remote control #1 from the clothes dryer in the bathroom and placed it on the dresser in the bedroom.\n[context] [  387s–  491s] I picked up the salt shaker from the countertop in the kitchen and placed it on the dining table in the living room.\n[context] [  491s–  572s] I picked up the tissue box from the tv stand in the living room and placed it on the dining table in the living room.\n[context] [  572s–  690s] I picked up remote control #2 from the tv stand in the living room and placed it on the dresser in the bedroom.', 'Object names used in this house:\napple, basketball, bed, book, bottle, bowl, box, bread, cell phone, chair, clothes dryer, countertop, credit card, dining table, dresser, egg, faucet, fridge, garbage bag, garbage can, house plant, kettle, knife, ladle, laptop, mug, newspaper, painting, pan, paper towel roll, pencil, pillow, plunger, remote control, salt shaker, sink, soap bottle, sofa, spatula, spoon, spray bottle, statue, teddy bear, television, tissue box, toilet, toilet paper, tomato, tv stand, vase, watch', 'Room names used in this house:\nbedroom, bathroom, kitchen, living room', 'Your plan:\n1. [ ] put the salt shaker back — it is now on the dining table in the living room; return it to the countertop in the kitchen', 'Goal: Pick up the salt shaker and place it in or on where it first was', 'Reply with the JSON object only.'], 'n_images': 4}
관측 0
관측 1
정책 실행(exo)
정책 실행(손목)
턴 3 pick성공  44 스텝 · 그리퍼 전환 5
에이전트의 이유
The previous pick attempt knocked the salt shaker onto its side on the dining table but did not grasp it. I will try picking it again.
skillpick
category / surface / roomsalt shaker / None / None
접지된 object_idsaltshaker_ae8db92b68885e8a5331b49cf1534319_1_0_6
접지된 receptacle_idNone
현재 방room_7
들고 있는 물체saltshaker_ae8db92b68885e8a5331b49cf1534319_1_0_6
정책 프롬프트Pick up the salt shaker
VLM 원문 응답
```json
{"note": null, "reasoning": "The previous pick attempt knocked the salt shaker onto its side on the dining table but did not grasp it. I will try picking it again.", "next": {"skill": "pick", "category": "salt shaker", "surface": null, "room": null}}
```
이 턴에 VLM 이 받은 프롬프트 전문
{'texts': ['Frames captured while your previous subgoal was executing:', 'Your current view:', 'Your notes so far:\nHere are my running notes from the episode, written by me while acting (subgoal by subgoal):\n\n[context] [    0s–   94s] I picked up remote control #1 from the dresser in the bedroom and placed it on the clothes dryer in the bathroom.\n[context] [   94s–  256s] I tried to move the egg from the countertop in the kitchen to the drawer of the fridge in the kitchen, but the attempt failed.\n[context] [  256s–  387s] I picked up remote control #1 from the clothes dryer in the bathroom and placed it on the dresser in the bedroom.\n[context] [  387s–  491s] I picked up the salt shaker from the countertop in the kitchen and placed it on the dining table in the living room.\n[context] [  491s–  572s] I picked up the tissue box from the tv stand in the living room and placed it on the dining table in the living room.\n[context] [  572s–  690s] I picked up remote control #2 from the tv stand in the living room and placed it on the dresser in the bedroom.', 'Object names used in this house:\napple, basketball, bed, book, bottle, bowl, box, bread, cell phone, chair, clothes dryer, countertop, credit card, dining table, dresser, egg, faucet, fridge, garbage bag, garbage can, house plant, kettle, knife, ladle, laptop, mug, newspaper, painting, pan, paper towel roll, pencil, pillow, plunger, remote control, salt shaker, sink, soap bottle, sofa, spatula, spoon, spray bottle, statue, teddy bear, television, tissue box, toilet, toilet paper, tomato, tv stand, vase, watch', 'Room names used in this house:\nbedroom, bathroom, kitchen, living room', 'Your plan:\n1. [ ] put the salt shaker back — it is now on the dining table in the living room; return it to the countertop in the kitchen', 'Goal: Pick up the salt shaker and place it in or on where it first was', 'Reply with the JSON object only.'], 'n_images': 4}
관측 0
관측 1
정책 실행(exo)
정책 실행(손목)
턴 4 navigate도착  경로 3.7631912320099623 m · 171 스텝
에이전트의 이유
The salt shaker has been successfully picked up from the dining table. Now I need to navigate to the kitchen countertop where it was originally located.
든 채 중립 자세로 이동(하네스 · EEF 목표 → IK → 액션) — 물체가 같이 옴 · 그리퍼 기준 미끄러짐 0.0078 m · 접촉 True · 70 스텝
skillnavigate
category / surface / roomNone / None / kitchen
접지된 object_idcountertop_872e879ab9901259aa392532ca183b2d_1_0_6
접지된 receptacle_idNone
현재 방room_7
들고 있는 물체None
정책 프롬프트—
VLM 원문 응답
```json
{"note": null, "reasoning": "The salt shaker has been successfully picked up from the dining table. Now I need to navigate to the kitchen countertop where it was originally located.", "next": {"skill": "navigate", "category": "salt shaker", "surface": "countertop", "room": "kitchen"}}
```
이 턴에 VLM 이 받은 프롬프트 전문
{'texts': ['Frames captured while your previous subgoal was executing:', 'Your current view:', 'Your notes so far:\nHere are my running notes from the episode, written by me while acting (subgoal by subgoal):\n\n[context] [    0s–   94s] I picked up remote control #1 from the dresser in the bedroom and placed it on the clothes dryer in the bathroom.\n[context] [   94s–  256s] I tried to move the egg from the countertop in the kitchen to the drawer of the fridge in the kitchen, but the attempt failed.\n[context] [  256s–  387s] I picked up remote control #1 from the clothes dryer in the bathroom and placed it on the dresser in the bedroom.\n[context] [  387s–  491s] I picked up the salt shaker from the countertop in the kitchen and placed it on the dining table in the living room.\n[context] [  491s–  572s] I picked up the tissue box from the tv stand in the living room and placed it on the dining table in the living room.\n[context] [  572s–  690s] I picked up remote control #2 from the tv stand in the living room and placed it on the dresser in the bedroom.', 'Object names used in this house:\napple, basketball, bed, book, bottle, bowl, box, bread, cell phone, chair, clothes dryer, countertop, credit card, dining table, dresser, egg, faucet, fridge, garbage bag, garbage can, house plant, kettle, knife, ladle, laptop, mug, newspaper, painting, pan, paper towel roll, pencil, pillow, plunger, remote control, salt shaker, sink, soap bottle, sofa, spatula, spoon, spray bottle, statue, teddy bear, television, tissue box, toilet, toilet paper, tomato, tv stand, vase, watch', 'Room names used in this house:\nbedroom, bathroom, kitchen, living room', 'Your plan:\n1. [ ] put the salt shaker back — it is now on the dining table in the living room; return it to the countertop in the kitchen', 'Goal: Pick up the salt shaker and place it in or on where it first was', 'Reply with the JSON object only.'], 'n_images': 4}
관측 0
관측 1
이동 전
이동(A* 주행)
이동 후
턴 5 pick_and_place실패  128 스텝 · 그리퍼 전환 5 · 물체 놓침
에이전트의 이유
I have navigated to the kitchen countertop while holding the salt shaker. Now I will place it on the countertop.
skillpick_and_place
category / surface / roomsalt shaker / None / None
접지된 object_idsaltshaker_ae8db92b68885e8a5331b49cf1534319_1_0_6
접지된 receptacle_idcountertop_872e879ab9901259aa392532ca183b2d_1_0_6
현재 방room_6
들고 있는 물체None
정책 프롬프트Place the salt shaker on the countertop.
preplace{"steps": 22, "settle_steps": 1, "ik_max_dq": 1.020605904765966, "rejected": [], "final_l2": 0.037080999489616934, "z": 1.0883868, "preplace": true, "xy": [1.8445468611061575, 5.794176032139421], "top_z": 0.9383868000000001}
VLM 원문 응답
```json
{"note": null, "reasoning": "I have navigated to the kitchen countertop while holding the salt shaker. Now I will place it on the countertop.", "next": {"skill": "pick_and_place", "category": "salt shaker", "surface": "countertop", "room": null}}
```
이 턴에 VLM 이 받은 프롬프트 전문
{'texts': ['Frames captured while your previous subgoal was executing:', 'Your current view:', 'Your notes so far:\nHere are my running notes from the episode, written by me while acting (subgoal by subgoal):\n\n[context] [    0s–   94s] I picked up remote control #1 from the dresser in the bedroom and placed it on the clothes dryer in the bathroom.\n[context] [   94s–  256s] I tried to move the egg from the countertop in the kitchen to the drawer of the fridge in the kitchen, but the attempt failed.\n[context] [  256s–  387s] I picked up remote control #1 from the clothes dryer in the bathroom and placed it on the dresser in the bedroom.\n[context] [  387s–  491s] I picked up the salt shaker from the countertop in the kitchen and placed it on the dining table in the living room.\n[context] [  491s–  572s] I picked up the tissue box from the tv stand in the living room and placed it on the dining table in the living room.\n[context] [  572s–  690s] I picked up remote control #2 from the tv stand in the living room and placed it on the dresser in the bedroom.', 'Object names used in this house:\napple, basketball, bed, book, bottle, bowl, box, bread, cell phone, chair, clothes dryer, countertop, credit card, dining table, dresser, egg, faucet, fridge, garbage bag, garbage can, house plant, kettle, knife, ladle, laptop, mug, newspaper, painting, pan, paper towel roll, pencil, pillow, plunger, remote control, salt shaker, sink, soap bottle, sofa, spatula, spoon, spray bottle, statue, teddy bear, television, tissue box, toilet, toilet paper, tomato, tv stand, vase, watch', 'Room names used in this house:\nbedroom, bathroom, kitchen, living room', 'Your plan:\n1. [ ] put the salt shaker back — it is now on the dining table in the living room; return it to the countertop in the kitchen', 'Goal: Pick up the salt shaker and place it in or on where it first was', 'Reply with the JSON object only.'], 'n_images': 4}
preplace 직전
정책에 넘긴 시작 자세
관측 0
관측 1
정책 실행(exo)
정책 실행(손목)
preplace 접근(하네스)
턴 6 abortabort
에이전트의 이유
unified turn invalid twice: you already picked up the salt shaker here — move on
하네스가 되물음(재질의)
you already picked up the salt shaker here — move on
you already picked up the salt shaker here — move on
skillabort
category / surface / roomNone / None / None
접지된 object_idNone
접지된 receptacle_idNone
현재 방room_6
들고 있는 물체None
정책 프롬프트—
VLM 원문 응답
```json
{"note": null, "reasoning": "With the salt shaker in hand, I will place it back onto the countertop in the kitchen.", "next": {"skill": "pick_and_place", "category": "salt shaker", "surface": "countertop", "room": "kitchen"}}
```
이 턴에 VLM 이 받은 프롬프트 전문
{'texts': ['Frames captured while your previous subgoal was executing:', 'Your current view:', 'Your notes so far:\nHere are my running notes from the episode, written by me while acting (subgoal by subgoal):\n\n[context] [    0s–   94s] I picked up remote control #1 from the dresser in the bedroom and placed it on the clothes dryer in the bathroom.\n[context] [   94s–  256s] I tried to move the egg from the countertop in the kitchen to the drawer of the fridge in the kitchen, but the attempt failed.\n[context] [  256s–  387s] I picked up remote control #1 from the clothes dryer in the bathroom and placed it on the dresser in the bedroom.\n[context] [  387s–  491s] I picked up the salt shaker from the countertop in the kitchen and placed it on the dining table in the living room.\n[context] [  491s–  572s] I picked up the tissue box from the tv stand in the living room and placed it on the dining table in the living room.\n[context] [  572s–  690s] I picked up remote control #2 from the tv stand in the living room and placed it on the dresser in the bedroom.', 'Object names used in this house:\napple, basketball, bed, book, bottle, bowl, box, bread, cell phone, chair, clothes dryer, countertop, credit card, dining table, dresser, egg, faucet, fridge, garbage bag, garbage can, house plant, kettle, knife, ladle, laptop, mug, newspaper, painting, pan, paper towel roll, pencil, pillow, plunger, remote control, salt shaker, sink, soap bottle, sofa, spatula, spoon, spray bottle, statue, teddy bear, television, tissue box, toilet, toilet paper, tomato, tv stand, vase, watch', 'Room names used in this house:\nbedroom, bathroom, kitchen, living room', 'Your plan:\n1. [ ] put the salt shaker back — it is now on the dining table in the living room; return it to the countertop in the kitchen', 'Goal: Pick up the salt shaker and place it in or on where it first was', 'Reply with the JSON object only.'], 'n_images': 4}
관측 0
관측 1

에이전트가 턴마다 본 것(모아보기)

턴별 관측 프레임 (1 fps)