Zero-WAM | Robbyant-Research/HumanGen 공개분 · generated human video ↔ source robot video
37,727 pair · 6개 소스: Simulation ICL(RoboTwin)과 Pre-train ICL External 5종입니다. 논문의 74.2K pair 중 in-house·real-world 서브셋은 공개되지 않았습니다.
생성 모델: 전체의 63%가 wan-local, 10%가 kling-v3입니다. RoboTwin은 전부 kling-v3입니다.
RoboTwin 재사용: 2,499 pair에 고유 human 영상은 994개입니다. 태스크당 50 pair를 채우려고 같은 태스크의 human 영상을 여러 robot episode에 붙였습니다.
품질 필터: 모든 pair가 semantic 5이고, physics는 3 또는 5만 남아 있습니다.
1. 분포
37,727
ICL pair
36,222
고유 human 영상
2,047
robot task 폴더
6
소스
소스별 pair 수
생성 모델 구성 (pair 비율)
kling-v3wan2.6-i2v-flashwan-localwan2.6 / 2.7-i2v
physics score 3인 pair 비율
소스별 수치 (차트의 표 버전 포함)
소스
pair
고유 human 영상
robot task
task당 pair (중앙값 / 최대)
kling-v3
wan2.6-flash
wan-local
wan2.6/2.7
physics 3
원본 일부 구간
human 영상 길이 (p10 / 중앙값 / p90)
캡션 단어 수: human 지시 / human dense / robot dense
RoboTwin 2.0
2,499
994
50
50 / 50
100.0%
0.0%
0.0%
0.0%
0.0%
0.0%
5.0 / 5.0 / 5.0 s (n=150)
16 / 16 / 15
AgiBot World
6,659
6,659
187
25 / 286
14.7%
84.8%
0.0%
0.5%
4.1%
85.3%
5.0 / 5.0 / 5.0 s (n=150)
13 / 169 / 59
RoboCOIN
6,102
6,102
238
14 / 646
3.0%
13.8%
83.0%
0.1%
9.9%
83.0%
5.0 / 6.5 / 10.0 s (n=150)
21 / 185 / 84
RoboMIND
6,072
6,072
392
19 / 27
1.1%
12.0%
86.8%
0.0%
7.0%
98.9%
5.0 / 5.0 / 8.0 s (n=150)
16 / 161 / 64
InternData-A1
11,230
11,230
1,161
2 / 254
0.9%
5.8%
93.2%
0.0%
10.6%
0.0%
5.0 / 5.0 / 5.0 s (n=150)
25 / 168 / 80
OXE (Bridge)
5,165
5,165
19
268 / 300
0.3%
41.2%
58.4%
0.1%
5.3%
99.7%
5.0 / 5.0 / 5.0 s (n=150)
12 / 160 / 72
human 영상 길이는 소스마다 무작위 150개의 mp4 헤더(mvhd)로 쟀습니다. robot task 폴더 수는 config의 경로(/videos/ 앞부분) 기준입니다.
2. 소스별 샘플 8개
소스마다 서로 다른 robot task에서 무작위로 뽑았고, 비율 5% 이상인 생성 모델과 physics 3 pair가 최소 1개씩 들어가게 했습니다(seed 0). RoboTwin은 학습 태스크(seen 43)에서만 뽑았습니다. 영상은 화면에 보일 때만 재생됩니다. 외부 5종의 로봇 영상은 공개 데이터가 ICL 구간만 잘라 episode 번호를 새로 매긴 clip이라, config의 원본 경로가 아니라 episode 메타데이터(episodes.jsonl의 icl)의 짝으로 찾았습니다.
RoboTwin 2.0 · Simulation ICL · 2,499 pair
pair가 많은 robot task: put_object_cabinet-demo_clean_collect_200-1000 (50), turn_switch-demo_clean_collect_200-1000 (50), click_bell-demo_clean_collect_200-1000 (50)
Human (generated)Robot · observation.images.cam_high
Human 지시
Use left hand to pick up the mug, rotate it, and place it in the middle of the table, then use right hand to pick up the mug and hang it on the rack.
Robot 태스크
Use left arm to pick the mug on the table, rotate the mug and put the mug down in the middle of the table, use the right arm to pick the mug and hang it onto the rack.
dense caption · sample 정보
Human Use left hand to pick up the mug, rotate it, and place it in the middle of the table, then use right hand to pick up the mug and hang it on the rack.
Robot Use left arm to pick the mug on the table, rotate the mug and put the mug down in the middle of the table, use the right arm to pick the mug and hang it onto the rack.
AgiBot World · Pre-train ICL (External) · 6,659 pair
pair가 많은 robot task: task_422 (286), task_390 (188), task_508 (183)
task_501
wan2.6-i2v-flashphysics 5fine
Human (generated)Robot · observation.images.head · 원본 episode 370 frames 2186–2310
Human 지시
Place the brown cloth held in the right hand onto the washstand.
Robot 태스크
Place the held cloth on the washstand with right arm.
dense caption · sample 정보
Human From a first-person perspective, the camera remains fixed. First, the right hand, which is gripping a brown soft cloth using the thumb and other four fingers, moves downwards and slightly forwards towards the flat white surface of the washstand, specifically targeting the empty area to the right of the sink basin and in front of the facial cleanser tube. Then, the right hand stops above the washstand surface and slowly opens its fingers to release the grip, allowing the soft brown cloth to fall and settle naturally onto the counter. Finally, the right hand withdraws slightly upwards, leaving the cloth behind. Final frame summary: The right hand is empty and hovering above the counter alongside the left hand, the crumpled brown cloth rests securely on the right side of the white washstand surface, while the facial cleanser tube, potted plant, faucet, soap dish, and toothbrush holder remain entirely undisturbed in their original locations.
Robot The cloth is held by the right robotic gripper above the washstand. The cloth is moved downwards and released by the right robotic gripper, falling onto the washstand. The cloth rests on the washstand.
Human (generated)Robot · observation.images.head · 원본 episode 213 frames 346–715
Human 지시
Grasp the waistband of the gray shorts with both hands and fold them in half towards the legs on the pale yellow bed sheet.
Robot 태스크
Grasp the waistband of gray shorts with both hands and fold it to the leg pale yellow bed sheet.
dense caption · sample 정보
Human Under a first-person perspective, in the center of a pale yellow bed sheet lies a pair of gray shorts, with the elastic waistband positioned on the right side and the legs on the left. In the upper left of the frame is a black remote control, and in the upper right is an open magazine. The operation begins with the left and right hands simultaneously reaching towards the waistband area on the right side of the gray shorts. The left hand pinches the top edge of the waistband using the thumb and index finger, while the right hand pinches the bottom edge of the waistband with its thumb and index finger. Next, both hands slightly lift the waistband and smoothly fold it horizontally to the left, bringing the waistband over to cover the leg area on the left side, effectively folding the shorts in half. The fabric undergoes natural soft-body deformation. In the final frame summary, the gray shorts are folded into a smaller, more compact square resting in the center of the bed sheet, the hands have released their grip and hover above or rest beside the shorts, and the remote control and open magazine remain entirely untouched in their original positions in the upper left and upper right corners.
Robot The gray shorts are lying flat on the pale yellow bed sheet, with the waistband on the right side and the legs on the left side. The two robotic grippers grasp the waistband of the gray shorts on the right side, lift it slightly, and fold it over to the left side, covering the legs. The gray shorts are folded in half and rest on the pale yellow bed sheet.
Human (generated)Robot · observation.images.head · 원본 episode 246 frames 263–322
Human 지시
Use the hammer held in the right hand to completely drive the pink column on the wooden toy bench to the bottom.
Robot 태스크
Use the held hammer in the right arm to completely drive pink column to the bottom.
dense caption · sample 정보
Human This is a first-person perspective video with a fixed camera. In the center of the frame is a piece of cardboard on the ground, upon which sits a wooden toy pounding bench with several colored wooden pegs. There are only two arms of the operator visible in the frame. The operator's right hand (wearing a dark long-sleeved shirt) tightly grips the handle of a wooden hammer with all five fingers, initially hovering above the upper right of the toy bench; the left hand rests naturally on the cardboard to the left. First, the right hand moves the wooden hammer to precisely align the hammerhead above the top of the protruding light pink peg located on the upper left of the toy bench. Then, the right hand swings the hammer downwards, repeatedly and smoothly striking the top of this light pink peg with the hammerhead. After each strike, the pink peg sinks further down into its hole. The right hand maintains a natural and smooth swinging motion with even force until the pink peg is driven completely to the bottom. Final frame summary: The light pink peg on the upper left of the toy bench is fully driven down, perfectly flush with the bench's surface; the right hand holds the hammer pausing above the toy bench, while the left hand remains resting on the cardboard on the left; the other colored pegs on the toy bench and all surrounding clutter remain exactly in their original positions.
Robot The hammer is held by the right robotic arm. The pink column is protruding from the wooden toy bench. The right robotic arm moves the hammer and strikes the pink column repeatedly, driving it downwards into the wooden toy bench. The pink column is driven completely to the bottom of the wooden toy bench. The hammer is held above the bench.
Human (generated)Robot · observation.images.head · 원본 episode 117 frames 253–317
Human 지시
Pull down the curtain cord with the right hand.
Robot 태스크
Pull down the curtain cord with the right arm.
dense caption · sample 정보
Human Presented from a first-person perspective, in the foreground is a wooden desk; on the left side are a ceramic mug and a potted succulent, directly in front is a window, and a beaded curtain cord hangs down vertically on the right edge of the window frame. First, the left arm wearing a light-colored sweater on the left remains stationary. Then, the right arm wearing a dark grey sweater lifts from the desk and reaches forward and to the right toward the beaded curtain cord hanging on the right side of the window. The index finger and thumb of the right hand pinch and grasp the middle section of the beaded cord. Next, the right hand pulls the cord straight down along the vertical axis for a distance. Final frame summary: The left arm remains resting stationary on the desk, the mug and succulent on the left are untouched and in their original positions, and the right arm is positioned on the right side of the window having just completed the downward pull, with the fingers still pinching the taut curtain cord.
Robot The curtain cord hangs vertically next to the window, with the robot's right gripper holding it. The curtain cord is pulled downwards by the robot's right gripper. The curtain cord is in a lower position, still held by the robot's right gripper.
Human (generated)Robot · observation.images.head · 원본 episode 234 frames 930–1128
Human 지시
Place the small pack of black tissues held in the right hand into the right blue target box.
Robot 태스크
Place the small pack of black tissues held in the right arm into the right blue target box.
dense caption · sample 정보
Human From a first-person perspective, the left hand rests passively on the desk. The right hand, gripping a small black tissue pack, moves to the right to hover over the blue plastic box. The right hand lowers, releases its grip, and places the tissue pack into the blue box, then withdraws. Final frame summary: The black tissue pack rests inside the blue box on the right, the left hand remains on the desk.
Robot The small pack of black tissues is held by the right robotic arm. The blue target box is placed on the right side of the table. The small pack of black tissues is moved downwards by the right robotic arm and then released, falling into the blue target box. The small pack of black tissues rests inside the blue target box.
Human (generated)Robot · observation.images.head · 원본 episode 150 frames 1070–1255
Human 지시
Place the held pomegranate into the plastic bag.
Robot 태스크
Place the held pomegranate into the plastic bag with left arm.
dense caption · sample 정보
Human First-person perspective. First, the left hand grips a red pomegranate with five fingers above a clear plastic bag in the center. Then, the left hand moves down, placing the fruit into the bag. Finally, the hand releases its grip and withdraws. Final frame: The pomegranate sits inside the central bag, the left hand is withdrawn, and other items are unchanged.
Robot A red pomegranate is held by the robot's left gripper. A clear plastic bag is positioned below the pomegranate in a wooden compartment. The robot's left gripper lowers the pomegranate into the clear plastic bag and opens to release it. The pomegranate falls into the bag. The pomegranate rests inside the clear plastic bag. The robot's left gripper is empty and positioned above the bag.
Human (generated)Robot · observation.images.head · 원본 episode 1068 frames 0–585
Human 지시
Fold the lower quarter of the blue round-neck short-sleeve garment to the center with both arms.
Robot 태스크
Fold the lower quarter of the blue round-neck short-sleeve garment to the center with both arms.
dense caption · sample 정보
Human From a first-person perspective, a light blue round-neck short-sleeve t-shirt is laid flat in the center of a wooden table. In the upper right corner of the table, there is a white mug and a stack of magazines. First, the left and right hands move downwards simultaneously to grasp the bottom edge of the t-shirt. Specifically, the left hand pinches the bottom left hem with the thumb and index finger, while the right hand simultaneously pinches the bottom right hem with the thumb and index finger. Then, both hands smoothly lift the bottom hem of the t-shirt simultaneously and fold it upwards. Fold the lower quarter of the t-shirt up so that the bottom edge is placed horizontally exactly at the center of the t-shirt. After completing the fold, both hands release the fabric and retract naturally backwards. Final frame summary: The left and right hands have moved away from the t-shirt, the light blue t-shirt remains in the center of the wooden table with its lower quarter neatly folded upwards to the center position, and the white mug and magazines in the upper right remain untouched in their original positions.
Robot A light blue round-neck short-sleeve t-shirt is laid flat on a teal surface. The robot's grippers grasp the bottom left and right edges of the t-shirt and fold the lower section upwards towards the center of the garment. The lower quarter of the t-shirt is folded up and rests at the center of the garment.
Human (generated)Robot · observation.images.head · 원본 episode 152 frames 2237–2748
Human 지시
Hand over the picked rectangular tray containing a glass and a croissant with both hands.
Robot 태스크
Hand over the picked rectangular tray with both arms.
dense caption · sample 정보
Human From a first-person perspective, both hands firmly grip the left and right raised edges of a rectangular tray patterned with lemon slices. The tray holds a glass of milky beverage, a wrapped straw, and a croissant in a clear plastic container. Slowly and steadily extend both arms forward horizontally to hand over the tray. During the forward movement, the tray remains level, and the glass and croissant remain stable. Finally, the frame ends with both hands holding the tray extended forward and suspended over the center front of the wooden counter, with the glass, wrapped straw, and croissant securely resting on it, while the coffee machine and other background cafe items remain undisturbed.
Robot The robot's two grippers are holding the left and right edges of a rectangular tray. On the tray, there is a glass filled with a milky beverage, a wooden stick, and a croissant in a plastic container. The robot arms move forward horizontally, carrying the tray and its contents steadily without dropping anything. The tray and its contents are held further forward by the robot arms, suspended in the air.
pair가 많은 robot task: Cobot_Magic_pour_drink (646), AIRBOT_MMK2_place_the_paper_drawer (194), RMC-AIDA-L_food_storage (163)
Cobot_Magic_prepare_breakfast
wan-localphysics 5fine
Human (generated)Robot · observation.images.cam_high_rgb · 원본 episode 169 frames 0–231
Human 지시
Pick up the croissant from the table and place it into the beige plate.
Robot 태스크
The robotic arm picks up the croissant from the table and places it into the beige plate.
dense caption · sample 정보
Human From a fixed first-person perspective, a person's two hands are resting on the bottom edge of a wooden table. On the left side of the wooden table lies a golden croissant, and on the right side sits an empty round beige plate. First, the left hand reaches forward to the left side of the table, firmly grasping the croissant with its fingers. Then, the left hand lifts the croissant off the table surface and moves it horizontally through the air towards the right, aiming directly above the beige plate. Next, the hand lowers the croissant and gently places it into the center of the beige plate. Finally, the left hand releases its grip and retracts slightly backwards. In the final frame, the left hand has moved away from the plate, and the golden croissant is stably resting inside the beige plate on the right side of the desk, while the original spot on the left side remains empty.
Robot The croissant is resting on the table surface outside the plate. The round beige plate is empty on the table. The croissant is grasped and lifted off the table surface, moved horizontally through the air above the table, and then lowered down. The croissant is stably placed inside the beige plate.
Human (generated)Robot · observation.images.cam_head_rgb · 원본 episode 17 frames 0–618
Human 지시
avoid the water bottle on the table and press the blue button
Robot 태스크
after avoiding the water bottle on the table, press the button.
dense caption · sample 정보
Human From a first-person perspective, two arms are resting on a white marble table, and the camera remains fixed. In the center of the table is a clear water bottle with a red cap, and directly behind it is a blue button. On the left are a folded napkin and receipt, and on the right are a teacup and spoon. First, the right hand lifts from the table and moves forward. Then, the right hand navigates around the right side of the bottle to avoid it. Finally, the right hand extends its index finger and presses down on the top center of the blue button. The left hand, wearing a watch, remains stationary on the left side of the table. In the final frame, the right index finger is pressing down on the blue button behind the water bottle, the water bottle remains upright and unmoved in the center of the table, the left hand rests on the left, and the napkin, receipt, teacup, and spoon remain in their original positions.
Robot A water bottle is placed on the table, and a blue button is located directly behind it. The right robot arm moves forward, bypasses the water bottle from the right side to avoid it, and presses the blue button. The water bottle remains stationary throughout the process. The right robot arm is pressing the blue button, and the water bottle remains in its original position on the table.
Human (generated)Robot · observation.images.cam_high_rgb · 원본 episode 812 frames 0–315
Human 지시
One hand picks up a yellow cup and positions it while the other hand pours green soda from a bottle into the cup.
Robot 태스크
The robotic arm picks up a yellow cup and positions it while another arm pours green soda from a bottle into the cup.
dense caption · sample 정보
Human The camera maintains a fixed third-person front view. First, one hand of the person reaches towards the right foreground, grasps the yellow cup on the table, lifts it, and moves it to the center above the table, holding it steady. Next, the other hand reaches towards the left foreground, grasps the green transparent soda bottle, lifts it, and positions it directly above the yellow cup in the center. Then, the hand holding the green soda bottle tilts it downwards to pour liquid into the yellow cup below; after pouring, the hand tilts the bottle back to an upright position. Finally, both hands lower the yellow cup and the green soda bottle back down, placing them onto the table. The final frame shows the yellow cup and the green soda bottle resting independently and securely on the table.
Robot A yellow cup and a green transparent soda bottle are resting independently on the table. First, one hand approaches and grasps the yellow cup, lifting it and moving it to a central position to hold it steady. Subsequently, the other hand grasps the green soda bottle, lifts it, and brings it over the positioned yellow cup. The hand then tilts the green soda bottle downwards to pour liquid into the yellow cup. After pouring, the bottle is tilted back upright. The yellow cup and the green soda bottle are returned to rest on the table.
Human (generated)Robot · observation.images.cam_high_left_rgb · 원본 episode 9 frames 0–216
Human 지시
Pick up the peeler and place it next to the orange on the table.
Robot 태스크
The robotic arm picks up the peeler and places it at the designated position next to the orange on the table.
dense caption · sample 정보
Human Under a fixed frontal perspective, a man wearing a grey shirt and a black apron stands behind a stainless steel counter. On the table, an orange is located in the lower center-left area, and a white peeler is resting on the middle-right side. First, one hand of the man, extending from the right side of the screen, moves towards the right side of the table and securely grasps the white peeler. Next, the hand smoothly lifts the peeler upward off the table surface. Then, holding the peeler, the hand moves towards the lower left of the frame across the table, approaching the location of the orange. Finally, the hand gently lowers the peeler onto the table directly next to the orange and releases its grip. The final frame summarizes the scene: the peeler rests securely on the table right next to the orange in the lower left-center area, and the hand stops moving.
Robot The peeler and the orange are resting separately on the flat table surface. The peeler is grasped, lifted off the table surface, moved through the air towards the location of the orange, and then smoothly lowered down. The peeler is resting securely on the table directly next to the orange.
Human (generated)Robot · observation.images.cam_high_rgb · 원본 episode 4 frames 2520–2880
Human 지시
Pick up the small white bowl from the white plate, place the chopsticks onto the white plate, and carry the blue spoon from the blue tray to the white plate
Robot 태스크
Place the chopsticks onto the white plate, Place the chopsticks onto the white plate, Pick up the blue spoon from the blue tray, Carry the blue spoon to the white plate
dense caption · sample 정보
Human The video shows a silver-haired woman in a green sweater standing behind a table organizing tableware. First, the hand on the left side of the frame reaches toward the white oval plate at the bottom-left of the desk, grasps the small white bowl resting on the plate, lifts it up, and moves it away from the white plate. Then, the hand on the right side of the frame reaches toward the long blue rectangular tray at the bottom-right of the desk, pinches the blue spoon inside the tray, and picks it up. Finally, this hand steadily carries the blue spoon over to the bottom-left white plate and places it down next to the blue chopsticks. Final frame summary: The small white bowl is successfully removed from the bottom-left white plate, and the blue spoon is successfully taken from the bottom-right blue tray and placed onto the bottom-left white plate.
Robot In the first frame, a pair of blue chopsticks and a small white bowl are resting on a white plate on the right. Another empty white plate is on the left. A long blue rectangular tray and white storage boxes are in the middle, while extra blue spoons and chopsticks are stored in holders on the left. Step 1: The robotic arm approaches the right white plate, grasps the small white bowl, and lifts it up to move it to the left. Step 2: In a subsequent operation scene, the robotic arm approaches a blue tray, grasps a blue spoon, and lifts it. The small white bowl is removed from the white plate and held by the robotic arm. The blue spoon is successfully picked up from the blue tray.
Human (generated)Robot · observation.images.cam_high_rgb · 원본 episode 406 frames 0–222
Human 지시
Pick up the orange from the table and place it into the wicker basket.
Robot 태스크
The robotic arm picks up the orange from the table and places it into the wicker basket.
dense caption · sample 정보
Human The video features a woman standing behind a concrete table, facing the camera. On the table, slightly left of center, there is an orange lump-like object. On the right side of the table, there is an empty rectangular brown wicker basket. First, one of the woman's hands reaches out to firmly grasp the orange object from the left side of the table. Then, the hand lifts the orange object upward, moves it smoothly through the air towards the right, and positions it directly above the center of the brown wicker basket. Finally, the hand releases its grip, allowing the orange object to fall downward into the basket, and the hand retracts. In the final frame, the orange object rests steadily inside the brown wicker basket on the right side of the table, completing the task.
Robot The orange object is resting directly on the table surface; the rectangular brown wicker basket is empty and placed nearby on the table. The orange object is grasped, lifted off the table surface, moved through the air to a position directly above the wicker basket, and then released to fall downward. The orange object is resting steadily inside the brown wicker basket.
Human (generated)Robot · observation.images.cam_high_rgb · 원본 episode 476 frames 0–369
Human 지시
pick up the egg yolk pastry from the table and place it in the white handbag
Robot 태스크
pick up the egg yolk pastry from the table and place it in the white handbag.
dense caption · sample 정보
Human Under a first-person perspective, there is a round white marble table in front of the camera. On the left side of the table lies a white canvas handbag with green handles, on the bottom right sits a yellow egg yolk pastry, and in the upper right area are clutters including a coffee cup on a saucer, a receipt, and napkins. First, the left hand reaches forward, using the thumb and index finger to pinch the upper opening edge of the white handbag on the left side of the table, slightly pulling it outward to keep the bag open. Simultaneously, the right hand reaches to the bottom right, using the thumb, index, and middle fingers to securely pinch the sides of the egg yolk pastry and lift it up smoothly. Then, the right hand holding the pastry moves to the left, positioning it right above the opening of the white handbag. Next, the fingers of the right hand release their grip, dropping the egg yolk pastry down into the interior of the handbag. Finally, both hands retract and rest flat on the table at the bottom of the frame. Final frame summary: The clutters such as the coffee cup, receipt, and napkins in the upper right remain completely untouched in their original positions, the yellow egg yolk pastry has been placed inside the white canvas handbag on the left side of the table, and both hands have returned to their initial resting positions at the front of the table.
Robot The egg yolk pastry and the white handbag are placed on the table. The robot arm approaches the egg yolk pastry, grasps it, lifts it up, moves it over the white handbag, and drops it inside. The egg yolk pastry is inside the white handbag.
Human (generated)Robot · observation.images.cam_high_rgb · 원본 episode 547 frames 0–422
Human 지시
Pick up the orange basket and place it horizontally in the middle of the table, then pick up the long bread and place it into the basket.
Robot 태스크
The robotic arm picks up the orange basket and places it horizontally in the middle of the table, then picks up the long bread and places it into the basket.
dense caption · sample 정보
Human From a fixed first-person perspective, there are two human arms wearing light grey long sleeves in the frame. On the desk, a long spiral bread is on the left side resting on folders, and an orange rectangular woven basket is on the right side resting on a keyboard. First, the right hand reaches to the right, firmly grasps the rim of the orange basket to lift it, moves it to the absolute center of the frame, adjusts its angle to be horizontal, and places it steadily on the desk. Then, the left hand reaches to the left, grasps the middle of the long spiral bread to lift it, moves it towards the center of the desk, and carefully places it down into the orange basket. Finally, the orange basket rests horizontally in the absolute center of the desk, with the long spiral bread securely placed inside it.
Robot In the first frame, an orange rectangular woven basket is placed diagonally on the left side of the table, and a long spiral bread is placed horizontally on the right side of the table. First, the orange basket is picked up, rotated, and placed horizontally in the center of the table. Following this, the long spiral bread is picked up from the right side and placed inside the orange basket. The orange basket rests horizontally in the middle of the table, with the long spiral bread resting securely inside it.
Human (generated)Robot · observation.images.camera_top · 원본 episode 217 frames 0–134
Human 지시
Pick up the light blue hexagonal cup, rotate it to an upright position, and place it back on the surface.
Robot 태스크
The robotic arm picks up the light blue hexagonal cup, rotates it to an upright position, and places it back on the table surface.
dense caption · sample 정보
Human From a fixed top-down perspective, a person wearing a yellow sweater is sitting at the bottom of the frame with two arms resting on the desk. First, the person's hand on the left side of the screen reaches forward towards the light blue hexagonal cup lying horizontally on its side on the slightly left part of the desk. Then, the hand firmly grasps the body of the light blue hexagonal cup and lifts it slightly off the white surface. Next, the hand rotates the cup 90 degrees in the air so that its opening faces straight upwards. Finally, the hand steadily places the upright cup down onto its flat base back at its original position on the left side of the desk, and then the hand releases and moves away. Final frame summary: The light blue hexagonal cup is standing upright on its flat base on the slightly left side of the white desk.
Robot The light blue hexagonal cup is lying horizontally on its side on the surface. A hand approaches and grasps the light blue hexagonal cup, lifts it up, and rotates it 90 degrees until its opening faces upwards. The light blue hexagonal cup is standing upright on its flat base on the surface.
Human (generated)Robot · observation.images.camera_top · 원본 episode 71 frames 0–123
Human 지시
pick up the purple sweet potato from the woven basket
Robot 태스크
pick up the sweet potato from the basket
dense caption · sample 정보
Human From a first-person perspective, there is a central woven basket containing a purple sweet potato, surrounded by a trowel, leaves, and stones. First, the right hand reaches to the central woven basket and grips the middle of the purple sweet potato using the thumb, index, and middle fingers. Next, the right hand lifts the sweet potato upwards out of the basket. Final frame summary: The right hand holds the sweet potato in mid-air, the central basket is empty, and all surrounding clutter remains unmoved.
Robot A purple sweet potato is placed inside a woven basket on a white table. The robot arm approaches the woven basket, grasps the purple sweet potato, and lifts it upwards out of the basket. The purple sweet potato is held in the air by the robot gripper, and the woven basket is left empty on the table.
Human (generated)Robot · observation.images.camera_front · 원본 episode 180 frames 0–698
Human 지시
Remove the silver lid from the steamer, place the brown potato into the steamer, and put the silver lid back on the steamer
Robot 태스크
Pick up the silver lid on the steamer, Carry the silver lid away from the steamer, Place the silver lid on the table, Pick up the brown potato on the table, Carry the brown potato to the steamer, Place the brown potato into the steamer, Pick up the silver lid on the table, Carry the silver lid to the steamer, Place the silver lid on the steamer
dense caption · sample 정보
Human The camera remains fixed, showing a woman sitting behind a kitchen counter. On the wooden counter on the left side of the frame, there is a light green multi-cooker covered with a glass lid featuring a light green knob. On the right side of the counter, there is a brown potato. First, the woman extends one hand to pinch the light green knob of the glass lid on the left side, lifts the lid vertically, and places it flat on the empty counter next to the cooker. Next, she uses one hand to firmly grasp the brown potato from the right side of the counter, moves it to the left, and places it inside the open light green pot. Finally, she extends one hand to pinch the knob of the glass lid on the counter again, lifts it, and steadily places it back onto the pot on the left side to cover it. In the final frame, the brown potato is located inside the light green multi-cooker pot, and the glass lid is firmly covering the pot.
Robot The silver lid is covering the pot section of the light green multi-cooker. The brown potato is resting on the table separately. First, the silver lid is lifted off the steamer and placed aside on the table. Second, the brown potato is picked up from the table, moved over, and placed inside the open steamer pot. Third, the silver lid is picked up again and placed back on top of the steamer pot to cover it. The brown potato is inside the steamer pot, and the silver lid is firmly covering the steamer pot.
Human (generated)Robot · observation.images.camera_top · 원본 episode 159 frames 0–114
Human 지시
Press the switch on the blue desk lamp to turn it on.
Robot 태스크
The robotic arm presses the switch on the blue desk lamp to turn it on.
dense caption · sample 정보
Human From a first-person perspective, two human arms wearing light blue long-sleeved shirts are resting on a dark grey desk, with the left hand on the left and the right hand on the right. On the right side of the desk, there is a blue desk lamp with a round base; a small black switch is clearly visible on the lamp's base. The camera remains fixed. First, the left hand remains completely stationary on the left side of the desk. The right hand lifts from the desk, extending the index finger while curling the other fingers. Then, the right hand moves towards the blue desk lamp on the right side. Next, the tip of the right index finger precisely targets and presses down the small black switch located on the base of the lamp. Finally, the right hand slightly retracts, and the blue desk lamp remains in its original position on the right but is now turned on, casting a bright light downwards from the lampshade onto the desk surface.
Robot The blue desk lamp is sitting on the surface and is currently turned off. An external force moves towards the lamp and presses the black button located on top of the lampshade. The blue desk lamp remains in its position but is now turned on, casting a bright light downwards onto the surface.
Human (generated)Robot · observation.images.camera_top · 원본 episode 286 frames 0–184
Human 지시
insert a bouquet of red flowers into a black cylindrical vase positioned on a white table surface
Robot 태스크
The robotic arm inserts a bouquet of red flowers into a black cylindrical vase positioned on a white table surface.
dense caption · sample 정보
Human From a first-person perspective, two arms wearing blue long-sleeve shirts are visible in the frame. In the initial state, the left hand is firmly gripping the stems of a bouquet of red tulips in the air, slightly to the left of the center, while the right hand rests flat on the right side of a white office desk. A black cylindrical vase sits upright on the right side of the desk. First, the left hand transports the bouquet horizontally to the right, positioning it directly above the black cylindrical vase. Then, the left hand aligns the bottom of the flower stems exactly over the top opening of the black vase. Next, the left hand slowly lowers the bouquet straight down, inserting the stems through the opening and fully into the vase. Finally, the left hand releases its grip and withdraws to the left. Final frame summary: The bouquet of red tulips is fully inserted into the black cylindrical vase, resting stably upright inside it on the right side of the white desk.
Robot The bouquet of red flowers is grasped and held in the air, while the black cylindrical vase is placed upright on the table. The bouquet of red flowers is transported horizontally towards the black cylindrical vase, then its stems are lowered downwards through the opening of the vase. The bouquet of red flowers is fully inserted into the black cylindrical vase, resting stably upright inside it.
Human (generated)Robot · observation.images.camera_top · 원본 episode 103 frames 0–690
Human 지시
Pick up the yellow starfruit from the mat and place it onto the light green plate.
Robot 태스크
The robotic arm picks up the yellow starfruit from the blue mat and places it onto the light green plate.
dense caption · sample 정보
Human From a first-person perspective, two human arms are resting on a white desk. On the desk, there is a yellow starfruit-shaped object situated on the bottom left, and a light green round plate located on the right side. First, the left hand moves to the bottom left and securely grasps the yellow starfruit-shaped object. Then, the left hand lifts the starfruit and moves horizontally to the right, positioning it directly above the light green plate on the right side of the desk. Finally, the left hand opens its fingers to release the object, placing it down into the plate. In the final frame, the yellow starfruit rests stably inside the light green plate on the right side of the desk, while the right hand remains inactive resting on the table.
Robot A light green round plate and a yellow flat circular object (starfruit) are resting separately on the surface. The hand reaches towards the yellow starfruit, grasps it securely, lifts it up, moves it over the light green plate, and opens to release it. The yellow starfruit rests stably inside the light green plate.
Human (generated)Robot · observation.images.camera_front · 원본 episode 98 frames 0–620
Human 지시
Pour the drink from the brown bottle into one blue cup, then pour the drink from the yellow bottle into the other blue cup.
Robot 태스크
The robotic arm pours the drink from the brown bottle into the right blue cup, then pours the drink from the yellow bottle into the left blue cup.
dense caption · sample 정보
Human Captured from a first-person egocentric perspective, on a grey office desk, there is a plastic bottle with dark brown liquid on the left, a clear plastic bottle with yellow liquid on the right, and a black rectangular tray in the dead center holding two identical light blue cups side-by-side. First, the left hand reaches out from the bottom left, firmly grips the brown bottle on the left desk, lifts it, and moves it directly above the left blue cup on the center tray. Then, the left hand tilts the brown bottle to pour the dark drink into the left blue cup, and after pouring, places the brown bottle exactly back to its initial position on the left before retracting. Next, the right hand reaches out from the bottom right, grips the clear plastic bottle with yellow liquid on the right desk, lifts it, and moves it directly above the right blue cup on the center tray. Subsequently, the right hand tilts this bottle to pour the yellow drink into the right blue cup, and finally places the clear bottle back to its original position on the right side of the desk, retracting the hand. In the final frame, both the brown bottle and the clear bottle with yellow liquid rest steadily in their original left and right positions on the desk, while the two light blue cups on the tray now successfully contain the dark brown and yellow drinks respectively.
Robot The brown bottle, the clear bottle with yellow liquid, and the two blue cups on a black tray are resting on the table. First, the brown bottle is picked up, moved above one of the blue cups, tilted to pour the drink, and then returned to its original place. Then, the clear bottle with yellow liquid is picked up, moved above the other blue cup, tilted to pour the drink, and finally placed back onto the table. Both bottles are back in their initial positions on the table, and the two blue cups remain on the tray.
Human (generated)Robot · observation.images.camera_top · 원본 episode 70 frames 0–107
Human 지시
Pick up the blue cube and place it on top of the pink cube.
Robot 태스크
The robotic arm picks up the blue cube and places it on top of the pink cube.
dense caption · sample 정보
Human The video begins in a workshop setting where a man in safety glasses stands behind a wide metal table. On the table, slightly to the left of the center, rests a blue cube, while a pink cube sits slightly to the right of the center. The man currently has his two hands resting flat on the table on the left and right sides of the frame. First, the hand located on the left side of the frame reaches towards the center and firmly grasps the sides of the blue cube with its fingers. Then, this hand lifts the blue cube vertically off the table and moves it horizontally to the right through the air until it is positioned directly above the pink cube. Next, the hand slowly lowers the blue cube downward, placing its bottom surface squarely and gently onto the top surface of the pink cube. Finally, the hand releases its grip, moves away from the cubes, and retracts slightly. In the final frame, the blue cube is stably stacked directly on top of the pink cube on the slightly right side of the metal table, with the man's hands resting naturally, successfully completing the stacking task.
Robot The blue cube and the pink cube are resting separately on the flat surface. The blue cube is lifted vertically from its resting position, moved horizontally through the air until it is positioned directly above the pink cube, and finally lowered down onto it. The blue cube is stably stacked directly on top of the pink cube.
Human (generated)Robot · images.rgb.head · 원본 episode 312 frames 0–613
Human 지시
Pick up the lotus root and place it on the pad with one hand and then pick up the calculator and place it on the pad with one hand
Robot 태스크
Pick up the lotus root and place it on the pad with left arm and then pick up the calculator and place it on the pad with left arm
dense caption · sample 정보
Human From a first-person egocentric perspective, a person is sitting at a white desk, with both hands initially resting at the bottom edge of the frame. In the center of the desk is a pink pad with a rabbit design. To the middle-left of the pad is a small pink rectangular block (calculator), and to the upper-right is a yellowish lotus root. First, the right hand reaches towards the upper right, grasps the yellowish lotus root with its fingers, lifts it, moves it to the central pink pad, and gently places it down. Next, the right hand moves to the middle left, grasps the small pink rectangular block, lifts it, moves it onto the pink pad, and places it next to the lotus root, while the left hand remains resting stationary at the bottom left edge throughout the entire process. Finally, the video ends with a summary frame where both the yellowish lotus root and the small pink rectangular block are resting securely on the central pink pad.
Robot The lotus root (yellowish lumpy object), the calculator (small pink rectangular block), and the pink pad with a rabbit design are all resting separately on the table. First, the lotus root is grasped by the robot, lifted, moved, and placed onto the pink pad. After releasing the lotus root, the robot moves towards the calculator to grasp it for the next placement action. The lotus root is resting on the pink pad, and the calculator is intended to be placed on the pad as well.
Human (generated)Robot · images.rgb.head · 원본 episode 683 frames 0–199
Human 지시
Push the white storage drawer unit with left hand
Robot 태스크
Push the storagefurniture_middle with left arm
dense caption · sample 정보
Human The video adopts a first-person perspective. In the center of the desk is a white storage drawer unit. On the left side of the frame are colorful folders, on the right is a pen holder, and in the top right corner is a keyboard. First, the left hand, initially resting at the bottom left, lifts from the desk and extends forward towards the white storage drawer unit in the center, while the right hand remains completely stationary at the bottom right throughout the video. The left hand opens flat, making contact with the front flat surface of the drawer below its handle using the palm and extended fingers. Then, the left arm steadily exerts force forward, smoothly pushing the white storage drawer unit straight backward along the desk surface for a certain distance. After the pushing action is complete, the left hand retracts back to its starting position on the desk. In the final frame, the white storage drawer unit is positioned further back at the upper-center area of the desk, while the colorful folders on the left, the pen holder on the right, and the keyboard at the top right remain completely unmoved in their original positions, with two human arms resting quietly in the lower portion of the frame.
Robot The white storage drawer unit is stationary on the table. The robot arm makes contact with the front of the white storage drawer unit and pushes it forward across the table. The white storage drawer unit is positioned further back on the table, away from its initial location.
Human (generated)Robot · images.rgb.head · 원본 episode 481 frames 0–518
Human 지시
Pick up the yellow bread block and place it on the marble patterned square pad, then pick up the black remote control and place it on the pad
Robot 태스크
Pick up the bread and place it on the pad with right arm and then pick up the remote control and place it on the pad with left arm
dense caption · sample 정보
Human From a first-person perspective, two human hands wearing white long-sleeved shirts are resting in the foreground on a white desk. First, the left hand reaches forward to the left to grasp the yellow cube-shaped bread block located on the left side of the desk. The left hand lifts the bread block, moves it horizontally to the right, and places it down onto the dark marble patterned square pad located on the right side of the desk, then releases it. Next, the right hand reaches forward to the center of the desk to grasp the black remote control. The right hand lifts the remote control, moves it to the right, and places it onto the same marble patterned square pad, positioning it right next to the yellow bread block, and then withdraws. Finally, both the yellow bread block and the black remote control are resting steadily on top of the dark marble patterned square pad on the right side of the frame.
Robot The yellow bread block, the black remote control, and the dark marble patterned square pad are placed separately on the table surface. Step 1: The yellow bread block is picked up, lifted, and placed onto the dark marble patterned square pad. Step 2: The black remote control is picked up, lifted, and placed onto the dark marble patterned square pad near the bread block. Both the yellow bread block and the black remote control are resting on top of the dark marble patterned square pad.
Human (generated)Robot · images.rgb.head · 원본 episode 80 frames 0–496
Human 지시
Pick up the long brown sweet potato and place it into the light blue dish, then pick up the whitish egg and place it into the same light blue dish
Robot 태스크
Pick up the sweet potato and place it in the dish with left arm and then pick up the egg and place it in the dish with left arm
dense caption · sample 정보
Human The video is shot from a first-person egocentric perspective. The camera remains fixed, showing a glass desk in a modern office with office chairs in the background. Two human hands are visible resting on the left and right edges of the desk respectively. On the left side of the desk sits a shallow light blue dish. On the right side, there is a long brown sweet potato positioned slightly back from the center, and a whitish oval egg positioned closer to the lower-right corner. First, the right hand reaches forward to firmly grasp the long brown sweet potato on the middle-right. Then, the right hand lifts the sweet potato, moves horizontally to the left, and gently places it into the light blue dish. Next, the right hand moves back towards the lower-right area of the desk to gently grasp the whitish egg. Finally, the right hand lifts the egg, moves to the left again, and carefully places it inside the light blue dish next to the sweet potato. During the entire process, the left hand remains perfectly stationary on the left edge of the desk. Final frame summary: Both the long brown sweet potato and the whitish egg are resting together inside the shallow light blue dish on the left side of the glass desk, with the right hand having completed the placement.
Robot The long brown sweet potato, the whitish oval egg, and the shallow light blue dish are resting separately on the table. First, the sweet potato is picked up from its initial position on the table, transported through the air, and released into the light blue dish. Subsequently, the egg is picked up from the table, transported over to the dish, and placed inside it next to the sweet potato. Both the sweet potato and the egg are resting together inside the light blue dish on the table.
Human (generated)Robot · images.rgb.head · 원본 episode 39 frames 0–492
Human 지시
Pick up the tissue and place it on the basket, then pick up the carrot and place it on the basket
Robot 태스크
Pick up the tissue and place it on the basket with right arm and then pick up the carrot and place it on the basket with right arm
dense caption · sample 정보
Human From a first-person egocentric perspective, we see a person's left and right hands resting on a wooden table. On the table, a closed green rectangular woven basket is positioned slightly left of center, a carrot is located on the right side of the table, and a small tissue pack is placed towards the upper right corner. First, the right hand moves forward to the upper right, securely grasps the small tissue pack with the fingers, lifts it, moves leftward, and places it stably on top of the green woven basket's lid. Then, the right hand moves back down to the right side of the table, firmly grips the carrot, lifts it, moves leftward, and places it on the basket's lid alongside the tissue pack. Finally, in the static final frame, both the small tissue pack and the carrot are resting stably on top of the closed green woven basket's lid.
Robot The small rectangular tissue pack, the carrot, and the rectangular woven basket are resting separately on a flat surface. First, the tissue pack is grasped, lifted, and placed onto the lid of the woven basket. Next, the carrot is grasped, lifted, and placed onto the lid of the basket alongside the tissue pack. Both the tissue pack and the carrot are resting on top of the woven basket's lid.
Human (generated)Robot · images.rgb.head · 원본 episode 405 frames 0–496
Human 지시
Pick up the toothpaste with the left hand and the wooden pestle with the right hand simultaneously, place the toothpaste into the white bowl, and then place the wooden pestle into the white bowl
Robot 태스크
Pick up the bamboo shoots with the right arm and pick up the tooth paste with the left arm meanwhile and place tooth paste in the bowl and then place bamboo shoots in the bowl
dense caption · sample 정보
Human From a first-person perspective, a person is sitting in front of an outdoor stone table. A white toothpaste tube is on the left side of the table, a wooden pestle is on the right side, and a white bowl is positioned in the upper center. In the foreground, there is a left arm wearing a silver watch and a right arm wearing a woven bracelet. First, the left hand reaches forward to the left to grasp and lift the toothpaste tube, while simultaneously, the right hand reaches forward to the right to grasp and lift the wooden pestle. Next, the left hand moves the toothpaste to the upper center, hovers over the white bowl, and releases its grip, dropping the toothpaste into the bowl. Then, the right hand moves the wooden pestle to the upper center, hovers over the white bowl, and releases its grip, dropping the pestle into the bowl alongside the toothpaste. Finally, the camera perspective remains fixed, both the white toothpaste tube and the wooden pestle are resting inside the white bowl in the upper center of the table, and the two hands naturally retreat from above the bowl.
Robot The white toothpaste tube and the long light brown wooden pestle are lying flat on the surface. The large shallow white bowl is empty and resting on the surface. 1. The left hand grasps and lifts the toothpaste while the right hand grasps and lifts the wooden pestle simultaneously. 2. The left hand moves the toothpaste over the white bowl and releases it, dropping it into the bowl. 3. The right hand then moves the wooden pestle over the white bowl and releases it, dropping it into the bowl alongside the toothpaste. Both the white toothpaste tube and the wooden pestle are resting inside the large white bowl.
Human (generated)Robot · images.rgb.head · 원본 episode 106 frames 0–482
Human 지시
Pick up the donut and place it on the drawing, then pick up the egg and place it on the drawing
Robot 태스크
Pick up the donut and place it on the drawing and then pick up the egg and place it on the drawing
dense caption · sample 정보
Human In a cafe setting, a woman stands behind a white marble counter with her hands initially resting on the edge. On the counter, there is a yellow donut in the front-left area, a small greenish-grey egg situated in the middle-left area, and a rectangular framed dark drawing resting flat on the right side. First, one of the woman's hands lifts from the counter, reaches towards the front-left area, pinches the yellow donut with her fingers, lifts it, moves it horizontally to the right, and carefully places it onto the left side of the rectangular drawing's surface. Then, the same hand reaches back to the middle-left area, gently grasps the greenish-grey egg, lifts it, moves it to the right, and places it squarely in the center of the rectangular drawing, subsequently retracting away from the objects. In the final frame, the yellow donut rests on the left side of the drawing and the greenish-grey egg sits in the center of the drawing, both fully supported by the drawing board on the right side of the counter.
Robot A yellow ring-shaped donut, a small greenish-grey round egg, and a rectangular drawing with a dark center and light border are resting on the table. The robot has not made contact with any object. 1. The robot arm grasps the yellow donut, lifts it from the table, and places it onto the left side of the rectangular drawing. 2. The robot arm then moves to grasp the small greenish-grey egg, lifts it, and places it onto the center of the rectangular drawing. Both the yellow donut and the small greenish-grey egg are resting on the surface of the rectangular drawing.
Human (generated)Robot · images.rgb.head · 원본 episode 240 frames 0–584
Human 지시
Pick up the small egg and place it on the light blue dish, then pick up the blue rectangular wrapper and place it on the light blue dish
Robot 태스크
Pick up the egg and place it in the dish with right arm and then pick up the donut and place it in the dish with right arm
dense caption · sample 정보
Human First-person perspective with a fixed camera looking down at a glass desk. In the absolute center of the frame, a light blue rectangular dish is placed upside down (with its bottom facing up). To the upper right of the dish rests a small pale-green egg, and to the lower right rests a blue rectangular wrapper. Two human arms are initially resting on the desk. First, the right hand reaches toward the upper right, grasps the small pale-green egg using the thumb and index finger, lifts it, moves it to the center of the desk, and gently places it onto the upturned bottom surface of the light blue dish. Next, the right hand moves to the lower right, grips the blue rectangular wrapper, lifts it, moves it to the center, and places it onto the bottom surface of the dish right next to the egg. During this entire sequence, the left hand remains stationary, resting on the left side of the desk. Finally, the video concludes with a static frame showing both the small pale-green egg and the blue rectangular wrapper resting steadily alongside each other on top of the upside-down light blue dish in the center of the desk.
Robot The small pale-green egg, the light blue dish, and the blue rectangular wrapper are resting separately on the table. First, the small pale-green egg is picked up and placed onto the light blue dish. Next, the blue rectangular wrapper is picked up and placed onto the light blue dish alongside the egg. Both the small pale-green egg and the blue rectangular wrapper are resting inside the light blue dish.
pair가 많은 robot task: bridge_oxe/0.1.0/bridge_oxe_0.1.0_lerobot_part_12 (300), bridge_oxe/0.1.0/bridge_oxe_0.1.0_lerobot_part_5 (294), bridge_oxe/0.1.0/bridge_oxe_0.1.0_lerobot_part_8 (291)
bridge_oxe/0.1.0/bridge_oxe_0.1.0_lerobot_part_7
wan-localphysics 5coarse
Human (generated)Robot · observation.images.image · 원본 episode 661 frames 0–47
Human 지시
Move the blue toy object from the orange towel to the side of the silver bowl.
Robot 태스크
Move the blue object from the towel to the side of the bowl.
dense caption · sample 정보
Human From a fixed first-person perspective, the scene displays a reflective glass table. On the upper left of the table is a silver bowl with handles, in the center are a black pen and a stack of yellow sticky notes, and on the right is an orange towel with a dark blue bumpy toy resting on it. Two human arms wearing light grey long sleeves are resting in the lower foreground. First, the right hand reaches out to the right and firmly grasps the dark blue toy resting on the orange towel. Then, the right hand lifts the blue toy upwards, breaking contact with the towel, and moves it horizontally across the frame to the left, approaching the silver bowl. Finally, the right hand comes to a halt, suspending the toy in mid-air right next to the right side of the silver bowl. Final frame summary: The right hand securely holds the dark blue toy suspended in the air next to the silver bowl on the left, the orange towel on the right is empty, and the silver bowl remains empty in its original position.
Robot The dark blue, bumpy toy object is resting entirely on top of the square orange towel. The small silver bowl with two side handles is empty and resting directly on the surface. The gripper descends and firmly grasps the dark blue bumpy object resting on the orange towel. It then lifts the blue object upwards, breaking contact with the towel, and moves it horizontally across the space, eventually coming to a halt mid-air right next to the side of the silver bowl. The dark blue object is held securely in the gripper, suspended in the air to the side of the silver bowl. The orange towel is empty and remains in its place. The silver bowl remains empty in its original position.
Human (generated)Robot · observation.images.image · 원본 episode 455 frames 0–44
Human 지시
Put the green pepper in the pot.
Robot 태스크
Put the pepper in the circle.
dense caption · sample 정보
Human From a fixed first-person perspective, there is a black marble-textured table. On the table, an empty stainless steel pot is positioned slightly to the left of the center, and a green pepper is positioned slightly to the right. There are also objects like an espresso cup, a crumpled napkin, and a receipt on the table. Two human arms extend from the bottom left and right of the frame. The operation begins with the right hand moving to the right, closing its fingers to firmly grasp the green pepper from the table. Next, the right hand lifts the green pepper and moves it horizontally to the left until it is positioned directly above the stainless steel pot. Then, the right hand opens its fingers to release the green pepper, allowing it to drop naturally into the pot. Throughout the entire process, the left arm remains resting stationary on the left side of the table. In the final frame, the green pepper is safely resting inside the stainless steel pot.
Robot The green pepper and the empty stainless steel pot are placed separately on the table. The robotic gripper lowers to firmly grasp the green pepper from the table, lifts it up, moves it horizontally until it is positioned directly above the stainless steel pot, and then opens to release the pepper, allowing it to drop into the pot. The green pepper is safely resting inside the stainless steel pot.
Human (generated)Robot · observation.images.image · 원본 episode 1000 frames 0–47
Human 지시
put the stainless steel pan between the spoon and the red food object
Robot 태스크
put the pan between the spoon and food
dense caption · sample 정보
Human First-person view of an office desk environment. The left arm remains resting quietly on the desk in the lower left corner. On the left side of the gray desk pad is a spoon with a thick light blue handle and a gray scoop. On the lower right side sits a small stainless steel pan resting on a yellow square towel. Currently, the right hand is holding a small, glossy red cup-like object in mid-air on the upper right. First, the right hand descends and smoothly places the red object onto the upper right area of the desk pad, then releases it. Next, the right hand moves to the lower right, approaches the stainless steel pan, and securely grasps its metal wire handle with its fingers. Then, the right hand lifts the pan off the yellow towel, moves it towards the center-left of the desk pad, and places it down precisely between the spoon on the left and the newly placed red object on the right. Finally, the right hand releases the handle and moves back slightly. The static final frame shows the red object on the upper right, the spoon on the left, and the stainless steel pan positioned perfectly in the middle between them on the gray desk pad, while the yellow towel on the lower right remains empty.
Robot The robot gripper is holding a small, glossy red plastic object in mid-air. A small round stainless steel pan is resting on a yellow square cloth on the left side of the wooden table. A spoon with a thick light blue handle and a gray scoop is placed on the lower right side of the table. 1. The robot gripper descends and releases the red plastic object, dropping it onto the wooden table. 2. The robot moves to the left, approaches the small stainless steel pan, and grasps its metal wire handle. 3. The robot lifts the pan off the yellow cloth, moves it towards the right, and places it down on the table, positioning it precisely between the spoon and the red plastic object. The red plastic object and the spoon are resting on the table. The small stainless steel pan is positioned on the table directly between the spoon and the red object. The yellow square cloth remains empty on the left side of the table.
Human (generated)Robot · observation.images.image · 원본 episode 277 frames 0–29
Human 지시
Remove the plush mushroom from the silver pot and place it in front along the edge of the table.
Robot 태스크
Remove the mushroom from the silver pot and place it in front along the edge of the table.
dense caption · sample 정보
Human From a first-person perspective, two hands wearing grey sleeves rest on the near edge of a grey textured table. On the middle-right side of the table, a plush toy mushroom with a white stalk and a brown cap sits inside a small silver metal pot, which rests on a light blue cloth. First, the right hand reaches towards the right side of the table and grasps the white stalk of the plush mushroom. Then, the right hand lifts the mushroom straight up, extracting it completely from the small silver pot. Next, the right hand moves the mushroom towards the bottom center of the frame, bringing it directly over the near front edge of the table. Finally, the right hand releases its grip, placing the plush mushroom down onto the table surface. Final frame summary: The plush toy mushroom rests on the near front edge of the table, and the empty silver pot remains in its original position on the light blue cloth on the right side.
Robot A plush toy mushroom (white stalk, brown cap) is resting inside a small silver metal pot with side handles. The pot is sitting on a light blue square cloth on the table. The plush toy mushroom is grasped by its white stalk, lifted upwards out of the small silver pot, moved forward towards the user over the table surface, and finally released to drop onto the table. The plush toy mushroom rests directly on the table surface near the front edge. The small silver pot remains empty on the light blue cloth in its original position.
Human (generated)Robot · observation.images.image · 원본 episode 529 frames 0–32
Human 지시
Move the yellow can with green dots to the back right area closer to the wall.
Robot 태스크
Move the can to the back nearer the wall on the right side.
dense caption · sample 정보
Human From a fixed first-person perspective, the video shows a human operating at a light blue glass desk. The scene contains a yellow cylindrical can with green polka dots on the mid-left, and a white keyboard on the right. First, the left hand reaches forward to the left and grasps the sides of the yellow can with green dots. Next, the left hand lifts the can slightly off the desk. Then, the left hand moves the can diagonally across the workspace towards the top-right area, specifically towards the edge behind the white keyboard. Finally, the left hand carefully places the can down onto the surface in the top-right area, releases its grip, and withdraws. Final frame summary: The yellow can with green polka dots rests securely in the top-right area of the desk near the keyboard, with both hands returned to a resting position.
Robot The yellow can with green polka dots is situated on the front part of the surface. A hand approaches and grasps the yellow can with green dots, lifts it slightly off the surface, moves it horizontally towards the back right corner, and carefully places it down near the wall. The yellow can with green polka dots rests in the back right corner of the surface, closer to the back wall.
Human (generated)Robot · observation.images.image · 원본 episode 938 frames 0–33
Human 지시
pick the silver vessel and place it on the orange cloth diagonally
Robot 태스크
pick vessel from orange cloth and move it diagonally
dense caption · sample 정보
Human From an egocentric perspective, a person's two arms are visible resting on a greyish-blue table. On the table, a silver metal bowl is located on the upper left, a toy spatula with a green handle is on the lower left, a toy carrot is on the upper right, and a folded orange cloth is on the lower right. First, the right hand reaches forward and to the upper left to firmly grasp the rim of the silver metal bowl. Then, the right hand lifts the silver bowl and moves it diagonally towards the lower right area. Next, the hand carefully places the silver bowl directly on top of the folded orange cloth in the lower right. Finally, the right hand releases its grip on the bowl and withdraws. In the final frame, the silver metal bowl rests steadily on top of the orange cloth in the lower right.
Robot The silver bowl, folded orange cloth, toy carrot, and toy spatula are resting separately on the wooden surface. The robot gripper moves to grasp the silver bowl, lifts it up, moves it diagonally towards the top left, and places it down on top of the orange cloth. The silver bowl is resting steadily on top of the orange cloth, while the toy carrot and spatula remain in their original positions.
Human (generated)Robot · observation.images.image · 원본 episode 358 frames 0–36
Human 지시
Take the silver pan from the orange cloth and place it next to the pink-handled fork.
Robot 태스크
Take the silver vessel and place it next to fork.
dense caption · sample 정보
Human First-person egocentric view. Two human arms (one left, one right) are resting on a textured marble table in the foreground. On the table, there is a fork with a thick pink handle on the middle-left, a blue spiky ball toy on the far top-left, and a small silver pan resting on a square orange cloth on the middle-right. First, the right hand reaches out towards the silver pan on the middle-right, grasps the rim of the pan, and lifts it up smoothly from the orange cloth. Then, the right hand carries the pan horizontally to the left across the table, moving towards the pink-handled fork. Finally, the right hand lowers the silver pan and places it directly on the marble table surface next to the pink-handled fork, and then releases it. Final frame summary: The silver pan rests on the table surface next to the pink-handled fork on the left side, leaving the orange cloth completely empty on the right.
Robot A small silver pan with a wire handle rests on a square orange cloth on the left side of the table. A fork with a thick pink handle lies in the middle, and a blue spiky ball is positioned on the right side. The robotic gripper grasps the rim of the silver pan, lifts it up from the orange cloth, moves it horizontally to the right across the table, and lowers it down next to the pink-handled fork. The silver pan is placed directly on the table surface right next to the pink-handled fork and the blue spiky ball, leaving the orange cloth empty on the left.
Human (generated)Robot · observation.images.image · 원본 episode 188 frames 0–47
Human 지시
Pick up the red spoon and place it on the yellow cloth.
Robot 태스크
Take the red spoon and put it on the yellow cloth.
dense caption · sample 정보
Human From an egocentric perspective, a flat concrete surface is visible. In the main foreground area, a red-handled spoon lies on the middle-left, and a folded yellow square cloth rests on the middle-right. Two human arms extend from the bottom of the frame. First, the left hand moves forward and to the left, reaching the red-handled spoon, and its fingers firmly grasp the red handle. Then, the left hand lifts the spoon off the surface and moves it horizontally across the frame to the right, stopping directly above the yellow cloth. Next, the left hand lowers the spoon, placing it gently and flatly onto the center of the top surface of the yellow cloth, before releasing its grip and retreating back towards its initial position at the bottom left. The right hand remains resting on the surface without participating. In the final frame, the red-handled spoon stably rests flat on top of the yellow square cloth on the right side of the workspace.
Robot In the first frame, the red-handled spoon and the yellow square cloth are resting separately on the flat surface. The spoon is not in contact with the cloth. 1. The robotic gripper approaches the red-handled spoon on the table. 2. The gripper grasps the red handle of the spoon. 3. The spoon is lifted up from its original position. 4. The spoon is transported horizontally over to the location of the yellow cloth. 5. The spoon is lowered and placed directly onto the top surface of the yellow cloth. 6. The gripper releases the spoon and moves away. The red-handled spoon is resting flat on top of the yellow square cloth.
Human (generated)Robot · observation.images.cam_high
Human 지시
Simultaneously use both hands to pick up an orange bread from the left and a yellow bread from the right, and place them into a wooden bowl in the center.
Robot 태스크
If there is one bread on the table, use one arm to grab the bread and put it in the basket, if there are two breads on the table, use two arms to simultaneously grab up two breads and put them in the basket.
dense caption · sample 정보
Human Simultaneously use both hands to pick up an orange bread from the left and a yellow bread from the right, and place them into a wooden bowl in the center.
Robot If there is one bread on the table, use one arm to grab the bread and put it in the basket, if there are two breads on the table, use two arms to simultaneously grab up two breads and put them in the basket.