Index
2026-10-20 · ego2robot · data gallery

ego2robot 데이터 갤러리: ALOHA / UR5e episode 56개를 눈으로 확인

사람 원본 | 데이터셋 이미지(export 그대로) | physics replay · frame별 flag timeline · 생성: tools/make_data_gallery.py (seed 0)

TL;DR

56
영상
44
accepted 샘플
12
rejected 샘플
245
ALOHA episodes
369
UR5e episodes

1 영상 읽는 법

① 사람 원본 (H2O)
H2O cam4 원본 frame (640×360). physics 구간 안에서는 GT 물체 이동/높이도 표시.
② 데이터셋 이미지
export의 observation.images.ego episode 영상을 decode한 그대로 (학습이 보는 이미지). rejected 후보는 export되지 않았으므로 work dir의 composite.mp4를 빨간 테두리로 표시.
③ physics replay
dataset action을 MuJoCo 트윈에서 재생한 결과를 어둡게 한 실제 frame 위에 그림. 초록 ghost = GT 물체, 빨강 ghost = 로봇 없이 둔 물체.

위 header: 로봇 · sequence · episode key · task · label source · split · 길이 / accept 여부와 reject 사유 · _phys 포함 여부 / physics 판정과 실패 원인 · 관통/grasp/frame_bad/ik_transition 비율.

아래 timeline (episode 전체, 흰 cursor = 현재 frame)

accepted episode의 timeline은 export parquet 값이고, work dir에서 다시 계산한 값과 일치함을 렌더링 때 확인함 (최대 차이 0).

2 데이터셋 요약

ALOHA · /home/nas_main/taewoongkang/Dataset/H2O_ego2robot/aloha/lerobot/h2o_ego2robot_aloha_v21

기본 export
245 ep / 24,253 frame
_phys export (L4)
164 ep / 14,721 frame
후보 → accept
933 → 245
reject
688
label (accepted)
gt 171, vlm 74
split (accepted)
train 133, val 38, test 74
physics (accepted)
fail 77, success 168
관통 flag (accepted)
no 209, yes 36
샘플
accepted 22 + rejected 6

reject 사유 분포 (한 후보가 여러 사유를 가질 수 있음): L1 other 616, L1 ik_fail 196, L1 object collision 66, L3 VLM 23, L2 gripper rate 18, L2 z-score 2

물체dataset ep샘플 accepted샘플 rejected
book3330
chips220
espresso6151
lotion8361
milk110
spray6552
동사dataset ep샘플 accepted샘플 rejected
apply1922
close1020
grab5441
open2020
place7621
put in1021
read3020
spray620
squeeze720
take out1321

UR5e · /home/nas_main/taewoongkang/Dataset/H2O_ego2robot/ur5e/lerobot/h2o_ego2robot_ur5e_v21

기본 export
369 ep / 32,212 frame
_phys export (L4)
266 ep / 24,047 frame
후보 → accept
933 → 369
reject
564
label (accepted)
gt 267, vlm 102
split (accepted)
train 213, val 54, test 102
physics (accepted)
fail 98, success 271
관통 flag (accepted)
no 334, yes 35
샘플
accepted 22 + rejected 6

reject 사유 분포 (한 후보가 여러 사유를 가질 수 있음): L1 other 444, L1 ik_fail 345, L1 object collision 174, L3 VLM 12, L2 z-score 10, L2 gripper rate 3

물체dataset ep샘플 accepted샘플 rejected
book1930
chips6921
cocoa5031
espresso4331
lotion7751
milk6431
spray4730
동사dataset ep샘플 accepted샘플 rejected
apply1420
close3220
grab6131
open4221
place13530
pour411
put in2020
read1321
spray620
squeeze511
take out3721

3 샘플 선택

로봇마다 seed 0의 층화 샘플. accepted: 물체 · 동사 · label source(gt/vlm) · split · physics 성공/실패 · 관통 flag · 길이(짧음/중간/긺, accepted 길이 3분위)의 모든 값을 먼저 덮고, 이후 가장 덜 나온 값 위주로 채움 (physics 실패 원인과 moved/static도 가중치 0.5로 균형). rejected: reject 사유 범주(L1 ik_fail, L1 object collision, L2, L3 VLM, ...)를 돌아가며 하나씩 고름 (첫 바퀴는 그 사유 하나로만 reject된 후보 우선).

4 갤러리

로봇
accept 여부
물체
동사
physics
관통 flag
split
label
표시 중: 56 / 56 episode
grab the lotion #aloha_subject1_h2_3_744-801
ALOHAacceptedphysics fail · knocked_phys 제외train · gt
subject1/h2/3 · frame 744–801 (58 frame, 1.9 s) · export episode 26 · object_penetration L 0% / R 0% · grasp L 0% / R 7% · frame_bad 0%
take out the espresso #aloha_subject4_o1_7_118-189
ALOHAacceptedphysics success_phys 포함관통 flag 26%test · vlm
subject4/o1/7 · frame 118–189 (72 frame, 2.4 s) · export episode 233 · object_penetration L 26% / R 0% · grasp L 0% / R 60% · frame_bad 26%
apply the spray #aloha_subject3_o1_5_348-662
ALOHAacceptedphysics success_phys 포함val · gt
subject3/o1/5 · frame 348–662 (315 frame, 10.5 s) · export episode 156 · object_penetration L 0% / R 0% · grasp L 0% / R 0% · frame_bad 0%
place the book #aloha_subject4_h1_2_12-103
ALOHAacceptedphysics fail · tracking_phys 제외test · vlm
subject4/h1/2 · frame 12–103 (92 frame, 3.1 s) · export episode 171 · object_penetration L 0% / R 0% · grasp L 29% / R 28% · frame_bad 0%
open the milk #aloha_subject4_h2_1_84-266
ALOHAacceptedphysics success_phys 제외관통 flag 50%test · vlm
subject4/h2/1 · frame 84–266 (183 frame, 6.1 s) · export episode 183 · object_penetration L 39% / R 43% · grasp L 0% / R 0% · frame_bad 50%
read the espresso #aloha_subject2_o2_6_209-363
ALOHAacceptedphysics fail · dropped_phys 제외관통 flag 1%train · gt
subject2/o2/6 · frame 209–363 (155 frame, 5.2 s) · export episode 99 · object_penetration L 1% / R 0% · grasp L 0% / R 100% · frame_bad 1%
close the lotion #aloha_subject3_o2_3_844-919
ALOHAacceptedphysics fail · not_held_phys 제외val · gt
subject3/o2/3 · frame 844–919 (76 frame, 2.5 s) · export episode 165 · object_penetration L 0% / R 0% · grasp L 54% / R 100% · frame_bad 0%
take out the chips #aloha_subject4_h1_4_460-607
ALOHAacceptedphysics fail · overextended_phys 제외관통 flag 41%test · vlm
subject4/h1/4 · frame 460–607 (148 frame, 4.9 s) · export episode 178 · object_penetration L 0% / R 41% · grasp L 0% / R 0% · frame_bad 41%
spray the spray #aloha_subject4_o2_4_163-219
ALOHAacceptedphysics success_phys 포함test · vlm
subject4/o2/4 · frame 163–219 (57 frame, 1.9 s) · export episode 241 · object_penetration L 0% / R 0% · grasp L 0% / R 100% · frame_bad 0%
put in the espresso #aloha_subject2_o1_6_422-520
ALOHAacceptedphysics fail · unstable_start_phys 제외train · gt
subject2/o1/6 · frame 422–520 (99 frame, 3.3 s) · export episode 94 · object_penetration L 0% / R 0% · grasp L 100% / R 78% · frame_bad 0%
squeeze the lotion #aloha_subject1_h1_3_255-424
ALOHAacceptedphysics fail · other_phys 제외train · gt
subject1/h1/3 · frame 255–424 (170 frame, 5.7 s) · export episode 5 · object_penetration L 0% / R 0% · grasp L 0% / R 100% · frame_bad 2%
grab the book #aloha_subject1_o2_2_179-267
ALOHAacceptedphysics fail · no_grasp_contact_phys 제외관통 flag 45%train · gt
subject1/o2/2 · frame 179–267 (89 frame, 3.0 s) · export episode 60 · object_penetration L 0% / R 45% · grasp L 45% / R 0% · frame_bad 45%
grab the spray #aloha_subject3_o2_5_716-747
ALOHAacceptedphysics fail · slipped_phys 제외val · gt
subject3/o2/5 · frame 716–747 (32 frame, 1.1 s) · export episode 167 · object_penetration L 0% / R 0% · grasp L 56% / R 0% · frame_bad 0%
open the lotion #aloha_subject4_h1_3_95-142
ALOHAacceptedphysics fail · passive_phys 제외test · vlm
subject4/h1/3 · frame 95–142 (48 frame, 1.6 s) · export episode 174 · object_penetration L 0% / R 0% · grasp L 19% / R 50% · frame_bad 0%
place the espresso #aloha_subject3_h2_7_47-113
ALOHAacceptedphysics fail · start_collision_phys 제외관통 flag 7%train · gt
subject3/h2/7 · frame 47–113 (67 frame, 2.2 s) · export episode 117 · object_penetration L 7% / R 0% · grasp L 0% / R 78% · frame_bad 7%
apply the spray #aloha_subject3_o2_5_380-715
ALOHAacceptedphysics success_phys 포함val · gt
subject3/o2/5 · frame 380–715 (336 frame, 11.2 s) · export episode 166 · object_penetration L 0% / R 0% · grasp L 0% / R 0% · frame_bad 0%
read the book #aloha_subject3_k2_2_256-452
ALOHAacceptedphysics fail · tracking_phys 제외관통 flag 2%val · gt
subject3/k2/2 · frame 256–452 (197 frame, 6.6 s) · export episode 134 · object_penetration L 0% / R 2% · grasp L 100% / R 0% · frame_bad 2%
grab the chips #aloha_subject3_h1_4_119-148
ALOHAacceptedphysics success_phys 제외관통 flag 47%train · gt
subject3/h1/4 · frame 119–148 (30 frame, 1.0 s) · export episode 104 · object_penetration L 7% / R 47% · grasp L 0% / R 0% · frame_bad 47%
spray the spray #aloha_subject4_k2_5_331-402
ALOHAacceptedphysics success_phys 포함test · vlm
subject4/k2/5 · frame 331–402 (72 frame, 2.4 s) · export episode 219 · object_penetration L 0% / R 0% · grasp L 0% / R 85% · frame_bad 0%
squeeze the lotion #aloha_subject4_k1_2_296-418
ALOHAacceptedphysics fail · dropped_phys 제외test · vlm
subject4/k1/2 · frame 296–418 (123 frame, 4.1 s) · export episode 202 · object_penetration L 0% / R 0% · grasp L 0% / R 100% · frame_bad 0%
close the lotion #aloha_subject3_o1_3_755-815
ALOHAacceptedphysics success_phys 포함관통 flag 11%val · gt
subject3/o1/3 · frame 755–815 (61 frame, 2.0 s) · export episode 151 · object_penetration L 11% / R 0% · grasp L 0% / R 100% · frame_bad 11%
put in the espresso #aloha_subject3_k1_7_425-489
ALOHAacceptedphysics fail · dropped_phys 제외train · gt
subject3/k1/7 · frame 425–489 (65 frame, 2.2 s) · export episode 132 · object_penetration L 0% / R 0% · grasp L 100% / R 0% · frame_bad 0%
apply the lotion #aloha_subject2_k2_3_363-684
ALOHArejectedphysics success_phys 제외train · gt
subject2/k2/3 · frame 363–684 (322 frame, 10.7 s) · 미export (composite.mp4) · object_penetration L 0% / R 0% · grasp L 0% / R 0% · frame_bad 28%
reject 사유: L1: ik_fail_ratio 0.280 > 0.1
grab the spray #aloha_subject4_k1_4_112-171
ALOHArejectedphysics fail · tracking_phys 제외관통 flag 5%test · vlm
subject4/k1/4 · frame 112–171 (60 frame, 2.0 s) · 미export (composite.mp4) · object_penetration L 5% / R 0% · grasp L 60% / R 67% · frame_bad 5%
reject 사유: L1: object collision ratio 0.050 > 0.0
place the cappuccino #aloha_subject2_o1_0_8-81
ALOHArejectedphysics fail · ik_fail_phys 제외관통 flag 80%train · gt
subject2/o1/0 · frame 8–81 (74 frame, 2.5 s) · 미export (composite.mp4) · object_penetration L 73% / R 80% · grasp L 0% / R 0% · frame_bad 80%
reject 사유: L1: ik_fail_ratio 0.405 > 0.1; L1: bad frame ratio 0.797 >= 0.5; L1: unsupported object 18 consecutive frames (lifted > 30 mm, no gripper holding it) >= 5; L2: gripper rate p99 7.05/s > 6.0
apply the spray #aloha_subject1_o2_5_328-453
ALOHArejectedphysics success_phys 제외train · gt
subject1/o2/5 · frame 328–453 (126 frame, 4.2 s) · 미export (composite.mp4) · object_penetration L 0% / R 0% · grasp L 0% / R 0% · frame_bad 0%
reject 사유: L2: dataset z-score outlier (> 4.0 sigma)
put in the espresso #aloha_subject2_k1_6_745-831
ALOHArejectedphysics success_phys 제외train · gt
subject2/k1/6 · frame 745–831 (87 frame, 2.9 s) · 미export (composite.mp4) · object_penetration L 0% / R 0% · grasp L 100% / R 72% · frame_bad 0%
reject 사유: L3: vlm score=3 robot_visible=True human_visible=True
take out the cocoa #aloha_subject3_o1_6_203-266
ALOHArejectedphysics fail · start_collision_phys 제외관통 flag 100%val · gt
subject3/o1/6 · frame 203–266 (64 frame, 2.1 s) · 미export (composite.mp4) · object_penetration L 100% / R 27% · grasp L 0% / R 0% · frame_bad 100%
reject 사유: L1: bad frame ratio 1.000 >= 0.5; L1: unsupported object 64 consecutive frames (lifted > 30 mm, no gripper holding it) >= 5
grab the book #ur5e_2f85_subject1_h2_2_132-225
UR5eacceptedphysics fail · knocked_phys 제외train · gt
subject1/h2/2 · frame 132–225 (94 frame, 3.1 s) · export episode 26 · object_penetration L 0% / R 0% · grasp L 51% / R 41% · frame_bad 0%
take out the espresso #ur5e_2f85_subject4_o2_6_414-492
UR5eacceptedphysics success_phys 포함관통 flag 28%test · vlm
subject4/o2/6 · frame 414–492 (79 frame, 2.6 s) · export episode 367 · object_penetration L 0% / R 28% · grasp L 100% / R 0% · frame_bad 28%
open the lotion #ur5e_2f85_subject3_k2_3_173-227
UR5eacceptedphysics fail · dropped_phys 제외val · gt
subject3/k2/3 · frame 173–227 (55 frame, 1.8 s) · export episode 219 · object_penetration L 0% / R 0% · grasp L 58% / R 100% · frame_bad 0%
place the cocoa #ur5e_2f85_subject2_o2_5_280-335
UR5eacceptedphysics success_phys 포함train · gt
subject2/o2/5 · frame 280–335 (56 frame, 1.9 s) · export episode 158 · object_penetration L 0% / R 0% · grasp L 79% / R 0% · frame_bad 7%
close the chips #ur5e_2f85_subject4_h2_4_360-538
UR5eacceptedphysics success_phys 포함관통 flag 8%test · vlm
subject4/h2/4 · frame 360–538 (179 frame, 6.0 s) · export episode 291 · object_penetration L 0% / R 8% · grasp L 100% / R 33% · frame_bad 8%
spray the spray #ur5e_2f85_subject4_k2_5_331-402
UR5eacceptedphysics fail · dropped_phys 제외test · vlm
subject4/k2/5 · frame 331–402 (72 frame, 2.4 s) · export episode 330 · object_penetration L 0% / R 0% · grasp L 0% / R 85% · frame_bad 0%
pour the milk #ur5e_2f85_subject3_o2_1_287-432
UR5eacceptedphysics success_phys 포함val · gt
subject3/o2/1 · frame 287–432 (146 frame, 4.9 s) · export episode 251 · object_penetration L 0% / R 0% · grasp L 0% / R 100% · frame_bad 0%
put in the espresso #ur5e_2f85_subject2_k2_6_758-837
UR5eacceptedphysics fail · other_phys 제외관통 flag 49%train · gt
subject2/k2/6 · frame 758–837 (80 frame, 2.7 s) · export episode 133 · object_penetration L 0% / R 49% · grasp L 100% / R 0% · frame_bad 49%
squeeze the lotion #ur5e_2f85_subject4_k1_2_296-418
UR5eacceptedphysics fail · tracking_phys 제외test · vlm
subject4/k1/2 · frame 296–418 (123 frame, 4.1 s) · export episode 304 · object_penetration L 0% / R 0% · grasp L 0% / R 100% · frame_bad 0%
read the book #ur5e_2f85_subject4_k2_2_162-243
UR5eacceptedphysics fail · no_grasp_contact_phys 제외test · vlm
subject4/k2/2 · frame 162–243 (82 frame, 2.7 s) · export episode 320 · object_penetration L 0% / R 0% · grasp L 33% / R 20% · frame_bad 0%
apply the spray #ur5e_2f85_subject3_k2_5_322-639
UR5eacceptedphysics success_phys 포함val · gt
subject3/k2/5 · frame 322–639 (318 frame, 10.6 s) · export episode 228 · object_penetration L 0% / R 0% · grasp L 0% / R 0% · frame_bad 1%
place the cocoa #ur5e_2f85_subject2_h2_5_0-68
UR5eacceptedphysics fail · start_collision_phys 제외관통 flag 10%train · gt
subject2/h2/5 · frame 0–68 (69 frame, 2.3 s) · export episode 99 · object_penetration L 10% / R 0% · grasp L 0% / R 77% · frame_bad 10%
grab the chips #ur5e_2f85_subject4_k1_3_98-147
UR5eacceptedphysics fail · passive_phys 제외test · vlm
subject4/k1/3 · frame 98–147 (50 frame, 1.7 s) · export episode 308 · object_penetration L 0% / R 0% · grasp L 40% / R 40% · frame_bad 0%
open the lotion #ur5e_2f85_subject4_o2_2_73-120
UR5eacceptedphysics fail · slipped_phys 제외관통 flag 31%test · vlm
subject4/o2/2 · frame 73–120 (48 frame, 1.6 s) · export episode 356 · object_penetration L 31% / R 0% · grasp L 0% / R 0% · frame_bad 31%
close the milk #ur5e_2f85_subject2_o1_1_729-930
UR5eacceptedphysics success_phys 포함train · gt
subject2/o1/1 · frame 729–930 (202 frame, 6.7 s) · export episode 135 · object_penetration L 0% / R 0% · grasp L 74% / R 70% · frame_bad 8%
place the lotion #ur5e_2f85_subject1_h2_3_0-52
UR5eacceptedphysics fail · not_held_phys 제외관통 flag 2%train · gt
subject1/h2/3 · frame 0–52 (53 frame, 1.8 s) · export episode 28 · object_penetration L 0% / R 2% · grasp L 0% / R 0% · frame_bad 2%
take out the espresso #ur5e_2f85_subject3_o1_7_674-703
UR5eacceptedphysics success_phys 포함관통 flag 10%val · gt
subject3/o1/7 · frame 674–703 (30 frame, 1.0 s) · export episode 247 · object_penetration L 0% / R 10% · grasp L 100% / R 0% · frame_bad 10%
put in the cocoa #ur5e_2f85_subject2_k1_4_295-355
UR5eacceptedphysics success_phys 포함train · gt
subject2/k1/4 · frame 295–355 (61 frame, 2.0 s) · export episode 109 · object_penetration L 0% / R 0% · grasp L 100% / R 38% · frame_bad 0%
read the book #ur5e_2f85_subject3_o1_2_277-494
UR5eacceptedphysics fail · dropped_phys 제외val · gt
subject3/o1/2 · frame 277–494 (218 frame, 7.3 s) · export episode 236 · object_penetration L 0% / R 0% · grasp L 100% / R 20% · frame_bad 0%
spray the spray #ur5e_2f85_subject3_k1_5_276-353
UR5eacceptedphysics fail · dropped_phys 제외관통 flag 3%train · gt
subject3/k1/5 · frame 276–353 (78 frame, 2.6 s) · export episode 204 · object_penetration L 3% / R 0% · grasp L 0% / R 100% · frame_bad 3%
grab the milk #ur5e_2f85_subject4_k2_1_965-1021
UR5eacceptedphysics fail · no_grasp_contact_phys 제외test · vlm
subject4/k2/1 · frame 965–1021 (57 frame, 1.9 s) · export episode 319 · object_penetration L 0% / R 0% · grasp L 70% / R 72% · frame_bad 0%
apply the lotion #ur5e_2f85_subject4_h1_3_540-675
UR5eacceptedphysics success_phys 포함관통 flag 1%test · vlm
subject4/h1/3 · frame 540–675 (136 frame, 4.5 s) · export episode 272 · object_penetration L 0% / R 1% · grasp L 61% / R 0% · frame_bad 1%
squeeze the lotion #ur5e_2f85_subject3_k1_3_260-391
UR5erejectedphysics success_phys 제외train · gt
subject3/k1/3 · frame 260–391 (132 frame, 4.4 s) · 미export (composite.mp4) · object_penetration L 0% / R 0% · grasp L 0% / R 100% · frame_bad 25%
reject 사유: L1: ik_fail_ratio 0.250 > 0.1
read the espresso #ur5e_2f85_subject4_k1_6_166-367
UR5erejectedphysics fail · dropped_phys 제외관통 flag 5%test · vlm
subject4/k1/6 · frame 166–367 (202 frame, 6.7 s) · 미export (composite.mp4) · object_penetration L 0% / R 5% · grasp L 100% / R 28% · frame_bad 5%
reject 사유: L1: object collision ratio 0.054 > 0.0
grab the cappuccino #ur5e_2f85_subject1_k1_0_68-115
UR5erejectedphysics fail · tracking_phys 제외관통 flag 83%train · gt
subject1/k1/0 · frame 68–115 (48 frame, 1.6 s) · 미export (composite.mp4) · object_penetration L 83% / R 79% · grasp L 0% / R 0% · frame_bad 83%
reject 사유: L1: bad frame ratio 0.833 >= 0.5; L1: unsupported object 10 consecutive frames (lifted > 30 mm, no gripper holding it) >= 5; L2: gripper rate p99 6.50/s > 6.0
open the chips #ur5e_2f85_subject3_k1_4_158-214
UR5erejectedphysics success_phys 제외관통 flag 30%train · gt
subject3/k1/4 · frame 158–214 (57 frame, 1.9 s) · 미export (composite.mp4) · object_penetration L 0% / R 30% · grasp L 100% / R 0% · frame_bad 35%
reject 사유: L2: dataset z-score outlier (> 4.0 sigma)
pour the milk #ur5e_2f85_subject1_h2_1_340-448
UR5erejectedphysics fail · other_phys 제외train · gt
subject1/h2/1 · frame 340–448 (109 frame, 3.6 s) · 미export (composite.mp4) · object_penetration L 0% / R 0% · grasp L 0% / R 100% · frame_bad 8%
reject 사유: L3: vlm score=3 robot_visible=True human_visible=True
take out the cocoa #ur5e_2f85_subject3_o2_6_192-264
UR5erejectedphysics fail · start_collision_phys 제외관통 flag 100%val · gt
subject3/o2/6 · frame 192–264 (73 frame, 2.4 s) · 미export (composite.mp4) · object_penetration L 100% / R 0% · grasp L 0% / R 48% · frame_bad 100%
reject 사유: L1: bad frame ratio 1.000 >= 0.5; L1: unsupported object 28 consecutive frames (lifted > 30 mm, no gripper holding it) >= 5