Daily Progress — 2026-10-01 (목)

Zero-WAMHigh

zero-shot cross-task manipulation(demo/language 컨텍스트) 논문 주장 검증·재현. 현재: w/o pretrain 재현 격차 −9.29pp의 원인(학습 스텝 32K vs 50K) 검증.

15SLURM 작업

337.7GPU-hours

0Commits

1작업 문서

어제 한 일

exp-zerowam_wopretrain_50k.md260929 · 260930 업데이트

job 165390(tail 48000→50000, own 8 GPU, 1h20m, 2.40 s/it) COMPLETED. 로그에 Checkpoint saved successfully at step 50000 확인 → train.py 최종 저장 수정이 동작. checkpoint_step_50000 22 GB.
lang 평가 165392(extra 7 GPU)는 1h49m 만에 PREEMPTED. 부분 결과(scale 1/23, stamp 2/23, microwave 0/6, stapler 14/29, bread 5/16, cup 2/18, stack 0/7)를 보존하고 태스크별 1 GPU extra job 7건(166675~166681)으로 이어받기 제출.
이어받기 결과: scale·stamp·stapler·bread·cup 5개 COMPLETED(4h~9h30m). open_microwave(12h57m)와 stack_blocks_three(5h50m)는 PREEMPTED → 재제출 반복(167014 PREEMPTED 3h42m, 167111 CANCELLED, 167124 own으로 RUNNING, 167170 PENDING).

ICL 중간값 (09-30 09:00 UTC, 50K vs 32K 최종): stamp_seal 45/100 vs 47 · move_stapler_pad 36/100 vs 40 · place_object_scale 10/100 vs 16 · place_bread_basket 20/67 vs 37 · place_empty_cup 18/86 vs 28 · open_microwave 6/30 vs 21 · stack_blocks_three 0/36 vs 0. 완료된 3태스크 모두 32K 이하. 스텝 부족 가설과 반대 방향.

SLURM

COMPLETED 7PREEMPTED 4CANCELLED 1RUNNING 2PENDING 1

RUNNING: 165391 zw50k_eval_icl (7 GPU, own), 167124 zw50k_lres4_stack_blocks_three (1 GPU, own)

오늘 할 일

165391 ICL 완료·저장 확인 (한도 30h; 시작 후 22h42m 경과)
lang 이어받기 완료분(5개 + stack/microwave 진행분)을 tmp/add_own_wopretrain.py로 병합 → 32K 대비 태스크별 Δ
ICL 36%에 접근하지 않으면 다른 원인 검토: 에피소드 서브샘플, 해상도(256×320 vs 224×288), 시드. 필요 시 48000 ckpt 평가
extra 작업 preempt 반복 — stack/microwave는 own 유지가 안전
남은 대조 (영상만 교체 + 캡션 고정) 미착수

EmbodiedSWEHigh

코딩 에이전트가 solve(env)로 로봇 조작을 푸는 EmbodiedSWE-Bench(Isaac Lab). 현재 목표: 가구 조립 태스크를 Franka 한 대로 실행.

40SLURM 작업

15.5GPU-hours

11/11one_leg seed 성공

3작업 문서

어제 한 일

exp-furniture_one_leg_solve.md260930
progress-furniture_bench_one_leg.md260930
progress-ikea_table_single_franka.md260930

IKEA 테이블 Franka 한 대 설정: assembly.ikea_table.franka.* 5개 모드 등록. 손 아래 방향 도달 반경 0.30~0.80m 전부 OK, 기둥 8/8, 다리 집기 4/4. 외력 0.05~1.5 N·m에서 상판 yaw ≤0.16°, 밀림 ≤1.3mm → 고정 장치 없이도 가능. .venv(Isaac Sim 5.1 / Lab 2.3.2) 설치. pytest 15 passed.

one_leg 이식: FurnitureBench scene·grader·SDF 에셋 변환·6개 환경 등록. 에셋 질량 top 151.0g / leg 23.1g(원본 일치). 외력 테스트에서 시계방향 3/3이 46.8mm까지 조여짐(dt 1/240), dt 1/480에서는 불안정.

one_leg 해법 (solve.py, diff_ik): 상판 모서리 밀기 → 다리 집기 → 관절 공간 세우기 → 나사 끝 조준 꽂기 → 재파지 조이기 6사이클. 디버깅 11건 해소. 워커 seed 0~10 11/11, 최종 높이 46.82mm(목표 46.875), 실행 14.6~16.1분/에피소드. 공식 verify_solution.py success: true(n54, 15m28s). 워커 렌더링은 nvidia/opengl 컨테이너 + libGLU/libOpenGL/libgomp를 LD_LIBRARY_PATH로 연결해 해결. 영상 1280×720 95초.

SLURM

COMPLETED 34PREEMPTED 2CANCELLED 2TIMEOUT 2

주요 job: one_leg_s0~s10(seed별 tmp/one_leg_seed.sh), one_leg_verify(공식 채점, 앞 2건은 6~7초 만에 종료 → libgomp 누락 원인 규명 후 3번째 통과), render_probe 4건, one_leg_eval 2건. TIMEOUT 2건(166852/166853)은 1h 한도에서 종료.

오늘 할 일

Q_STAND(seed 0 측정 하드코딩) → IK 계산으로 일반화. FurnitureBench med/high 랜덤화 대응
IKEA 테이블 4다리 solve.py 작성 (미착수)
에이전트 평가(eval/scripts/run_agent.py)가 Docker를 요구하는데 워커에 Docker 없음 → 실행 방식 결정
에피소드 15분 단축(다리 23g, SDF 512, 192 반복)
FurnitureBench 에셋 USD는 로컬 전용 → HF 배포는 메인테이너 권한 필요

GPT-PolicyMedium

상용 VLM 에이전트를 gradient 없이 로봇 정책으로 쓰는 in-context robot learning. FurnitureBench one_leg에서 latency·비용·성공률 측정.

15SLURM 작업

3.2GPU-hours

3/3수정 후 시드 성공

$10–24API 환산/성공 episode

어제 한 일

exp-furniture_one_leg_fixture.md260930 · 5개 업데이트 섹션
context.md갱신

실패 원인 추적: 탁판 고정 지시 + 80 decision(165738) → 전 과정 수행 후 done, 시뮬 판정 미조립(reward 0, $11.36, 17m15s). 원인 1은 잘못된 다리: one_leg은 leg4(tags 20–23)만 판정. 지정 후 run 3 float32 관절한계 버그 크래시 → 수정, run 4는 수평 1mm 내 삽입 / 23mm 덜 조임, run 5는 조이기 요령 지시가 역효과(성공 0/5).

screw 도구 추가: 파지 유지하며 수직축 회전 + 하방 preload 반복, joint7 한계 전 open→되감기→재파지. run 6(166093) 첫 시뮬 성공(reward 1.0, sim 240.7s, step 54). 성공 시 episode 종료 로직 추가. run 7(166490)은 구멍 접근 중 경로 이탈(158mm 오차)·home IK 실패로 FAILED.

어댑터 수정 후 시드 0/1/2: 3/3 성공 (166507~166509). protective stop(추종 오차 0.35rad) + joint-space home 보간. step 61/77/114, sim 266/377/471s, screw 5/1/3회. 기본 300s 예산 기준 1/3. screw 설정 전체 4/5. 테스트 94 passed.

SLURM

COMPLETED 7FAILED 3PENDING 5

FAILED: 165393(home IK, 7m), 165843(float32 한계 버그, 14m), 166490(run 7 home IK, 16m). PENDING: 167125~167129 (screw-tool, own, 1 GPU, 1h)

오늘 할 일

167125~167129 결과 수집 → 시드 n≥10 성공률, randomness med/high
기본 300s 예산 성공률 향상: 탁판 고정·접근 경로 탐색 효율화(탁판 밀기 도구 등)
protective stop 실충돌 검증 미완, 체결 여부 검증 수단 부재(force/torque, part pose 피드백)
조임 토크로 탁판 회전 문제, 스레드 refresh 턴이 비용의 44%
src/gpt_policy/simulation/ 미커밋 — 커밋 정리

활동 없음

SLURM 요약

sacct --starttime=2026-09-30, 70 jobs (.batch / step 제외). 프로젝트명은 WorkDir, 실행 명령은 SubmitLine의 template.sh 뒤 inner command 기준. RUNNING 행의 시간은 리포트 생성 시점까지의 경과.

Job ID	프로젝트 (WorkDir)	JobName	실행 명령	GPU	QOS	상태	시간	노드
163512	Zero-WAM	zw_wopre50k	NUM_STEPS=50000 bash script/train_zerowam_wopretrain.sh	8	own	COMPLETED	11:53:00	n1
165390	Zero-WAM	zw_wopre50k_tail	NUM_STEPS=50000 bash script/train_zerowam_wopretrain.sh	8	own	COMPLETED	01:26:35	n13
165391	Zero-WAM	zw50k_eval_icl	bash script/eval_zerowam_robotwin.sh ckpt/checkpoint_step_50000 icl	7	own	RUNNING	22:42:42	n23
165392	Zero-WAM	zw50k_eval_lang	bash script/eval_zerowam_robotwin.sh ckpt/checkpoint_step_50000 lang	7	extra	PREEMPTED	01:49:30	n44
165393	GPT-Policy	gpt-policy-furniture	--qos=own --time=00:50:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg.json	1	own	FAILED	00:07:17	n17
165738	GPT-Policy	gpt-policy-furniture	--qos=own --time=00:45:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-fixture.json	1	own	COMPLETED	00:17:15	n15
165843	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:00:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-leg4.json	1	own	FAILED	00:13:55	n28
165908	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:00:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-leg4.json	1	own	COMPLETED	00:19:59	n17
165938	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:20:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-leg4-screw.json	1	own	COMPLETED	00:31:30	n17
165946	EmbodiedSWE	render_probe	bash tmp/render_probe.sh	0	own	CANCELLED	00:00:00	None assigned
165961	EmbodiedSWE	render_probe	bash tmp/render_probe.sh	1	extra	CANCELLED	00:18:21	n30
165990	EmbodiedSWE	render_probe	bash tmp/render_probe.sh	1	extra	COMPLETED	00:05:43	n17
166031	EmbodiedSWE	render_probe	bash tmp/render_probe.sh	1	extra	COMPLETED	00:03:38	n28
166093	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:20:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-screw-tool.json	1	own	COMPLETED	00:23:17	n6
166376	EmbodiedSWE	one_leg_eval	bash tmp/one_leg_eval.sh	1	extra	COMPLETED	00:50:04	n43
166490	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:20:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-screw-tool.json	1	own	FAILED	00:15:52	n31
166491	EmbodiedSWE	one_leg_eval	bash tmp/one_leg_eval.sh	1	extra	PREEMPTED	00:26:00	n97
166507	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:20:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-screw-tool.json	1	own	COMPLETED	00:14:24	n32
166508	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:20:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-screw-tool.json	1	own	COMPLETED	00:21:21	n31
166509	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:20:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-screw-tool.json	1	own	COMPLETED	00:26:17	n28
166518	EmbodiedSWE	one_leg_s0	bash tmp/one_leg_seed.sh 0 record	1	extra	COMPLETED	00:15:56	n32
166519	EmbodiedSWE	one_leg_s1	bash tmp/one_leg_seed.sh 1	1	extra	COMPLETED	00:12:25	n17
166520	EmbodiedSWE	one_leg_s2	bash tmp/one_leg_seed.sh 2	1	extra	PREEMPTED	00:42:30	n39
166675	Zero-WAM	zw50k_lres_place_object_scale	START_SEED=100032 EPISODE_OFFSET=23 bash script/eval_zerowam_robotwin_task.sh ckpt/checkpoint_step_50000 place_object_scale lang 77 0	1	extra	COMPLETED	05:23:10	n56
166676	Zero-WAM	zw50k_lres_stamp_seal	START_SEED=100034 EPISODE_OFFSET=23 bash script/eval_zerowam_robotwin_task.sh ckpt/checkpoint_step_50000 stamp_seal lang 77 0	1	extra	COMPLETED	05:20:03	n57
166677	Zero-WAM	zw50k_lres_open_microwave	START_SEED=100007 EPISODE_OFFSET=6 bash script/eval_zerowam_robotwin_task.sh ckpt/checkpoint_step_50000 open_microwave lang 94 0	1	extra	PREEMPTED	12:57:30	n48
166678	Zero-WAM	zw50k_lres_move_stapler_pad	START_SEED=100029 EPISODE_OFFSET=29 bash script/eval_zerowam_robotwin_task.sh ckpt/checkpoint_step_50000 move_stapler_pad lang 71 0	1	extra	COMPLETED	04:03:55	n51
166679	Zero-WAM	zw50k_lres_place_bread_basket	START_SEED=100021 EPISODE_OFFSET=16 bash script/eval_zerowam_robotwin_task.sh ckpt/checkpoint_step_50000 place_bread_basket lang 84 0	1	extra	COMPLETED	09:29:44	n44
166680	Zero-WAM	zw50k_lres_place_empty_cup	START_SEED=100020 EPISODE_OFFSET=18 bash script/eval_zerowam_robotwin_task.sh ckpt/checkpoint_step_50000 place_empty_cup lang 82 0	1	extra	COMPLETED	07:58:19	n55
166681	Zero-WAM	zw50k_lres_stack_blocks_three	START_SEED=100007 EPISODE_OFFSET=7 bash script/eval_zerowam_robotwin_task.sh ckpt/checkpoint_step_50000 stack_blocks_three lang 93 0	1	extra	PREEMPTED	05:50:30	n58
166699	EmbodiedSWE	one_leg_s0	bash tmp/one_leg_seed.sh 0 record	1	extra	COMPLETED	00:19:22	n31
166700	EmbodiedSWE	one_leg_s1	bash tmp/one_leg_seed.sh 1	1	extra	COMPLETED	00:15:46	n60
166701	EmbodiedSWE	one_leg_s2	bash tmp/one_leg_seed.sh 2	1	extra	COMPLETED	00:23:55	n28
166702	EmbodiedSWE	one_leg_s3	bash tmp/one_leg_seed.sh 3	1	extra	COMPLETED	00:13:08	n32
166757	EmbodiedSWE	one_leg_s3	bash tmp/one_leg_seed.sh 3 record	1	extra	COMPLETED	00:53:16	n27
166852	EmbodiedSWE	one_leg_s0	bash tmp/one_leg_seed.sh 0	1	extra	TIMEOUT	01:00:00	n27
166853	EmbodiedSWE	one_leg_s1	bash tmp/one_leg_seed.sh 1	1	extra	TIMEOUT	01:00:00	n42
166854	EmbodiedSWE	one_leg_s2	bash tmp/one_leg_seed.sh 2	1	extra	COMPLETED	00:55:01	n46
166855	EmbodiedSWE	one_leg_s3	bash tmp/one_leg_seed.sh 3	1	extra	COMPLETED	00:55:01	n31
166856	EmbodiedSWE	one_leg_s4	bash tmp/one_leg_seed.sh 4	1	extra	COMPLETED	00:55:01	n37
166857	EmbodiedSWE	one_leg_s5	bash tmp/one_leg_seed.sh 5	1	extra	COMPLETED	00:55:01	n34
166992	EmbodiedSWE	one_leg_s0	bash tmp/one_leg_seed.sh 0	1	extra	COMPLETED	00:15:34	n41
166993	EmbodiedSWE	one_leg_s1	bash tmp/one_leg_seed.sh 1	1	extra	COMPLETED	00:14:13	n42
166994	EmbodiedSWE	one_leg_s2	bash tmp/one_leg_seed.sh 2	1	extra	COMPLETED	00:15:57	n52
166995	EmbodiedSWE	one_leg_s4	bash tmp/one_leg_seed.sh 4	1	extra	COMPLETED	00:15:50	n53
166996	EmbodiedSWE	one_leg_s5	bash tmp/one_leg_seed.sh 5	1	extra	COMPLETED	00:27:35	n46
166997	EmbodiedSWE	one_leg_s3	bash tmp/one_leg_seed.sh 3 record	1	extra	COMPLETED	00:19:06	n49
167006	EmbodiedSWE	one_leg_s5	bash tmp/one_leg_seed.sh 5	1	extra	COMPLETED	00:15:17	n66
167007	EmbodiedSWE	one_leg_s6	bash tmp/one_leg_seed.sh 6	1	extra	COMPLETED	00:15:34	n67
167008	EmbodiedSWE	one_leg_s7	bash tmp/one_leg_seed.sh 7	1	extra	COMPLETED	00:14:54	n46
167009	EmbodiedSWE	one_leg_s8	bash tmp/one_leg_seed.sh 8	1	extra	COMPLETED	00:15:22	n49
167010	EmbodiedSWE	one_leg_s9	bash tmp/one_leg_seed.sh 9	1	extra	COMPLETED	00:16:08	n51
167011	EmbodiedSWE	one_leg_s10	bash tmp/one_leg_seed.sh 10	1	extra	COMPLETED	00:15:14	n52
167014	Zero-WAM	zw50k_lres2_stack_blocks_three	START_SEED=100040 EPISODE_OFFSET=35 bash script/eval_zerowam_robotwin_task.sh ckpt/checkpoint_step_50000 stack_blocks_three lang 65 0	1	extra	PREEMPTED	03:42:00	n40
167024	EmbodiedSWE	one_leg_verify	bash tmp/one_leg_verify.sh	1	extra	COMPLETED	00:00:06	n52
167025	EmbodiedSWE	one_leg_s0	bash tmp/one_leg_seed.sh 0	1	extra	COMPLETED	00:14:53	n47
167026	EmbodiedSWE	one_leg_s1	bash tmp/one_leg_seed.sh 1	1	extra	COMPLETED	00:14:54	n51
167027	EmbodiedSWE	one_leg_s2	bash tmp/one_leg_seed.sh 2	1	extra	COMPLETED	00:15:26	n54
167028	EmbodiedSWE	one_leg_s3	bash tmp/one_leg_seed.sh 3	1	extra	COMPLETED	00:14:39	n57
167029	EmbodiedSWE	one_leg_s4	bash tmp/one_leg_seed.sh 4	1	extra	COMPLETED	00:15:10	n58
167038	EmbodiedSWE	one_leg_verify	bash tmp/one_leg_verify.sh	1	extra	COMPLETED	00:00:07	n54
167043	EmbodiedSWE	one_leg_verify	bash tmp/one_leg_verify.sh	1	extra	COMPLETED	00:15:28	n54
167111	Zero-WAM	zw50k_lres3_stack_blocks_three	START_SEED=100060 EPISODE_OFFSET=52 bash script/eval_zerowam_robotwin_task.sh ckpt/checkpoint_step_50000 stack_blocks_three lang 48 0	1	extra	CANCELLED	01:08:23	n37
167124	Zero-WAM	zw50k_lres4_stack_blocks_three	START_SEED=100064 EPISODE_OFFSET=56 bash script/eval_zerowam_robotwin_task.sh ckpt/checkpoint_step_50000 stack_blocks_three lang 44 0	1	own	RUNNING	03:23:42	n7
167125	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:00:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-screw-tool.json	0	own	PENDING	00:00:00	None assigned
167126	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:00:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-screw-tool.json	0	own	PENDING	00:00:00	None assigned
167127	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:00:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-screw-tool.json	0	own	PENDING	00:00:00	None assigned
167128	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:00:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-screw-tool.json	0	own	PENDING	00:00:00	None assigned
167129	GPT-Policy	gpt-policy-furniture	--qos=own --time=01:00:00 scripts/run_furniture.sbatch --input-json configs/tasks/furniture-one-leg-screw-tool.json	0	own	PENDING	00:00:00	None assigned
167170	Zero-WAM	zw50k_lres2_open_microwave	START_SEED=100076 EPISODE_OFFSET=49 bash script/eval_zerowam_robotwin_task.sh ckpt/checkpoint_step_50000 open_microwave lang 51 0	0	own	PENDING	00:00:00	None assigned

Daily Progress — 2026-10-01 (목)

오늘의 우선순위

TL;DR

프로젝트별 카드

어제 한 일

SLURM

오늘 할 일

어제 한 일

SLURM

오늘 할 일

어제 한 일

SLURM

오늘 할 일

활동 없음

SLURM 요약