[LAB   ] 2026-11-06 18:35:05  GPU=NVIDIA A40  episode=run_0001_20261019_194818  log=/Volumes/KIOXIA/r2r2r-data/lab/Log22  
[MODEL ] --model corr2: corr の 1499 から trim2_corr_v4b で続きを学習した 1499 (依頼 57 / 58)
         PixelstoMotors/pi05-mecharm-trim-corr-lora-v2 → /workspace/ckpt/corr_v2_1499  設定 pi05_mecharm_trim_corr_v4_lora  assets local/mecharm_redblock_box_teach_v1_trim2_corr_v4b

=== 1. Pod を作る
[POD   ] 作った: id b6j7nc4m74h7ru  GPU=NVIDIA A40  1時間 0.59 ドル  (記録 /Volumes/KIOXIA/r2r2r-data/lab/Log22/pod/lab_pod.json)
    (2 秒)

=== 2. Pod が立ち上がるまで待つ
[POD   ] SSH の IP とポートを API から受け取った (<POD_IP>:<SSH_PORT>)
[POD   ] 入れた: cbf7cd00209d
NVIDIA A40, 46068 MiB
    (28 秒)

=== 3a. スクリプトを送る
    (56 秒)

=== 3b. huggingface_hub を入れ, 鍵を渡す
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
Hint: The `hf-cli` skill is not installed. Run `hf skills add -g` to teach your AI agents how to use the `hf` CLI.
✓ Logged in
  user: PixelstoMotors
    (10 秒)

=== 3c. モデルを取る (hf download PixelstoMotors/pi05-mecharm-trim-corr-lora-v2)

Fetching 28 files:   0%|          | 0/28 [00:00<?, ?it/s]
Fetching 28 files:   4%|▎         | 1/28 [00:00<00:02, 10.00it/s]
Fetching 28 files:  32%|███▏      | 9/28 [00:00<00:00, 38.45it/s]
Fetching 28 files:  46%|████▋     | 13/28 [00:10<00:15,  1.05s/it]
Fetching 28 files:  50%|█████     | 14/28 [00:10<00:13,  1.08it/s]
Fetching 28 files:  61%|██████    | 17/28 [00:13<00:10,  1.07it/s]
Fetching 28 files:  68%|██████▊   | 19/28 [00:15<00:08,  1.01it/s]
Fetching 28 files:  71%|███████▏  | 20/28 [00:15<00:06,  1.17it/s]
Fetching 28 files:  79%|███████▊  | 22/28 [00:16<00:03,  1.57it/s]
Fetching 28 files:  82%|████████▏ | 23/28 [00:17<00:03,  1.27it/s]
Fetching 28 files:  86%|████████▌ | 24/28 [00:18<00:03,  1.33it/s]
Fetching 28 files:  89%|████████▉ | 25/28 [00:18<00:01,  1.59it/s]
Fetching 28 files:  93%|█████████▎| 26/28 [00:19<00:01,  1.22it/s]
Fetching 28 files:  96%|█████████▋| 27/28 [00:20<00:00,  1.39it/s]
Fetching 28 files: 100%|██████████| 28/28 [00:28<00:00,  2.60s/it]✓ Downloaded
  path: /workspace/ckpt/corr_v2_1499

Fetching 28 files: 100%|██████████| 28/28 [00:28<00:00,  1.00s/it]
6.0G	/workspace/ckpt/corr_v2_1499
    (32 秒)

=== 3d. openpi を取る (215abfb)
215abfb docs(droid): fix config search instruction (#1023)
    (14 秒)

=== 3e. uv sync
downloading uv 0.13.7 x86_64-unknown-linux-gnu
WARN: The following commands are shadowed by other commands in your PATH: uv uvx
Using CPython 3.11.13 interpreter at: /usr/bin/python3.11
Creating virtual environment at: .venv
Resolved 279 packages in 5ms
   Building openpi-client @ file:///workspace/openpi/packages/openpi-client
   Building openpi @ file:///workspace/openpi
   Updating https://github.com/huggingface/lerobot (0cf864870cf29f4738d3ade893e6fd13fbd7cdb5)
Downloading virtualenv (5.8MiB)
Downloading sympy (6.0MiB)
Downloading imageio-ffmpeg (28.1MiB)
Downloading labmaze (4.7MiB)
Downloading lxml (4.7MiB)
Downloading pillow (4.4MiB)
Downloading nvidia-cufft-cu12 (190.9MiB)
Downloading opencv-python-headless (47.7MiB)
Downloading numpy (17.4MiB)
Downloading llvmlite (40.4MiB)
Downloading nvidia-cublas-cu12 (374.9MiB)
Downloading matplotlib (8.2MiB)
Downloading scipy (35.9MiB)
Downloading ruff (11.0MiB)
Downloading av (34.7MiB)
Downloading nvidia-cuda-nvrtc-cu12 (22.6MiB)
Downloading hf-xet (4.9MiB)
Downloading nvidia-cuda-cupti-cu12 (8.5MiB)
Downloading nvidia-cudnn-cu12 (544.5MiB)
Downloading nvidia-cusparse-cu12 (206.5MiB)
Downloading nvidia-curand-cu12 (53.7MiB)
Downloading pyarrow (40.3MiB)
Downloading pandas (12.5MiB)
Downloading nvidia-nccl-cu12 (192.0MiB)
Downloading dm-control (37.5MiB)
Downloading opencv-python (60.1MiB)
Downloading nvidia-cusolver-cu12 (150.9MiB)
Downloading polars (34.6MiB)
Downloading torchvision (7.1MiB)
Downloading transformers (10.3MiB)
Downloading jax-cuda12-pjrt (99.8MiB)
Downloading rerun-sdk (68.1MiB)
Downloading numcodecs (8.4MiB)
Downloading tensorstore (17.5MiB)
Downloading cmake (26.6MiB)
Downloading jax-cuda12-plugin (15.9MiB)
Downloading wandb (20.4MiB)
Downloading nvidia-nvjitlink-cu12 (18.8MiB)
Downloading fonttools (4.7MiB)
Downloading mujoco (4.4MiB)
Downloading triton (148.5MiB)
Downloading jaxlib (100.2MiB)
Downloading nvidia-cusparselt-cu12 (149.5MiB)
Downloading torch (783.1MiB)
Downloading h5py (4.3MiB)
Downloading numba (3.7MiB)
Downloading pyopengl (3.0MiB)
Downloading hf-transfer (3.4MiB)
Downloading diffusers (3.4MiB)
Downloading nvidia-cuda-nvcc-cu12 (38.7MiB)
 Downloaded pyopengl
Downloading debugpy (3.0MiB)
 Downloaded hf-transfer
 Downloaded diffusers
Downloading tokenizers (2.9MiB)
Downloading jax (2.3MiB)
 Downloaded numba
Downloading widgetsnbextension (2.1MiB)
 Downloaded mujoco
 Downloaded pillow
Downloading networkx (1.9MiB)
Downloading ml-dtypes (2.1MiB)
 Downloaded h5py
 Downloaded labmaze
 Downloaded fonttools
Downloading pydantic-core (1.9MiB)
Downloading jedi (1.5MiB)
 Downloaded hf-xet
Downloading aiohttp (1.6MiB)
Downloading kiwisolver (1.4MiB)
 Downloaded virtualenv
Downloading torchcodec (1.3MiB)
 Downloaded lxml
 Downloaded sympy
Downloading sentencepiece (1.2MiB)
 Downloaded widgetsnbextension
Downloading pygments (1.2MiB)
 Downloaded jax
Downloading setuptools (1.1MiB)
Downloading nvidia-cufile-cu12 (1.1MiB)
 Downloaded kiwisolver
 Downloaded networkx
 Downloaded ml-dtypes
 Downloaded pydantic-core
 Downloaded aiohttp
 Downloaded jedi
Downloading pymunk (1.0MiB)
 Downloaded debugpy
 Downloaded tokenizers
 Downloaded torchvision
 Downloaded torchcodec
 Downloaded sentencepiece
 Downloaded pygments
 Downloaded setuptools
 Downloaded nvidia-cufile-cu12
 Downloaded pymunk
 Downloaded matplotlib
 Downloaded numcodecs
 Downloaded nvidia-cuda-cupti-cu12
   Building evdev==1.9.2
 Downloaded transformers
 Downloaded ruff
   Building antlr4-python3-runtime==4.9.3
 Downloaded pandas
 Downloaded jax-cuda12-plugin
 Downloaded numpy
 Downloaded tensorstore
 Downloaded nvidia-nvjitlink-cu12
      Built openpi-client @ file:///workspace/openpi/packages/openpi-client
      Built openpi @ file:///workspace/openpi
 Downloaded wandb
 Downloaded nvidia-cuda-nvrtc-cu12
 Downloaded imageio-ffmpeg
      Built antlr4-python3-runtime==4.9.3
 Downloaded cmake
 Downloaded polars
 Downloaded av
 Downloaded scipy
 Downloaded llvmlite
 Downloaded pyarrow
 Downloaded nvidia-cuda-nvcc-cu12
 Downloaded dm-control
 Downloaded opencv-python-headless
 Downloaded nvidia-curand-cu12
    Updated https://github.com/huggingface/lerobot (0cf864870cf29f4738d3ade893e6fd13fbd7cdb5)
   Building lerobot @ git+https://github.com/huggingface/lerobot@0cf864870cf29f4738d3ade893e6fd13fbd7cdb5
 Downloaded opencv-python
      Built evdev==1.9.2
 Downloaded rerun-sdk
      Built lerobot @ git+https://github.com/huggingface/lerobot@0cf864870cf29f4738d3ade893e6fd13fbd7cdb5
 Downloaded jaxlib
 Downloaded jax-cuda12-pjrt
 Downloaded nvidia-cusparselt-cu12
 Downloaded nvidia-cusolver-cu12
 Downloaded triton
 Downloaded nvidia-nccl-cu12
 Downloaded nvidia-cufft-cu12
 Downloaded nvidia-cusparse-cu12
 Downloaded nvidia-cublas-cu12
 Downloaded nvidia-cudnn-cu12
 Downloaded torch
Prepared 240 packages in 14.77s
Installed 240 packages in 1.79s
 + absl-py==2.3.0
 + aiohappyeyeballs==2.6.1
 + aiohttp==3.12.4
 + aiosignal==1.3.2
 + annotated-types==0.7.0
 + antlr4-python3-runtime==4.9.3
 + asttokens==3.0.0
 + attrs==25.3.0
 + augmax==0.4.1
 + av==17.0.0
 + beartype==0.19.0
 + beautifulsoup4==4.13.4
 + blinker==1.9.0
 + cachetools==5.5.2
 + certifi==2025.4.26
 + cffi==1.17.1
 + cfgv==3.4.0
 + charset-normalizer==3.4.2
 + chex==0.1.90
 + click==8.2.1
 + cloudpickle==3.1.1
 + cmake==4.0.2
 + comm==0.2.2
 + contourpy==1.3.2
 + crc32c==2.7.1
 + cycler==0.12.1
 + datasets==3.6.0
 + debugpy==1.8.14
 + decorator==5.2.1
 + deepdiff==8.5.0
 + diffusers==0.33.1
 + dill==0.3.8
 + distlib==0.3.9
 + dm-control==1.0.14
 + dm-env==1.6
 + dm-tree==0.1.9
 + docker-pycreds==0.4.0
 + docstring-parser==0.16
 + donfig==0.8.1.post1
 + draccus==0.10.0
 + einops==0.8.1
 + equinox==0.12.2
 + etils==1.12.2
 + evdev==1.9.2
 + executing==2.2.0
 + farama-notifications==0.0.4
 + filelock==3.18.0
 + flask==3.1.1
 + flatbuffers==25.2.10
 + flax==0.10.2
 + fonttools==4.58.1
 + frozenlist==1.6.0
 + fsspec==2025.3.0
 + gcsfs==2025.3.0
 + gdown==5.2.0
 + gitdb==4.0.12
 + gitpython==3.1.44
 + glfw==2.9.0
 + google-api-core==2.24.2
 + google-auth==2.40.2
 + google-auth-oauthlib==1.2.2
 + google-cloud-core==2.4.3
 + google-cloud-storage==3.1.0
 + google-crc32c==1.7.1
 + google-resumable-media==2.7.2
 + googleapis-common-protos==1.70.0
 + gym-aloha==0.1.1
 + gymnasium==0.29.1
 + h5py==3.13.0
 + hf-transfer==0.1.9
 + hf-xet==1.1.2
 + huggingface-hub==0.32.3
 + humanize==4.12.3
 + identify==2.6.12
 + idna==3.10
 + imageio==2.37.0
 + imageio-ffmpeg==0.6.0
 + importlib-metadata==8.7.0
 + importlib-resources==6.5.2
 + iniconfig==2.1.0
 + inquirerpy==0.3.4
 + ipykernel==6.29.5
 + ipython==9.2.0
 + ipython-pygments-lexers==1.1.1
 + ipywidgets==8.1.7
 + itsdangerous==2.2.0
 + jax==0.5.3
 + jax-cuda12-pjrt==0.5.3
 + jax-cuda12-plugin==0.5.3
 + jaxlib==0.5.3
 + jaxtyping==0.2.36
 + jedi==0.19.2
 + jinja2==3.1.6
 + jsonlines==4.0.0
 + jupyter-client==8.6.3
 + jupyter-core==5.8.1
 + jupyterlab-widgets==3.0.15
 + kiwisolver==1.4.8
 + labmaze==1.0.6
 + lerobot==0.1.0 (from git+https://github.com/huggingface/lerobot@0cf864870cf29f4738d3ade893e6fd13fbd7cdb5)
 + llvmlite==0.44.0
 + lxml==5.4.0
 + markdown-it-py==3.0.0
 + markupsafe==3.0.2
 + matplotlib==3.10.3
 + matplotlib-inline==0.1.7
 + mdurl==0.1.2
 + mergedeep==1.3.4
 + ml-collections==1.0.0
 + ml-dtypes==0.4.1
 + mpmath==1.3.0
 + msgpack==1.1.0
 + mujoco==2.3.7
 + multidict==6.4.4
 + multiprocess==0.70.16
 + mypy-extensions==1.1.0
 + nest-asyncio==1.6.0
 + networkx==3.5
 + nodeenv==1.9.1
 + numba==0.61.2
 + numcodecs==0.16.1
 + numpy==1.26.4
 + numpydantic==1.6.9
 + nvidia-cublas-cu12==12.6.4.1
 + nvidia-cuda-cupti-cu12==12.6.80
 + nvidia-cuda-nvcc-cu12==12.9.41
 + nvidia-cuda-nvrtc-cu12==12.6.77
 + nvidia-cuda-runtime-cu12==12.6.77
 + nvidia-cudnn-cu12==9.5.1.17
 + nvidia-cufft-cu12==11.3.0.4
 + nvidia-cufile-cu12==1.11.1.6
 + nvidia-curand-cu12==10.3.7.77
 + nvidia-cusolver-cu12==11.7.1.2
 + nvidia-cusparse-cu12==12.5.4.2
 + nvidia-cusparselt-cu12==0.6.3
 + nvidia-ml-py==12.575.51
 + nvidia-nccl-cu12==2.26.2
 + nvidia-nvjitlink-cu12==12.6.85
 + nvidia-nvtx-cu12==12.6.77
 + oauthlib==3.2.2
 + omegaconf==2.3.0
 + opencv-python==4.11.0.86
 + opencv-python-headless==4.11.0.86
 + openpi==0.1.0 (from file:///workspace/openpi)
 + openpi-client==0.1.0 (from file:///workspace/openpi/packages/openpi-client)
 + opt-einsum==3.4.0
 + optax==0.2.4
 + orbax-checkpoint==0.11.13
 + orderly-set==5.4.1
 + packaging==25.0
 + pandas==2.2.3
 + parso==0.8.4
 + pexpect==4.9.0
 + pfzy==0.3.4
 + pillow==11.2.1
 + platformdirs==4.3.8
 + pluggy==1.6.0
 + polars==1.30.0
 + pre-commit==4.2.0
 + prompt-toolkit==3.0.51
 + propcache==0.3.1
 + proto-plus==1.26.1
 + protobuf==4.25.8
 + psutil==7.0.0
 + ptyprocess==0.7.0
 + pure-eval==0.2.3
 + pyarrow==20.0.0
 + pyasn1==0.6.1
 + pyasn1-modules==0.4.2
 + pycparser==2.22
 + pydantic==2.11.5
 + pydantic-core==2.33.2
 + pygments==2.19.1
 + pymunk==7.0.0
 + pynput==1.8.1
 + pynvml==12.0.0
 + pyopengl==3.1.9
 + pyparsing==3.2.3
 + pysocks==1.7.1
 + pytest==8.3.5
 + python-dateutil==2.9.0.post0
 + python-xlib==0.33
 + pytz==2025.2
 + pyyaml==6.0.2
 + pyyaml-include==1.4.1
 + pyzmq==26.4.0
 + regex==2024.11.6
 + requests==2.32.3
 + requests-oauthlib==2.0.0
 + rerun-sdk==0.23.1
 + rich==14.0.0
 + rsa==4.9.1
 + ruff==0.11.12
 + safetensors==0.5.3
 + scipy==1.15.3
 + sentencepiece==0.2.0
 + sentry-sdk==2.29.1
 + setproctitle==1.3.6
 + setuptools==80.9.0
 + shtab==1.7.2
 + simplejson==3.20.1
 + six==1.17.0
 + smmap==5.0.2
 + soupsieve==2.7
 + stack-data==0.6.3
 + sympy==1.14.0
 + tensorstore==0.1.74
 + termcolor==3.1.0
 + tokenizers==0.21.1
 + toml==0.10.2
 + toolz==1.0.0
 + torch==2.7.1
 + torchcodec==0.4.0
 + torchvision==0.22.1
 + tornado==6.5.1
 + tqdm==4.67.1
 + tqdm-loggable==0.2
 + traitlets==5.14.3
 + transformers==4.53.2
 + treescope==0.1.9
 + triton==3.3.1
 + typeguard==4.4.2
 + typing-extensions==4.13.2
 + typing-inspect==0.9.0
 + typing-inspection==0.4.1
 + tyro==0.9.22
 + tzdata==2025.2
 + urllib3==2.4.0
 + virtualenv==20.31.2
 + wadler-lindig==0.1.6
 + wandb==0.19.11
 + wcwidth==0.2.13
 + websockets==15.0.1
 + werkzeug==3.1.3
 + widgetsnbextension==4.0.14
 + wrapt==1.14.1
 + xxhash==3.5.0
 + yarl==1.20.0
 + zarr==3.0.8
 + zipp==3.22.0
    (21 秒)

=== 3f. addon・設定・assets (pi05_mecharm_trim_corr_v4_lora)
installed: policies/mecharm_policy.py + config.py (pi05_mecharm_lora)
installed: pi05_mecharm_trim_lora (repo_id=local/mecharm_redblock_box_teach_v1_trim2, 4,000 steps)
installed: pi05_mecharm_trim_corr_v4_lora (repo_id=local/mecharm_redblock_box_teach_v1_trim2_corr_v4b, 1500 steps, init=/workspace/ckpt/corr_v1_1499/params)
    (2 秒)

=== 4. サーバーの起動
[SERVE ] --model corr2: --policy.config=pi05_mecharm_trim_corr_v4_lora --policy.dir=/workspace/ckpt/corr_v2_1499
INFO:websockets.server:server listening on 0.0.0.0:8000