Human video to humanoid motion transfer: training and verification in simulation
Run under lab conditions; no installation at a customer site.
- Industry
- R&D, demos and events
- Customer
- Robotlar.org R&D
- Period
- 2026-07 → sürüyor
- Operating mode
- Human in the loop
Environment
Where the robot works
Lab: a local development machine, a rented cloud GPU and a physics simulator. Execution on the robot has not been done at this stage.
Problem
Operational problem
Event and demo requests ask the robot to perform a specific choreography. The ready-made motion library is limited, and hand-programming a new motion takes a long time.
Task
What the robot does
- 1Three-dimensional body motion is extracted from human video.
- 2The motion is retargeted onto the robot's joint structure.
- 3A motion-tracking policy is trained in the physics simulator.
- 4The policy is evaluated in simulation and failing motions are discarded.
Hardware
Hardware in use
Unitree G1 EDU
The target platform; at this stage only its kinematic model was used.
Yerel GPU iş istasyonu
Data preparation and evaluation.
Kiralık bulut GPU
Policy training; the instance is shut down when the job finishes.
Components used in the installation.
Software
Software and AI
Video kestirimi ve yeniden hedefleme
Body motion from video, then to robot joint angles.
MuJoCo tabanlı eğitim
The motion-tracking policy is trained with reinforcement learning.
ONNX aktarımı
The trained policy is exported in a form the robot runtime can load.
Workflow
Mission flow
- Video selection and licence check: motion data closed to commercial use is not used.
- Motion extraction and transfer onto the robot skeleton.
- Training in simulation; the cloud instance is shut down when the job ends.
- Evaluation and elimination in simulation.
- Next step: a controlled trial on the robot (not done).
Human
The human role
Human in the loop
Motion selection, licence checks, training settings and elimination decisions stay with the engineer. Running on the robot happens only under a safety procedure and physical supervision.
Safety
Safety approach
- No motion trial has been run on the robot; that step is bound to a separate safety procedure.
- Licence gate: motion datasets closed to commercial use are excluded from the start.
Limits
What the system cannot do
- This is lab work. The policy has not been run on the robot, and a motion that works in simulation is not guaranteed to work on the real one.
- The licence of the motion data decides the outcome: part of the common datasets is closed to commercial use.
- Training needs cloud GPU budget; every new motion means a separate training run.
- Not ready for a customer demo: stage choreography, safety distance and operator procedure are not defined.
Results
Measured results
No measurement on record.
- The end-to-end chain was built and run in simulation.
- The licence chain was mapped: which motion data is open to commercial use is documented.
No measurement was taken on the robot; there is no success-rate, repeatability or robustness data.
Platform
The robot used
Values are manufacturer data.
Site survey for a similar deployment
This deployment was configured around a single site's conditions. Floor, route, network coverage, charging point and safety rules differ from site to site; the scope is set by the survey report.