原创性声明
本文为作者原创，在个人Blog首次发布，如需转载请注明引用出处。（yanzhang.cg@gmail.com 或 https://graphicyan.github.io/）
0 引言
1 物理引擎综述
1.1 常见的物理引擎
下图中展示了常见的物理引擎，关于其性能分析可见文献[1]。其中ODE、Bullet、DART消费级机器人仿真中应用广泛，OpenSim用于人体肌肉组织仿真，MuJoCo(/mud͡ʒowkow/)主要应用于强化学习和最优化控制领域，RaiSim是ETH Robotic Systems Lab开发的目前最新的物理引擎，应用于他们实验室开发的四足机器人ANYmal。Bullet、PhysX、Havok应用于游戏，Bullet由于版权优势在一些机器人领域有替代MuJoCo趋势。Flex是NVIDIA开发的引擎,在流体模拟、NVIDIA自家的GPU加速方面有优势，NVIDIA内部研究机构也有尝试应用于机器人[2]。
ODE
[ODE](https://www.ode.org/) is an **open-source** physics engine. It is probably the engine most commonly used in robotics applications, most notably in the VRC. It is integrated with Gazebo and V-REP, as well as other robotics frameworks. ODE implements a **sophisticated integrator** for angular DOFs, a feature that contributes to its performance in the energy and angular momentum tests. ODE has an **iterative solver** and an **exact solver**. As part of the VRC effort, OSRF has developed implicit damping for ODE (John Hsu, personal communication) but this has not yet been merged with the main version in the official repository.
Bullet
[Bullet ](https://pybullet.org/wordpress/)is another **open-source** physics engine that is also integrated with many of the popular robotics software platforms, including V-REP and Gazebo. While Bullet has built-in functionality for **spring-dampers** at hinge joints, its damping functionality does not conform to the standard PD controller design pattern: it is impulsebased, and therefore uses abnormal units to specify damping (instead of using Nms/rad, the damper’s parameter roughly corresponds to “what fraction of the velocity should be retained in the next timestep”).
PhysX
[PhysX](https://github.com/NVIDIA-Omniverse/PhysX) together with Havok, is among the most widely used gaming engines. It also makes the most drastic compromises in terms of physical accuracy – in particular it **ignores** **Coriolis forces** . This alone makes it unsuitable for robotics applications where accuracy is important. While PhysX supports hinge joints, we used a constrained 6D joint instead for the benefit of using a built-in PD controller class PxD6Drive. We also implemented an articulation-based instance, but we experienced several difficulties: first, articulation joints have 3 DOFs, so in order to create a hinge we had to impose artificially-tight swing limits. Furthermore, the articulation API does not offer a direct way to compute joint angles. We therefore do not include these results in our comparisons.
Havok
[Havok](https://www.havok.com/) also ignores Coriolis forces, and therefore is also unsuitable to applications where accuracy is important. Note however that a Havok engineer suggested a possible way to manually introduce such forces . Havok does not support plane geometries, and therefore we implemented the ground plane using a big box, setting the collision margin (setRadius) to 0. The Havok API does not provide a way to query the angle of a hinge constraint; we tried to implement this feature in several ways, including following the example of ODE’s codebase and advice from Havok engineers on the developers’ forum , but none of these solutions worked well enough. Therefore, we currently have no working PD controller for Havok, and it is therefore excluded from the grasping test in the current version.
MuJoCo
[MuJoCo](https://mujoco.org/) is the engine we have developed and used extensively in our research over the past 5 years. MuJoCo has a built-in implementation of Hinge PD controllers that uses implicit damping. We present results for two versions of MuJoCo, using the semi-implicit Euler intrator vs. the 4th-order Runge-Kutta integrator.
1.2 游戏物理 vs 机器人仿真
游戏市场对机器人仿真领域冲击很大。积极上说，游戏产业的发展加速了对物理现象仿真的研究，并且随着游戏物理引擎逼真度的提升，越来越多的功能被应用在机器人仿真领域。不过机器人仿真跟游戏还是有上述本质区别，最基本的从底层力学工具就已经完全区分开: 正经的 游戏物理引擎 一般不过多考虑刚体系统的性能优化，采用简单的牛顿力学工具，有助于游戏开发者理解原理；机器人物理引擎 则倾向于采用Featherstone的空间向量方法(如下图)，保证尽可能高效的刚体动力学算法实现(如MuJoCo、RaiSim)。
游戏引擎侧重点在视觉上的真实感。
机器人引擎（动力学仿真）侧重点在物理上的正确性。
1.3 机器人仿真
机器人物理引擎狭义上仅对刚体前向动力学的模拟器,如ODE。这种物理引擎输入用户端控制量(如关节电机力矩、空间作用力)，对刚体系统碰撞、接触和力学进行解算，输出整个系统的加速度，再通过积分器得到下一个系统状态，相关算法可参考前向动力学。如Gazebo、V-rep这些仿真器内置了多种常见物理引擎，可以让用户自行选择。
广义上的物理引擎还需要上述传感器、驱动器、视觉渲染等模块，此时物理引擎更倾向于一种轻量的仿真器。比如 MuJoCo 即属于这种广义物理引擎，可以将引擎模块用于其他仿真器中(如集成在Unity)，也可以直接用物理引擎来做仿真，如pybullet。
1.4 机器人引擎数据对比
1.4.1 使用率
1.4.2 特性排序
1.4.3 仿真效率对比
- Grasping (35-DOF Robotic Arm)
- Humanoid (25-DOF Humanoid)
- Planar Chain (5-DOF Planar Kinematic Chain)
- 27 Capsules (Randomly-oriented capsules, 27x6 = 162DOFs)
MuJoCo 遥遥领先[6]。
1.5 学界研究方向
1.5.1 传统仿真
数学模型为王，spring-mass, PBD，x-CCD，IPC等。
1.5.2 可微方向（AI)
目前在机器人初现的是将物理引擎跟控制结合起来。如强化学习中默认环境转移 P(st+1|st,at) 是黑箱，即使对环境建模也只是通过采样方法得到一个环境近似模型(即基于模型强化学习)，并没有利用到物理建模的先验知识。最近的研究如将物理引擎变成可微分形式[3], 将控制看成一个结合环境模型的优化问题[4]，可以将控制和物理仿真结合在一起。另外就是减小sim-to-real gap。因为物理引擎内部已经建立了环境的系统模型，一些机器人标定、辨识的底层工具箱完全可以整合在引擎中。
2 MuJoCo
2.1 简介
MuJoCo is a free and open source physics engine that aims to facilitate research and development in robotics, biomechanics, graphics and animation, and other areas where fast and accurate simulation is needed.
https://mujoco.org/
https://github.com/deepmind/mujoco
MuJoCo全称为Multi-Joint dynamics with Contact，主要由华盛顿大学的Emo Todorov教授开发，应用于最优控制、状态估计、系统辨识等领域，在机器人动态多点接触的应用场合(如多指灵巧手操作)有明显优势。不同于其他引擎采用urdf或者sdf等机器人模型， MuJoCo引擎团队自己开发了一种机器人的建模格式(MJCF)，来支持更多的环境参数配置。另外开发者提出了一种全新的soft contact模型，来表征复杂的柔性表面接触和摩擦力,以此推导出全新的约束求解器和动力学模型,详细内容可见文献[5] (该文献也是笔者读过的最叹为观止的机器人领域论文，短短八页内容出现了近50个公式)。为了提高仿真性能，MuJoCo做了AVX指令等大量优化，是极少的选择C语言来实现的现代物理引擎之一。
2.2 核心特点
- Simulation in generalized coordinates, avoiding joint violations
- Inverse dynamics that are well-defined even in the presence of contacts
- Unified continuous-time formulation of constraints via convex optimization
- Constraints include soft contacts, limits, dry friction, equality constraints
- Simulation of particle systems, cloth, rope and soft objects
- Actuators including motors, cylinders, muscles, tendons, slider-cranks
- Choice of Newton, Conjugate Gradient, or Projected Gauss-Seidel solvers
- Choice of pyramidal or elliptic friction cones, dense or sparse Jacobians
- Choice of Euler or Runge-Kutta numerical integrators
- Multi-threaded sampling and finite-difference approximations
- Intuitive XML model format (called MJCF) and built-in model compiler
- Cross-platform GUI with interactive 3D visualization in OpenGL
- Run-time module written in ANSI C and hand-tuned for performance
2.2.1 结合先进接触力学、广义坐标系统
2.2.2 柔性、凸模型并且解析可逆的接触力学
- 现代物理引擎大多数求解线性互补问题来处理约束，MuJoCo允许软体接触和其他约束，并包含一个独有的逆动力学模型来做数据分析。提出了新的摩擦力模型，支持滚动摩擦、扭转摩擦等多种摩擦力仿真。
2.2.3 肌腱仿真
2.2.4 广义驱动器模型
2.2.5 可灵活配置的仿真流程
2.2.6 先进的模型、编译方法
2.2.7 强大的环境建模语言
2.2.8 高效的软体材料仿真
2.3 对比游戏引擎
- Gaming engines use a more modern approach where contact forces are found by solving an optimization problem. However, they often resort to the over-specified **Cartesian **representation where joint constraints are imposed numerically, causing inaccuracies and instabilities when elaborate kinematic structures are involved. MuJoCo was the first general-purpose engine to combine the best of both worlds: simulation in **generalized coordinates **and optimization-based contact dynamics.
2.4 数据格式
2.5 约束模型
2.5.1 等值约束
可以通过r(q)=0来建模等值约束，其中r是任意可微标量/向量函数,q是位置函数，r在语义上对应了残差(residual)，约束雅克比J中对应的块为∂r/∂q。注意由于四元数特性，∂r/∂q产生了nV维度的向量而不是nQ。
在其他应用如游戏引擎中，等值约束可以用于创建环关节(loop joints)，但是在MuJoCo中不推荐这样使用——因为这样会造成精度和求解效率下降。MuJoCo中唯一合理使用等值约束的地方是对不能采用运动链的柔性关节(soft joints)的建模。有五种等值约束类型和其constraint residual的维度。
2.5.2 摩擦损失
摩擦损失(dry friction, static friction, load-independent friction)，为静摩擦力。跟阻尼或者粘滞力相似，其作用方向跟运动方向相反，不能建模为速度相关的力。因此这里定义其为约束，即静摩擦力幅值的上限。这个值可以在模型元素frictionloss中指定，可以施加在joint或者tendon上。
与其他约束类型不同，摩擦损失没有位置残差，因此我们设置r(q)=0。事实上为了整合进这个约束，我们的约束求解器需要进行扩展。另外受约束关节或者线缆的速度作为了速度“residual”，因为这个约束是为了尽量使速度趋于零，对应约束雅克比中的块为关节位置(或者线缆长度)对q的偏导:对应标量关节时one-hot向量，对应线缆时力臂向量。
2.5.3 限位
限位是单边的空间残差约束，是不等值约束。限位可适用于关节(MJCF文件中limited)和线缆(range)。残差r(q)定义为当前值跟最大限制的距离。如果没有越过限制，则距离为正；反之为负。当此距离小于MJCF中定义的margin时，此约束被激活(不过并不意味着可以抵消margin的限制，设置margin为0。约束力是通过求解器中的参数来定义的)。
有可能关节/线缆的上界和下界同事激活。此时它们都会被包含在标量约束集合中。但是这种情况不符合物理现实，需要手动增加限制范围或者修改margin值来避免。注意避免用小范围限制来近似等值约束，而是直接使用等值约束，通过调节求解器参数来保证约束的松弛度(这样能保证更高的计算效率，因为等值约束计算更快)。
2.5.4 接触
接触是最复杂的约束类型，因为接触建模的复杂性。MuJoCo可以支持通用的接触模型(切向、扭转滚动摩擦力)，支持椭圆摩擦锥和四面体摩擦锥。
MuJoCo接触采用点接触方法，几何上定义两个刚体间的接触点和固接在该点上的空间坐标系,均表达在全局坐标系中。坐标系的第一个轴(X轴)指向接触法向，另外两个轴(Y和Z)定义了切空间。定义法向轴为X轴的原因是为了让第一维度为法线方向，从而支持多种参数(比如可以没有Y、Z方向的摩擦力)。跟限位一样，当两个geom分离时为正、插入时为负。接触点在两个geom表面中间位置。
2.6 Unity插件
https://mujoco.readthedocs.io/en/latest/unity.html
包括模型导入、仿真等功能，用Unity来渲染看结果。
3 参考文献
[1] Ivaldi, Serena, et al. “Tools for simulating humanoid robot dynamics: a survey based on user feedback.” 2014 IEEE-RAS International Conference on Humanoid Robots. IEEE, 2014.
[2] Chebotar, Yevgen, et al. “Closing the sim-to-real loop: Adapting simulation randomization with real world experience.” 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019.
[3] Heiden, Eric, et al. “NeuralSim: Augmenting Differentiable Simulators with Neural Networks.” arXiv preprint arXiv:2011.04217 (2020).
[4] UW CSE Robotics: Emo Todorov, “Goal-directed Dynamics”
[5] Todorov, Emanuel. “Convex and analytically-invertible dynamics with contacts and constraints: Theory and implementation in mujoco.” 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2014.
[6] Simulation Tools for Model-Based Robotics: Comparison of Bullet, Havok, MuJoCo, ODE and PhysX