Nvidia Isaac Sim 1 - 介绍

1. Isaac Sim 是什么？

NVIDIA Isaac Sim™ is a reference application built on NVIDIA Omniverse that enables developers to develop, simulate, and test AI-driven robots in physically-based virtual environments.

NVIDIA Isaac Sim™ 是构建在 NVIDIA Omniverse 之上的参考应用，使开发者能够在基于物理的虚拟环境中开发、仿真、测试由 AI 驱动的机器人。

2. 设计

Isaac Sim comes with a collection of workflows for importing and tuning mechanical systems designed in the most common formats including Onshape, the Unified Robotics Description Format (URDF), and the MuJoCo XML Format (MJCF). This is made possible through the use of the Universal Scene Description (USD), an easily extensible, open source 3D scene description API that serves as the unifying data interchange format at the heart of Isaac Sim.

Isaac Sim 提供一套工作流，可用于导入及调优以常见格式设计的机械系统，包括 Onshape、统一机器人描述格式（URDF）以及 MuJoCo XML 格式（MJCF）。这之所以能够实现，是因为它使用通用场景描述（USD）：一种易于扩展的开源 3D 场景描述 API，它也作为 Isaac Sim 核心的统一数据交换格式。

3. 调优与训练

The core functionality of Isaac Sim is the simulation itself: a high-fidelity GPU-based PhysX engine, capable of supporting multi-sensor RTX rendering at an industrial scale. Isaac Sim’s direct access to the GPU enables the platform to support the simulation of various kinds of sensors including cameras, Lidars, and contact sensors. This in turn facilitates the simulation of digital twins, allowing your end-to-end pipelines to run before ever needing to turn on a real robot. Isaac Sim provides a suite of tools for collecting synthetic data with Replicator, orchestrating simulated environments through Omnigraph, tuning PhysX simulation parameters to match reality, and finally training control agents through various methods like Reinforcement Learning (RL) with Isaac Lab.

Isaac Sim 的核心功能是仿真本身：它采用高保真的、基于 GPU 的 PhysX 引擎，能够支持工业级规模下的多传感器 RTX 渲染。Isaac Sim 可直接访问 GPU，因此平台能够支持多种传感器的仿真，包括相机、激光雷达（LiDAR）和接触传感器。进而使数字孪生仿真成为可能，使端到端流程在真正启动实体机器人之前就能先运行起来。Isaac Sim 还提供一整套工具：使用 Replicator 采集合成数据；通过 OmniGraph 编排仿真环境；通过调优 PhysX 仿真参数的方式，贴近真实世界。最终通过多种方法（比如借助 Isaac Lab 的强化学习）训练控制智能体。

4. 部署

Isaac Sim comes pre-equipped with all of the components necessary to not only deploy agents to real robots, but also build applications that are fully integrable with such systems. Omniverse provides APIs for app infrastructure including GUI creation and file management. The Isaac Sim platform also provides bridge APIs to ROS 2, for direct communication between live robots and the simulation, as well as NVIDIA Isaac ROS, a collection of performant, hardware-accelerated ROS 2 packages for making autonomous robots.

Isaac Sim 预置全部必需组件，不仅能将智能体部署到实体机器人，还能构建可与这些系统完全集成的应用。Omniverse 提供用于应用基础设施的 API，包括 GUI 创建和文件管理。Isaac Sim 平台还提供面向 ROS 2 的桥接 API，可实现真实机器人与仿真之间的直接通信；同时也支持 NVIDIA Isaac ROS - 一套高性能、硬件加速的 ROS 2 软件包，用于构建自主机器人。

5. 系统架构

The purpose of Isaac Sim is to support the creation of new robotics tools and empower the ones that already exist. The platform provides a flexible API for both C++ and Python and can be integrated into a project to varying degrees depending on your needs. The goal of the platform is not to compete with current or existing software, but to collaborate with and enhance it. To this end, many components of Isaac Sim are open source and freely available for independent use. You may want to design your robot in OnShape, simulate its sensors with Isaac Sim, and control the stage through ROS or some other messaging system. Likewise, it is also possible to build a complete, standalone application entirely on the platform provided by Isaac Sim!

Isaac Sim 的目标是支持新机器人工具的创建，及赋能现有工具。该平台同时提供 C++ 和 Python 的灵活 API，可根据需求以不同程度集成到项目中。该平台的目标不是与当前或已有软件竞争，而是与其协作，增强其能力。为此，Isaac Sim 的许多组件都是开源的，可免费独立使用。可以在 OnShape 中设计机器人，在 Isaac Sim 中仿真其传感器，通过 ROS 或其他消息系统控制场景。同样，也可以完全基于 Isaac Sim 提供的平台构建完整、独立的应用程序。

6. Omniverse Kit

Isaac Sim uses the Omniverse™ Kit, a toolkit for building native Omniverse applications and microservices. Omniverse Kit provides a wide variety of functionality through a set of lightweight plugins. Plugins are authored with C interfaces for persistent API compatibility; however, a Python interpreter is also provided for accessible scripting and customization.

The Python API can be used to write new extensions to Omniverse Kit or new experiences for Omniverse.

Isaac Sim 使用 Omniverse™ Kit - 用于构建原生 Omniverse 应用和微服务的工具包。Omniverse Kit 通过一组轻量级插件提供丰富多样的功能。插件采用 C 接口编写，以确保 API 的长期兼容性；平台也提供 Python 解释器，以便更便捷地进行脚本编写和功能定制。

Python API 可用于为 Omniverse Kit 编写新扩展，或为 Omniverse 构建新 Experience。

7. 开发工作流

Isaac Sim is built on C++ and Python, and operates most commonly through the use of compiled plugins and bindings respectively. This means the platform is capable of supporting a wide variety of workflows for building and interacting with projects that make use of Isaac Sim. Isaac Sim comes with a full, standalone Omniverse application for interacting with and simulating robots, and while this is the most common way users interact with the platform, it is by no means the only method. Isaac Sim also provides direct Python development support in the form of extensions for VS Code and Jupyter Notebooks. Isaac Sim is not limited to synchronous operation either, and can operate with hardware in the loop through ROS 2, facilitating sim-to-real transfer and digital twins.

Isaac Sim 基于 C++ 和 Python 构建，最常用的工作方式是分别使用已编译的插件和绑定。这意味着该平台能够支持多种工作流，以构建使用 Isaac Sim 的项目，及与其进行交互。Isaac Sim 自带完整且独立的 Omniverse 应用，用于与机器人交互及进行仿真；虽然这是最常见的使用方式，但绝不是唯一方式。Isaac Sim 还通过 VS Code 扩展和 Jupyter Notebook 的形式直接支持 Python 开发。Isaac Sim 也不限于同步运行模式，还可通过 ROS 2 实现硬件在环运行，从而支持仿真到真实（sim-to-real）迁移和数字孪生。

8. USD

NVIDIA Isaac Sim uses the USD interchange file format to represent scenes. Universal Scene Description (USD) is an easily extensible, open-source 3D scene description file format developed by Pixar for content creation and interchange among different tools. Because of its power and versatility, USD is being adopted widely, not only in the visual effects community, but also in architecture, design, robotics, manufacturing, and other disciplines.

NVIDIA Isaac Sim 使用 USD 交换文件格式表示场景。通用场景描述（USD）是易于扩展的开源 3D 场景描述文件格式，由 Pixar 开发，用于不同工具之间的内容创建与交换。由于其强大的功能和广泛的适用性，USD 正在被广泛采用，不仅在视觉特效领域如此，在建筑、设计、机器人技术、制造业以及其他领域也同样如此。

9. Appendix 1 - 数字孪生

数字孪生是将现实世界中的对象、设备、产线、建筑、城市或流程，在数字空间里建立一个“动态对应体”。

它不只是 3D 模型，而是持续接收现实对象的数据，比如传感器数据、运行状态、环境变化、历史记录等，用这些数据模拟、监控、分析和预测现实对象的行为。

举个例子：一台真实机械臂在工厂里运行，数字孪生系统里也有一台对应的虚拟机械臂。真实机械臂的关节角度、温度、负载、故障信息将实时同步到虚拟模型中。工程师可以在数字空间里观察它、预测故障、测试新动作策略，而不必直接打断真实设备。

简而言之：

数字孪生 = 现实对象 + 实时数据 + 数字模型 + 仿真/预测/优化能力。