NVIDIA Isaac: Everything You Need To Know About NVIDIA’s New Platform

NVIDIA Isaac SDK is the first open-source Robotic AI Development Platform with Simulation, Navigation, and Manipulation.

It’s a robust platform that helps you build smarter robots.

NVIDIA Isaac SDK heavily relies on AI.

As they put it: “AI makes it possible for robots to perceive and interact with their environments in novel ways, enabling them to perform tasks that were unthinkable—until now.”

NVIDIA Isaac SDK comes with a collection of powerful GPU-powered algorithms, frameworks, basic applications that support accelerated robotic development. It also works hand-in-hand with Isaac SIM, which allows for the development, testing, and training of robots in a virtual environment.

In short: NVIDIA Isaac SDK heavily uses GPUs to increase performance and help you run better simulations faster.

What Can NVIDIA Isaac SDK Help You With?

NVIDIA Isaac SDK can help you create, modify & simulate your entire factory, even before installing any equipment. 

There are a lot of premade pallets, cardboard boxes, shelves, totes, bins, and everything that you’d see in your standard warehouse.

It’s all out there, in the simulation. 

The great thing about the simulation is that the physics are amazingly accurate. 

You don’t want to spend months in a simulation trying to create the perfect Robot for your warehouse and then having it all crash and burn because the gravity in your simulation is different from real-life gravity.

You can simulate your parts with 3d models. Add in the weights, center of gravity, and the simulation will interact with it really close like it would in the actual manufacturing process. 

Besides doing cool simulations, you can also use NVIDIA’s AI in your simulation to add stuff like:

  • 3d vision
  • Part recognition
  • Neural networks
  • Reinforcement learning

Create real-time AI simulations using the power of their RTX graphics cards. 

Who Can Use The NVIDIA Isaac SDK

Anyone can download the software right away by simply heading to NVIDIAs download page, but this isn’t the biggest obstacle. 

Learning new programming languages is the biggest challenge, and fortunately for all kids fresh out of college. The whole thing can be programmed inside of python using the Isaac SDK.

Before Isaac, industrial applications were generally programmed by ladder logic and more archaic types of languages. 

Seeing as this is a python based application is excellent for companies too. If you’re looking to hire your team for the job, you will be able to tap into a much greater talent pool of excellent developers since python is a much more popular language.

The development of robotic applications is still a tough job for most companies, especially when most developers relied on Gazebo. The developers of the latter are much smaller in scale compared to NVIDIA and now Unity. 

Seeing as more prominent companies are getting into Robotics shows just how powerful and useful they are. 

For accelerated robotic development, NVIDIA provided a collection that helps in development, training, and testing. Thus the complexity of robotic development was reduced to a great extent. Developers can now try the Isaac collection, which is well documented and have proper community support for robotic development.

The NVIDIA Isaac Ecosystem

NVIDIA Isaac Ecosystem

Nvidia Isaac SDK consists of several parts that work well together to create some pretty powerful simulations.

1. Isaac Engine

Isaac engines is a software framework for building modular robotic applications.

It consists of computational graphs & CUDA messaging, Visualization Tools, and Python API & Ros Bridge.

It’s used to build robotic applications based on many small components that pass messages between each other and can be customized any way you like.

2. Isaac GEMs

Isaac GEMs are a collection of GPU-powered algorithms that help accelerate the development of robotic applications.

It consists off:

  • Multi-class segmentation DNN
  • Stereo Visuallnertial odometry
  • Multi lidar localization & Collision Avoidance
  • 2D skeleton pose estimation DNN
  • 3d object pose estimation (with depth) DNN
  • Superpixels
  • Motion planning for navigation & manipulation
  • Audio DNNs
  • Object detection DNN
  • Global planner with cost-maps
  • Reinforcement learning
  • Sensor and robot integrations
  • Stereo depth DNN
  • GPU accelerated AprilTags
  • DeepStream for Robotics
  • And much more

3. Isaac Sim

Isaac Sim is a virtual robotics laboratory and a high-fidelity 3d world simulator that accelerates research, design, and development by reducing cost and risk.

This helps you test robots in different scenarios.

Robots can be simulated with virtual sensors (RGB, stereo, depth, LIDAR, IMU)

It consists off:

  • Domain randomization
  • Scenario management
  • Sensor models
  • Robot models

4. Isaac Apps

These are basic applications that make use of the NVIDIA Isaac SDK engine to showcase the real power of the NVIDIA Isaac SDK and help you get started quickly.

Problems NVIDIA Isaac SDK Solves

One of the biggest problems with old simulation software is that you don’t know how your automation will respond to your environment. 

NVIDIA Isaac acknowledges that you need an excellent way to simulate what your parts will do whenever you’re automating.

What this accomplishes is that it speeds up your development time because it’s clearing up the unknown unknowns.

This means that robot development is much more rapid deployment. As we mentioned before, you’re now able to get a bigger talent pool of programmers involved in your projects.

The best part of the NVIDIA Isaac SDK is using cloud computing to do all of your development.

Anyone in the world can easily buy an instance like the Amazon Elastic Compute Cloud and develop these programs remotely. This means that even people who don’t have a $2,000 graphics card.

Your average laptop should be able to do the job.

2020 Nvidia Isaac SDK Update

The 2020.1 version brought us a lot of new possibilities for Nvidia Isaac.

Here’s a summary from Nvidia’s official website:

  • 3D Object Pose Estimation: Accurate DNN models consist of object detection, 3D pose estimation, and pose refinement using depth sensor data. This can be used for various purposes, e.g., for detection/pose estimation of a dolly in a warehouse for a robot to drive under or for detection/pose estimation of a container for a cobot (like Universal Robots’ UR10) to pick and place objects.
  • Multi Lidar Support: In a grid-map based global localization and for particle-filter in continuous localization, having two lidars helps with robust localization in an ambiguous environment; and, reduced location uncertainty. Multiple lidars also help with much better perception (360 degrees) for collision avoidance.
  • Path Planning with Cost maps for Navigation: This release offers a way to customize graphs, in an open space (like a factory) where robots can use optimal paths (i.e., have a notion of lanes) rather than taking the shortest path to the destination.
  • Motion Planning for Manipulation: LQR-based motion planning is included for manipulation robots.
  • Virtual Factory Space: A massive 125 meters x 50 meters area in Isaac Sim to train and test robots.
  • Docker Container: Enables robotics ML engineers use Isaac Sim to generate synthetic images and train an object detection DNN (included with Isaac SDK) with NVIDIA’s Transfer Learning Toolkit. 
  • Experimental RL Application: Teaches a robot to navigate under a cart placed in front of it, using a deep neural network.
  • Support for Quadruped: With support for quadruped robots like Unitree Laikago, this release adds to the other robot platforms supported, such as differential wheelbase, holonomic wheelbase and 6 DoF Cobots.
  • Support for New Sensors and Robots: This release also expands on the ecosystem of sensor and robot platforms that have embraced Isaac SDK. This includes support from our ecosystem partners including LIPS AE400 stereo camera, SICK Microscan3 Pro lidars, Basler cameras, Universal Robots’ Cobots, and more.

You may also like:

Leave a Reply

Your email address will not be published. Required fields are marked *