View
Case Study
View
more
Request a quote
example
Back to Blog
06 June 2024

Edge AI and Robotics with Generative AI Will Greatly Benefit Industries 

By Caio Viturino and Bobby Carlton 
Nvidia Omniverse, Edge

Edge AI’s capabilities are set to redefine industries, offering faster development cycles, increased accuracy, and the ability to recognize and interact with unseen elements.

With the robust Edge AI simulation platform Omniverse Isaac Sim by NVIDIA, FS Studios has successfully delivered projects for major companies such as Universal Robots, UPS, Third Wave Automation, Agility Robotics and others. Our simulation team possesses the expertise to develop customized solutions for industrial, agricultural, and domestic environments using NVIDIA’s simulation tools and artificial intelligence. Through simulation, we optimize processes, generate databases of images, point clouds, and more, for neural network training and beyond. 

In the face of the exponential growth of Omniverse tools, investing in setting up industrial cells to assess performance or train robots for object manipulation no longer makes sense. Now, all of this is achievable through simulation and Ai alone. FS Studio artists work with powerful design tools seamlessly integrated with Omniverse tools, enabling us to swiftly create a digital twin of your factory, assembly line, agriculture, or home environment with exceptional quality. 

We are dedicated to transforming how industries tackle problem-solving and training by harnessing the potential of cutting-edge simulation technology. The Omniverse tools are the ideal pathway forward! 

Generative AI: Transforming Industries at the Edge 

Generative AI, fueled by transformer models and large language models, has become a force to reckon with, impacting virtually every industry. NVIDIA is extending its reach to the edge, offering applications that go beyond traditional convolutional neural network (CNN) models. This includes defect detection, real-time asset tracking, autonomous planning and navigation, human-robot interactions, and more. 

The potential impact is significant, as generative AI is expected to add $10.5 billion in revenue for manufacturing operations globally by 2033, according to ABI Research. 

Case Study Project A) Working With a 3PL To Use Synthetic Data and Ai for Computer Vision Models 

For this project, our client sought to validate whether incorporating NVIDIA Omniverse Isaac Sim and Ai into their computer vision model could significantly enhance its performance. The primary objective was to create a diverse dataset of photorealistic objects, simulating different package types, shapes, sizes, and colors. These synthetic objects would be utilized to train and fine-tune their AI algorithms, enabling them to recognize and classify packages effectively under challenging real-world conditions. 

Our Solution 

Synthetic Object Creation: 

FS Studio embarked on the task of generating 100 photorealistic objects of varying shapes, sizes, and colors using Omniverse Replicator. These objects were meticulously crafted to mimic real-world items commonly encountered in logistics and transportation scenarios. Notably, the project involved the creation of objects such as Forever bags, Paint Cans, Exhaust Manifolds, Barbells, Pallets, and more, as specified by the client. The primary focus was on ensuring the highest quality textures and materials to replicate the appearance of actual objects. 

Environment Simulation: 

In addition to creating objects, FS Studio designed a realistic trailer interior and simulated various lighting conditions within the Omniverse environment. The objective was to accurately replicate the challenging lighting scenarios often encountered in package handling facilities, aiding the AI model in adapting to diverse illumination sources and conditions. 

AI Data Enhancement: 

The synthetic data generated was strategically integrated into the client’s AI training pipeline. These high-quality synthetic objects and environments enabled the AI system to learn and adapt to a wide range of package characteristics, including shapes, colors, and sizes. By leveraging synthetic data, the client could significantly improve their AI’s ability to detect and classify packages accurately, regardless of the surrounding conditions. 

The Results 

By harnessing the power of synthetic data generation through NVIDIA Omniverse, our client was able to enhance their computer vision model’s performance significantly. The diverse dataset of photorealistic objects and simulated environments allowed their AI systems to adapt and excel in recognizing various types of packages under challenging real-world conditions. 

This Omniverse project exemplifies how synthetic data can revolutionize computer vision capabilities and drive improvements in logistics and package handling processes. 

Case Study Project B) Revolutionizing Robot Programming Education: Universal Robotics Academy’s Innovative Virtual Learning Environment 

Universal Robotics Academy came to FS Studio to develop a prototype that utilizes Nvidia’s Omniverse and Isaac simulation environment, complemented by custom UI elements for them to eliminate the need for actual equipment and traditional in-class learning, that could be used for training. 

Our first step was to create a high-quality 3D space drawing on assets provided by Universal Robot, including URDFs of UR arms and CAD files of various components essential for building a work cell, CNC machine, grippers, and other miscellaneous assets. This approach not only enhances the learning experience but also eliminates the constraints associated with physical equipment. 

For our simulation environment, FS Studio used Nvidia’s Omniverse and Isaac simulation environment with some custom UI elements. The goal was to make the environment a high-quality 3D environment based on the assets provided by Universal Robot which included URDFs of the UR arms and the CAD files of the various components required to build the work cell, CNC machine, various grippers, and miscellaneous assets. 

The figure below displays a proof of concept we developed for Universal Robots to validate operations in manufacturing cells with the UR5e and UR10e robots using Omniverse tools. 

Edge AI

In a paradigm-shifting move, NVIDIA has ushered in a new era for edge AI and robotics by bringing the transformative power of generative AI to the forefront. With major expansions to the NVIDIA Jetson platform, the company aims to address challenges in areas like edge, robotics, and logistics systems, introducing advancements that promise to revolutionize defect detection, real-time asset tracking, autonomous planning, navigation, human-robot interactions, and more. 

Jetson Generative AI Lab: Empowering Developers 

To empower developers in harnessing the capabilities of generative AI, NVIDIA has introduced the Jetson Generative AI Lab. This initiative provides developers with access to optimized tools and tutorials for deploying open-source Large Language Models (LLMs), diffusion models for interactive image generation, Vision Language Models (VLMs), and Vision Transformers (ViTs) that combine vision AI with natural language processing. 

Developers can leverage the NVIDIA TAO Toolkit to create efficient and accurate AI models for edge applications. This low-code interface facilitates fine-tuning and optimization of vision AI models, including ViTs and foundational models. The toolkit also introduces VisualChangeNet, a new transformer-based model for defect inspection. 

Metropolis and Isaac Framework Expansions: Unleashing Power at the Edge 

The Metropolis framework, designed to enhance critical operational efficiency and safety issues through vision AI, is set for significant expansions. By the end of the year, an extended set of Metropolis APIs and microservices will be available on the Jetson platform, providing developers with more tools for creating complex vision-based applications. 

The Isaac platform, pivotal in domains like warehouse automation, smart manufacturing, and agriculture, sees substantial upgrades. The Isaac ROS (Robot Operating System) framework has reached general availability, offering perception and simulation capabilities. Isaac ROS 2.0, the latest release, allows developers to create and bring high-performance robotics solutions to the market with Jetson. 

AI Reference Workflows and JetPack 6: Streamlining Development 

To streamline AI application development, NVIDIA has introduced AI reference workflows based on Metropolis and Isaac frameworks. These workflows enable developers to adopt the entire AI workflow or integrate individual components selectively, reducing both development time and cost. Three distinct AI workflows include Network Video Recording, Automatic Optical Inspection, and Autonomous Mobile Robot. 

JetPack 6, expected by year’s end, empowers AI developers by providing access to cutting-edge computing features without requiring a full Jetson Linux upgrade. This facilitates faster development timelines and liberates developers from Jetson Linux dependencies. 

Eureka: A Self-Improving AI Agent for Teaching Robots Complex Skills 

In a groundbreaking move, NVIDIA Research introduces Eureka, an AI agent powered by GPT-4 that autonomously teaches robots complex skills. Eureka-generated reward programs outperform expert human-written ones on more than 80% of tasks, leading to an average performance improvement of over 50% for robots. 

Eureka leverages GPU-accelerated simulation in Isaac Gym for efficient training. The AI agent is self-improving, utilizing human feedback to modify rewards for better alignment with developers’ visions. This breakthrough in reinforcement learning showcases the potential of combining large language models and GPU-accelerated simulation technologies. 

Nvidia Edge

Looking Ahead: A Bright Future for Edge AI and Robotics 

As the AI landscape evolves, NVIDIA’s significant updates to the Jetson platform signal a foundational shift in how developers approach edge AI and robotics. Generative AI’s capabilities are set to redefine industries, offering faster development cycles, increased accuracy, and the ability to recognize and interact with unseen elements. 

With NVIDIA’s commitment to empowering developers through tools, frameworks, and innovative AI agents like Eureka, the Jetson platform is poised to shape the future of AI and robotics. The potential applications span industries, from manufacturing to logistics, as generative AI unlocks new possibilities at the edge. 

Looking at the two examples of past projects above, it is obvious that FS Studio pioneers a transformative solution, leveraging the advanced capabilities of Nvidia’s Omniverse Isaac Sim, Omniverse Replicator, Ai and other cutting edge simulation tools provided by Nvidia that deliver the solutions needed by our customers.  

FS Studio has an incredible team fully capable of using Nvidia Omniverse platform’s versatility, powerful rendering engine, and randomized scenario generation capabilities to deliver high-fidelity environments where materials, positions, lights, and cameras dynamically adapt.  

This innovative approach fosters the creation of complex and ever-evolving training grounds, propelling AI model development to unprecedented heights and with the growth of AI-supported technologies, the collection of data for training raised a new level of complexity. Bigger, variadic and photo-realistic datasets are more and more requested to enable the models to interact with the real world right out of the box. 

If edge AI and robotics is a solution you need. Please reach out to Bobby Carlton via his Calendly to set up a free consultation call. Or send him an email to bobby.carlton AT FSStudio.com.