Generative AI-Powered Visual AI Agents

Discover a collection of reference workflows that use vision language models to deliver rich, interactive visual perception capabilities for a wide range of industries.

Workloads

Computer Vision / Video Analytics

Industries

Retail/ Consumer Packaged Goods
Manufacturing
Smart Cities/Spaces
Healthcare and Life Sciences

Business Goal

Return on Investment
Innovation

Products

NVIDIA Metropolis
NVIDIA AI Enterprise

Overview
Technical Implementation
FAQ

Power A New Wave Of Applications

Traditional video analytics applications and their development workflows are typically built on fixed-function, limited models that are designed to detect and identify only a select set of predefined objects. With generative AI and foundation models, you can now build applications with fewer models that have incredibly complex and broad perception and rich contextual understanding. This newer generation of Vision Language Models (VLMs) is giving rise to smart, powerful visual AI agents.

What Is a Visual AI Agent?

A visual AI agent can combine both vision and language modalities to understand natural language prompts and perform visual question-answering. For example, answering a broad range of questions in natural language that can be applied against a recorded or live video stream. This deeper understanding of video content enables more accurate and meaningful interpretations, improving the functionality of video analytics applications and analysis of real-world scenarios. These agents promise to unlock entirely new industrial application possibilities.

Streamline Every Industrial Operation

Highly perceptive, accurate, and interactive visual AI agents will be deployed throughout our factories, warehouses, retail stores, airports, traffic intersections, and more. This will have a tremendous impact on operations teams looking to make better decisions using richer insights generated from natural interactions. Managers and operations teams will communicate with these agents in natural language, all powered by generative AI and VLMs with NVIDIA NIM™ microservices at their core.

Explore the technical implementation.

Quick Links

Read the Blog About the NVIDIA AI Blueprint Announcement

Read Blog : AI Gets Physical - New NVIDIA NIM Microservices Bring Generative AI to Digital Environments

GTC Talk: Harnessing Generative AI and Large Language Model With Vision AI Agents

Develop With NVIDIA NIM

NVIDIA NIM is a set of inference microservices that includes industry-standard APIs, domain-specific code, optimized inference engines, and enterprise runtime. It delivers multiple VLMs for building your visual AI agent that can process live or archived images or videos to extract actionable insight using natural language. We’ve created a reference workflow of a visual AI agent that you can try out to accelerate your development process.

Quick Links

Try the Visual AI Agent Reference Workflow

NVIDIA AI Blueprint for Video Search and Summarization

This NVIDIA AI Blueprint helps enterprises build visual agents for video search and summarization — all powered by generative AI, vision language models (VLMs), large language models (LLMs) and NVIDIA NIM. The visual AI agents are given tasks through natural language prompts and can perform complex assignments like video summarization or visual question and answering, unlocking entirely new application possibilities.

Apply for early access to the NVIDIA AI Blueprint for video search and summarization.

Quick Links

Get Early Access to the NVIDIA AI Blueprint

Read the Blog About the NVIDIA AI Blueprint Announcement

Read the Blog: Build a Video Search and Summarization Agent

Build Edge Agents With Jetson Platform Services

You can build visual AI agents powered by the NVIDIA Jetson™ edge AI platform using the new feature of NVIDIA JetPack™—Jetson Platform Services. The generative AI application is completely running on an NVIDIA Jetson Orin™ device that’s capable of detecting events to generate alerts and facilitate interactive Q&A sessions.

Explore frequently asked questions.

Quick Links

Download the Reference Workflow for Jetson™

Tech Blog : Develop Generative AI-Powered Visual AI Agents for the Edge

NIM is a set of easy-to-use microservices designed for secure, reliable deployment of high-performance AI model inferencing across the cloud, data center, and workstations. It supports a wide range of AI models, including open-source community and NVIDIA AI Foundation models, to ensure seamless, scalable AI inferencing—on-premises or in the cloud—using industry-standard APIs. All NIM microservices and associated preview APIs can be found at build.nvidia.com.

To get started with NIM microservices and NVIDIA AI Blueprints, visit build.nvidia.com to create an account and explore the available NIM microservices. You can check out the VLM NIMs available here.

All users can get started for free with the preview APIs on build.nvidia.com. Each new account can receive up to 5,000 credits to try out the APIs. To continue development after credits run out, you can deploy the downloadable NIM microservices locally to your hardware or to a cloud instance. Developers can also access NIM via the NVIDIA Developer Program. See details in this FAQ.

Downloadable NIM microservices require an NVIDIA AI Enterprise License. To learn more and try them for free, visit this page.

The NIM developer forum is the best place to ask questions and engage with our developer community. You can access the forums here.

Build Visual AI Agents

Explore the reference workflow, powered by multiple visual language models, to easily build your visual AI agent.

Get Early Access for NVIDIA AI Blueprint

Try NIM Reference Workflow

Download NVIDIA VIA Microservices Try NIM Reference Workflow

Developers in Action

Search and Summarize Vast Volumes of Visual Data

See how global partners are using NVIDIA NIM microservices and NVIDIA AI Blueprints today to advance infrastructure automation and build smarter spaces.

Read the Blog