- Home
- Augmented Reality Software
- NVIDIA AR SDK
NVIDIA AR SDK
What it is
NVIDIA AR SDK is a free developer toolkit for adding AI face and body modeling to video applications. It uses NVIDIA GPUs with Tensor Cores for high-throughput, low-latency processing. Face detection and tracking produce bounding boxes, while facial landmark tracking predicts 68 or 126 points and head poses. Three-dimensional body pose estimation predicts 77 keypoints in 2D and 3D camera coordinates. Eye Contact estimates gaze angles and can redirect eyes toward the camera within a permitted range. Additional features include facial-expression estimation, audio-driven LipSync animation, and Active Speaker Detection from video with synchronized audio. The Core package contains runtime libraries, headers and APIs, documentation, integration guides, installation scripts, and a README. NVIDIA distributes Core and optional features through NGC; an account login is required to download Core. Sample applications are available on GitHub, and the guide covers Linux containers. Windows on Arm does not support LipSync or Active Speaker Detection.
For and against
For
- Tracks faces, landmarks, bodies, and gaze
- Predicts 77 body keypoints
- Includes LipSync and speaker detection
- Provides runtime libraries and integration guides
- Free through NVIDIA GPU Cloud
Against
- NGC account required for Core download
- Windows on Arm has feature restrictions
- Requires NVIDIA GPUs with Tensor Cores
The verdict
NVIDIA AR SDK offers a focused set of real-time face, body, gaze, and audio features for application developers. Its GPU requirement, NGC access process, and Windows on Arm limitations should be checked before adoption.
Who it suits
This SDK suits developers building video-processing applications with AI-powered augmented-reality features on Linux or Windows x86-64 systems.
On the record
- Access
- The catalog says users need to be logged in to an NGC account to download the SDK Core.catalog.ngc.nvidia.com · read 7 Oct 2026
- Additional features
- The SDK also includes facial expression estimation, LipSync animation from audio, and Active Speaker Detection using video and synchronized audio.docs.nvidia.com · read 7 Oct 2026
- Body pose
- 3D Body Pose Estimation predicts 77 body keypoints in 2D and 3D camera coordinates from video frames.docs.nvidia.com · read 7 Oct 2026
- Core package
- The Core package contains runtime libraries, development headers and APIs, documentation, integration guides, feature installation scripts, and a README.catalog.ngc.nvidia.com · read 7 Oct 2026
- Deployment
- The user guide includes instructions for running the SDK in a Linux container.docs.nvidia.com · read 7 Oct 2026
- Distribution
- The SDK Core and optional features are distributed through NVIDIA GPU Cloud (NGC), and the Core package provides a script to fetch features from NGC.catalog.ngc.nvidia.com · read 7 Oct 2026
- Eye Contact
- Eye Contact estimates gaze angles and can redirect a person's eyes toward the camera within a permissible range.docs.nvidia.com · read 7 Oct 2026
- Face features
- Face detection and tracking uses bounding boxes, while facial landmark tracking predicts 68 or 126 landmark points and head poses.docs.nvidia.com · read 7 Oct 2026
- Features
- Features include face detection and tracking, facial landmark tracking, body pose estimation, eye contact, facial expression estimation, LipSync, and active speaker detection.docs.nvidia.com · read 7 Oct 2026
- GPU processing
- The SDK uses NVIDIA GPUs with Tensor Cores for high-throughput, low-latency processing.catalog.ngc.nvidia.com · read 7 Oct 2026
- Integration
- NVIDIA provides sample applications demonstrating the SDK features in a GitHub repository.catalog.ngc.nvidia.com · read 7 Oct 2026
- Intended users
- The catalog describes the SDK as a resource for developers building video processing applications with AI-powered AR features.catalog.ngc.nvidia.com · read 7 Oct 2026
- License
- Use is governed by NVIDIA's Software License Agreement and Product-Specific Terms for NVIDIA AI Products; associated models follow the license specified for each model.catalog.ngc.nvidia.com · read 7 Oct 2026
- NIM containers
- The catalog lists Eye Contact and LipSync as available AR SDK features in ready-to-deploy NIM containers.catalog.ngc.nvidia.com · read 7 Oct 2026
- Purpose
- The SDK provides AI features for real-time modeling and tracking of human faces and bodies from video.catalog.ngc.nvidia.com · read 7 Oct 2026
- Samples
- NVIDIA links sample applications for all AR SDK features from its GitHub repository.catalog.ngc.nvidia.com · read 7 Oct 2026
- Support
- The catalog directs NVIDIA AI Enterprise customers to the NVIDIA AI Enterprise Support Portal and documentation hub.catalog.ngc.nvidia.com · read 7 Oct 2026
- Windows on Arm limit
- On Windows on Arm, LipSync and Active Speaker Detection are unsupported, and only the Body Pose Estimation or Body Detection path is available.docs.nvidia.com · read 7 Oct 2026
