At Gamescom 2024, NVIDIA announced our first on-device small language model (SLM) for improving the conversation abilities of game characters. We also announced that the first game to showcase NVIDIA ACE and digital human technologies is Amazing Seasun Games’ Mecha BREAK, bringing its characters to life and providing a more dynamic and immersive gameplay experience on NVIDIA GeForce RTX AI PCs.
NVIDIA ACE introduces an SLM purpose-built for roleplaying
NVIDIA ACE is a suite of digital human technologies that provide speech, intelligence, and animation powered by generative AI. It now includes the first NVIDIA SLM, Nemotron-4 4B Instruct. This new model is available as an NVIDIA NIM for cloud and on-device deployment by game developers. NVIDIA NIM is a set of easy-to-use microservices that speed up the deployment of foundation models on any cloud or data center.
Nemotron-4 4B Instruct is optimized through distillation, pruning, and INT4 quantization for speed and on-device inference. VRAM usage has been minimized to approximately 2 GB, providing a significantly faster time-to-first-token compared to large language models (LLMs).
- Distillation is the process of training a smaller model (called a student) to mimic the outputs of a larger model (a teacher).
- Pruning is the process of removing non-critical outputs of the student model.
- Quantization is the process of reducing the precision of a model’s weights and activations.
Nemotron-4 4B Instruct was distilled from the Nemotron-4 15B LLM and purpose-built with instruction tuning, a technique for fine-tuning models on instructional prompts to better perform specific tasks. This model provides better role-play, retrieval augmented generation (RAG), and function-calling capabilities allowing game characters to more intuitively comprehend player instructions, respond to gamers, and perform more accurate and relevant actions.
You can use the Nemotron-4 4B Instruct NIM in your digital human technology pipeline through the cloud or take advantage of over 100M NVIDIA GeForce RTX-powered PCs and laptops for on-device inference.
To make it easier for you to use Nemotron-4 4B Instruct NIM, the Unreal Engine 5 ACE sample application has been updated with on-device plugin support.
First game premiere with NVIDIA ACE NIM
Mecha BREAK, developed by Amazing Seasun Games, a Kingsoft Corporation game subsidiary, features the NVIDIA Nemotron-4 4B Instruct NIM running on the device in the first showcase of ACE-powered interactions in a game.
The NVIDIA Audio2Face-3D NIM and Whisper, OpenAI’s automatic speech recognition model, provide facial animation and speech recognition running on-device. Elevenlabs provides the character’s voice through the cloud.
In this showcase demo, NVIDIA ACE and digital human technologies enable players to interact with a mechanic that can help them choose from a diverse range of mechs, to complement their playstyle or team needs, assist in appearance customization, and give advice on how to best prepare their colossal war machine for battle.
Game developers explore the latest digital human technologies
NVIDIA ACE and digital human technologies continue to expand their footprint within the gaming industry.
Global game publisher and developer Perfect World Games is advancing its NVIDIA ACE and digital human technology demo, Legends, with new AI-powered vision capabilities. Within the demo, the character Yun Ni can see gamers and identify people and objects in the real world using the computer’s camera powered by ChatGPT-4o, adding an augmented reality layer to the gameplay experience.
These capabilities unlock a new level of immersion and accessibility for PC games. For more information, see NVIDIA ACE and download the NIM to begin building game characters powered by generative AI.
