tweak logo

Lightweight AI - NVIDIA releases small language model

Lightweight AI - Nvidia
Translate from : Lightweight AI - NVIDIA frigiver lille sprogmodel
The Mistral-NeMo-Minitron 8B is a miniaturized, high-precision AI model, tailored for GPU-accelerated data centers and high-end workstations with NVIDIA RTX hardware. It combines speed and accuracy to perfection.
By Maria
26 aug. 2024 10:44 am
f

The Mistral-NeMo-Minitron 8B is a "miniaturized version" of the new and highly accurate Mistral NeMo 12B AI model. It is tailored for GPU-accelerated data centers, the cloud and high-end workstations with NVIDIA RTX hardware.

When scaling AI models, precision is often sacrificed to ensure performance. But Mistral AI and NVIDIA's new "Mistral-NeMo-Minitron 8B" deliver the best of both worlds. It is small enough to run in real time on a workstation or desktop computer with a high-end GeForce RTX 40 Series graphics card.

NVIDIA highlights that the 8B or 8 billion variant excels in benchmark tests for AI chatbots, virtual assistants, content production and educational tools. Mistral-NeMo-Minitron 8B is available and packaged as an NVIDIA NIM microservice (downloadable via Hugging Face). It currently outperforms Llama 3.1 8B and Gemma 7B in accuracy in at least nine popular AI language model benchmark tests.

"We combined two different AI optimization methods—pruning to reduce Mistral NeMo's 12 billion parameters to 8 billion, and distillation to improve precision," said Bryan Catanzaro, vice president of applied deep learning at NVIDIA. "Thus, Mistral-NeMo-Minitron 8B delivers comparable precision to the original model, but at a lower computational cost."

"Pruning" and "distillation" for AI training involves shrinking the neural network by removing components that "contribute the least to precision" and then re-training the pruned model via distillation.

NVIDIA has confirmed that they also have an even "smaller" version called Nemotoron-Mini-4B-Instruct that is optimized for low memory and faster response times on NVIDIA GeForce RTX AI PCs and laptops. For more information on the Mistral-NeMo-Minitron 8B, you can visit NVIDIA's technical blog.

Latest gadgets

Most read gadgets

Latest gadgets

Our Partners