Mistral AI and NVIDIA Unveil Mistral NeMo 12B

Mistral AI and NVIDIA have joined forces to introduce the Mistral NeMo 12B, a groundbreaking enterprise AI model. This state-of-the-art language model, designed for diverse applications such as chatbots, multilingual tasks, coding, and summarization, represents a significant advancement in AI technology.

Leveraging Mistral AI’s expertise in training data and NVIDIA’s optimized hardware and software ecosystem, Mistral NeMo 12B offers unprecedented accuracy, flexibility, and efficiency.

Guillaume Lample, cofounder and chief scientist of Mistral AI, highlighted the importance of this collaboration:

Training and Performance

Mistral NeMo 12B was trained on the NVIDIA DGX Cloud AI platform, benefiting from scalable access to the latest NVIDIA architecture.

The model’s training process involved 3,072 H100 80GB Tensor Core GPUs on DGX Cloud, utilizing NVIDIA AI architecture, including accelerated computing, network fabric, and software to increase training efficiency.

The use of NVIDIA TensorRT-LLM for accelerated inference performance and the NVIDIA NeMo development platform for building custom generative AI models further enhances its capabilities.

Accuracy and Efficiency

The Mistral NeMo 12B excels in multi-turn conversations, math, common sense reasoning, world knowledge, and coding.

With a 128K context length, it processes extensive and complex information more coherently and accurately, ensuring contextually relevant outputs.

The model’s 12-billion parameters and the use of the FP8 data format for model inference reduce memory size and speed deployment without compromising accuracy.

Licensing and Community Support

Released under the Apache 2.0 license, Mistral NeMo fosters innovation and supports the broader AI community.

This open model license allows enterprises to integrate Mistral NeMo into commercial applications seamlessly, encouraging widespread adoption and adaptation.

Enterprise-Grade Deployment

Mistral NeMo comes packaged as an NVIDIA NIM inference microservice, offering performance-optimized inference with NVIDIA TensorRT-LLM engines.

This containerized format allows for easy deployment anywhere, providing enhanced flexibility for various applications.

Models can be deployed in minutes, rather than several days, ensuring rapid implementation.

The NIM features enterprise-grade software, including dedicated feature branches, rigorous validation processes, and robust security and support. Comprehensive support includes direct access to an NVIDIA AI expert and defined service-level agreements, delivering reliable and consistent performance.

Designed to fit on the memory of a single NVIDIA L40S, NVIDIA GeForce RTX 4090, or NVIDIA RTX 4500 GPU, the Mistral NeMo NIM offers high efficiency, low compute cost, and enhanced security and privacy.

Optimized Training and Inference

Combining the expertise of Mistral AI and NVIDIA engineers, the Mistral NeMo model benefits from optimized training and inference processes.

Trained with Mistral AI’s expertise in multilinguality, coding, and multi-turn content, the model utilizes NVIDIA’s full stack for accelerated training.

It employs efficient model parallelism techniques, scalability, and mixed precision with Megatron-LM, part of NVIDIA NeMo.

Availability and Deployment

The flexibility of Mistral NeMo allows it to run anywhere — cloud, data center, or RTX workstation — making it ready to revolutionize AI applications across various platforms.

Enterprises can experience Mistral NeMo as an NVIDIA NIM today via ai.nvidia.com, with a downloadable NIM version expected soon.

By combining Mistral AI’s data training expertise with NVIDIA’s optimized hardware and software, Mistral NeMo 12B is set to open up new opportunities for companies, offering a powerful, reliable, and efficient AI solution for enterprise applications.

If you want more updates related to AI, subscribe to our Newsletter