At its “.NEXT 2024” conference this week in Barcelona, Nutanix is launching version 2 of its turnkey AI infrastructure, GPT-in-a-Box, the result of new partnerships with NVidia and Hugging Face. Internal implementation of AI models has never seemed simpler!
Announced at the .NEXT conference, Nutanix’s collaboration with NVidia and Hugging Face opens up new perspectives and choices in terms of AI implementation. Thanks to the integration of inference microservices with the NVidia NIM platform and the Hugging Face catalog on the Nutanix infrastructure ” GPT-in-a-Box“, companies will now be able to break free from public clouds to train, personalize, but above all, implement and infer their AI models with complete security and simplicity.
Simplifying AI deployments outside of the public cloud
According to the study State of Enterprise AI sponsored by Nutanix and produced by Vanson Bourne, 90% of companies have made AI a priority. But their approach is hampered by skills shortages, data security concerns and the need to modernize infrastructure to accommodate new applications.
More than half are also looking for efficient solutions to transfer information between internal, public cloud and remote IT in shops or factories, in other words at the edge. In general, public cloud players offer solutions to all of these issues, but not all companies necessarily have the desire—or ability—to rely on hyperscaler offerings to implement their AI, whether their choices are motivated by data security concerns or even cost.
Nutanix is aware that not all models, especially very large models, will generally be trained locally. It often makes more sense to do this in the cloud because it is a “one time” operation that requires a large amount of resources. This is also partly true for “fine-tuning”, even if it is essential for certain companies to do it locally, if only to preserve the confidentiality of this “fine-tuning”, a scenario where GPT-in-a-Box then makes perfect sense.
On the other hand, local inference execution and RAG are much more attractive primarily due to respect for data compliance and confidentiality or to meet edge latency constraints, but also in terms of costs. Because those AI treatments (inference and RAG) are the most intensively used and for which local execution can prove more economical than cloud execution.
AI alternative to public clouds for internal implementation
By partnering with NVidia, Nutanix is responding to exactly this problem. Enriching its GPT-in-a-Box platform with NVidia NIM microservices, Nutanix offers an alternative to public cloud AI platforms. Called GPT-in-a-Box 2.0, the new solution allows companies to train and deploy a wide range of AI models, either open source or custom, in-house and using NVidia GPUs (Nutanix plans to expand its platform to AMD and Intel GPUs in the future) .
And since Nutanix remains a major expert in deploying infrastructure closer to customers, GPT-in-a-Box 2.0 can be used not only to centralize model training in data centers (and eventually in the cloud), but above all to simplify and make deployment cost-effective conclusions locally and at the edge, for example in stores or factories, while avoiding expensive data transfers that generate delays.
Accessible through “standard programming interfaces” and exposing models in API form according to standards defined by OpenAI, GPT-in-a-Box 2.0 should also ” reduce the complexity often associated with adopting AI solutions by eliminating the need for advanced technical skills » according to Nutanix.
Supplement to the Hugging Face catalog
Technically speaking, the GPT-in-a-Box 2.0 platform is based on containers. Its architecture thus made it very easy for Nutanix to add microservices support (and model catalog) to the NVidia NIM platform.
But the latter is not the only integrated catalog of models. Thanks to a technology partnership with Hugging Face, GPT-In-Box 2.0 integrates a validated and optimized subset of the most popular models from Hugging Face.
The technology agreement allows Nutanix to build the Hugging Face model inference engine into its platform, but also to determine which are the most interesting and relevant models to build into GPT-in-a-Box.
From a practical point of view, once the GPT-in-a-Box 2.0 platform is installed and operational, the developer or administrator simply needs to select the model that suits their needs from the NVidia NIM or Hugging Face catalog. The model is then downloaded and installed with one click. It can then be tested directly from the Nutanix user interface. The platform then exposes this model’s API by providing its internal URL and generates access keys. Applications then only have to call the model using the URL and key without the need for rewriting (since the model call follows the same syntax as the OpenAI API which is now established as the de facto standard for all models). It’s hard to make it simpler.
Optimize data control and security
Nutanix believes that GPT-in-a-Box primarily meets the needs of small and medium-sized organizations that do not necessarily have the resources to afford the cloud or invest in complex AI infrastructure, which makes it all the more expensive to recruit rare and therefore expensive skills. But GPT-in-a-Box 2.0 is also perfectly suited for any company that has sovereignty restrictions or simply doesn’t want to put sensitive data in the cloud. Especially since the Nutanix platform also offers advanced data storage and security functions, and new GPT-in-a-Box 2.0 features include automating the deployment of inference at the edge with granular access control, ensuring improved security for AI models.