New Relic uses AI observability with NVIDIA NIM integration

New Relic has announced its integration with NVIDIA NIM inference microservices, marking the introduction of AI observability into the New Relic platform.

This development aims to simplify the complexities and reduce the costs associated with deploying and monitoring generative AI (GenAI) applications.

According to New Relic, this integration will offer customers a comprehensive view across the AI stack for applications developed with NVIDIA NIM. This added functionality is expected to enhance AI application performance and security while providing a more straightforward setup process.

CEO of New Relic, Ashan Willy, highlighted the importance of observability in today’s competitive landscape.

"In today’s hyper-competitive market, organisations cannot afford to wait years for AI ROI," he said.

"Observability solves this by providing visibility across the AI stack. We are pioneering AI observability by extending our platform to include AI apps built with NVIDIA NIM. Combining NVIDIA’s AI technology with our expertise in observability and APM gives enterprises a competitive edge in the AI race."

Peter Marelas, Chief Architect, APJ at New Relic, also noted the significant growth in generative AI within the Asia Pacific region. He stated, "As Asia Pacific organisations brace for huge growth in generative AI users within the next five years, it is crucial for organisations to ensure they have visibility into their AI apps. Our support for NVIDIA NIM allows organisations to understand how their NVIDIA NIM deployments are performing in real-time."

Amanda Saunders, Director of AI Software at NVIDIA, added, "As enterprises race to adopt generative AI, NVIDIA NIM can help businesses quickly deploy applications in production. New Relic’s integration with NVIDIA NIM enables IT and development teams to optimise their AI applications by rapidly observing and responding to operational insights."

This integration is particularly aimed at aiding organisations in adopting AI more quickly to achieve faster returns on investment. The integration promises to provide users with full-stack visibility, deep trace insights, and critical GPU utilisation metrics, among other features. It also enhances data security by allowing the exclusion of sensitive data from monitoring processes.

One of the key features includes a holistic view of the entire AI stack, covering applications, NVIDIA GPU-based infrastructure, and the AI layer. Users can also fix performance and quality issues, such as bias, toxicity, and hallucinations, by tracing the lifecycle of AI responses. Additionally, the platform offers model inventory tracking and performance comparison capabilities to optimise model selection based on infrastructure and user requirements.

New Relic’s platform supports a wide range of AI models. This includes Databricks DBRX, Google’s Gemma, Meta’s Llama 3, Microsoft’s Phi-3, Mistral Large and Mixtral 8x22B, and Snowflake’s Arctic.

These models can now be monitored in real-time for performance and cost metrics, enabling organisations to deploy AI applications more confidently and accelerate time-to-market.

This integration is the latest in a series of moves by New Relic to expand its capabilities in the AI space. Recently, New Relic joined NVIDIA’s AIOps partner ecosystem, leveraging NVIDIA AI’s accelerated computing to combine observability and AI. New Relic AI monitoring is part of its all-in-one observability platform and is offered via a usage-based pricing model.

With this integration, New Relic continues its mission to provide engineers with a comprehensive platform for observability, offering insights that improve the planning, deployment, and operation of AI applications. The addition of NVIDIA NIM support further solidifies New Relic’s position in the growing AI market.