Persistent GenAI Hub

Create GenAI experiences quickly, efficiently, and responsibly

Enterprises want to take advantage of Generative AI’s immense potential — but they must do so quickly and don’t want to start from scratch. Companies want to layer in and integrate new AI-powered capabilities across previous investments, assets, and functions such as data, infrastructure, applications, analytics, and customer support for fully GenAI-enabled services.

To accelerate GenAI-powered enterprise transformation, we’ve developed the Persistent GenAI Hub so companies can create faster, more efficient, and secure GenAI experiences at scale, anchored by the principles of Responsible AI. Enterprises can adopt GenAI across different large language models (LLMs) and clouds, with no provider lock-in, integrated with existing assets and enabled by pre-built accelerators. With the Persistent GenAI Hub, enterprises have a complete platform and roadmap for accelerated GenAI adoption and faster go-to-market for new AI-powered services.

Components of Persistent GenAI Hub

Playground

Playground is a no-code tool for domain experts to conduct GenAI experiments with large language models (LLMs) on enterprise data. It provides a single uniform interface to LLMs from private providers like Azure OpenAI, AWS Bedrock, and Google Gemini, and open models from Hugging Face like LLaMA2 and Mistral. Playground includes connectors to popular data sources like Vector, SQL, and Graph databases, and a Prompt Library to store and reuse domain-specific prompts.

Agents Framework

Agents Framework is built on best-in-class open LLM application libraries like LangChain and LlamaIndex and provides GenAI developers an agile way to build applications following patterns like Retrieval Augmented Generation (RAG). For advanced use cases, agents can assist with building applications that automate reasoning and dynamically solve problems using a pre-defined library of tools.

Evaluation Framework

Any application deployed on GenAI Hub has in-built access to an Evaluation Framework that brings together best-in-class patterns and metrics to validate GenAI applications like RAG. This framework uses an “AI to validate AI” approach and can auto-generate ground-truth questions to be verified by a human-in-the-loop. Metrics can be run on each upgrade to attain true observability of LLM application performance, as well a measure of any drift and bias that can be addressed.

Gateway

Gateway provides a cross-LLM router so an application can be ported to multiple LLMs seamlessly. It also handles load balancing and service priority for different LLMs. By monitoring token consumption and with a dynamic rate card, the Gateway provides a true picture of costs at the user and application level. Abstraction via the Gateway also enables advanced use cases like prompt anonymization and domain-specific guardrails.

Custom Model Pipelines

As enterprises increase their GenAI usage, they will soon need highly customized LLMs and small language models (SLMs). The GenAI Hub provides pipelines for building Custom Models that can be plugged into the Playground for evaluation. Leveraging best-in-class methods like Parameter-Efficient Fine-Tuning (PEFT) and Low-Rank Adaptation (LoRA), we provide a low-code approach to prepare data for fine-tuning models, running these pipelines in the cloud or on-premise GPU clusters.