Architecture

This architecture outlines a solution for securely deploying a local AI chatbot using InstructLab and Skupper, while ensuring data privacy and seamless communication between isolated environments. The model is hosted in a private environment, and the service is exposed through a public OpenShift deployment, allowing external access while safeguarding the underlying data and models.

The core elements include: - InstructLab for hosting, training, and serving the AI model in a secure environment. - Red Hat Service Interconnect (RHSI) combined with Skupper to create a Virtual Application Network (VAN) between private and public sites, enabling secure communication. - OpenShift for exposing the chatbot to external users while maintaining scalability and secure connectivity to the model.

This solution enables secure communication between services hosted in different environments using Skupper and Red Hat Service Interconnect, ensuring data privacy and secure model serving.

Architecture Diagram

Architecture Overview

Common Challenges

  1. Secure Model Hosting: Protecting sensitive AI models while allowing controlled external access.

  2. Hybrid Connectivity: Facilitating communication between private and public environments without compromising security.

  3. Data Privacy: Ensuring that private data remains in a protected environment while providing real-time AI responses to external users.

Technology Stack

  • Red Hat supported products

    • Red Hat OpenShift Red Hat OpenShift is a Kubernetes-based container platform that allows developers to build, deploy, and manage containerized applications. It provides a robust platform for scaling and automating applications across hybrid cloud environments, ensuring reliability and security.

    • Red Hat Service Interconnect (RHSI) Red Hat Service Interconnect (RHSI) enables secure communication between services across different environments. Using a Virtual Application Network (VAN), it ensures that services in private, public, and hybrid cloud environments can seamlessly interact without exposing sensitive data directly.

    • InstructLab InstructLab is an AI model platform that simplifies the process of training, serving, and managing large language models. It is designed to be flexible and secure, making it ideal for environments where models need to remain private but still serve external requests through controlled access paths.

    • Podman Podman is a container engine that allows users to manage containers without the need for a daemon. It provides a secure and lightweight environment for running containers, making it ideal for deploying AI models and services in isolated environments.

    • Skupper Skupper is a secure, distributed messaging platform that enables communication between services across different environments. It creates a secure network overlay that allows services to interact without exposing sensitive data directly, ensuring data privacy and secure communication. We will use the cli version of Skupper to create a secure connection between the private and public environments.

An in-depth look at the solution’s architecture

This architecture is based on a hybrid setup where the AI model is trained and served via InstructLab in a private environment (Private Local Environment) and can only be accessed by external users through a public OpenShift site (Openshift Cluster), using Skupper for secure communication. The Red Hat Service Interconnect (RHSI) ensures that data exchanged between these two environments remains secure, creating a Virtual Application Network (VAN) between the sites.

Private Local Environment hosts the AI model using InstructLab and is responsible for: - Receiving user input exclusively from Openshift Cluster (OpenShift). - Sending the input to the LLaMA3 model for processing. - Returning the model’s response to Openshift Cluster securely.

Openshift Cluster (OpenShift) serves as the only access point for external clients and is responsible for: - Exposing the AI chatbot to external users. - Sending requests to the private InstructLab model in Private Local Environment. - Displaying the AI response from the model hosted in Private Local Environment.

By design, the model running in the private environment (Private Local Environment) is isolated and cannot be accessed directly by external clients. All interactions with the model are mediated by OpenShift (Openshift Cluster), ensuring a secure and controlled access path.

About the Technology Stack

This solution uses Red Hat Service Interconnect (RHSI) and InstructLab to secure AI model deployment. OpenShift ensures flexible scaling of the chatbot service, while Skupper enables seamless and secure communication between the isolated sites, creating a robust hybrid cloud environment for AI-driven applications.