Exposing Local AI Models While Keeping Data Protected

A brief introduction of this solution pattern.

In today’s data-sensitive landscape, companies often need to run AI models within secure, private environments while still providing external access to their services. This challenge can be met by combining Red Hat Service Interconnect (RHSI) with OpenShift and InstructLab to enable secure, seamless communication between isolated AI models and public-facing applications.

In this pattern, a local AI chatbot is trained and served via InstructLab in a protected environment where the model remains shielded from external access. Using Red Hat Service Interconnect, the model is exposed as a service within a public OpenShift environment, ensuring that sensitive data stays private while maintaining the flexibility to serve external requests. This approach simplifies managing secure hybrid environments and demonstrates how companies can safely deploy and operate AI services without compromising on security or accessibility.

Contributors: Rafael Zago


This architecture leverages Red Hat Service Interconnect to ensure secure communication between isolated environments and public-facing services, while InstructLab is used to serve AI models from a protected environment. This combination offers a robust solution for organizations needing to balance data security with the flexibility of public access.

1. Use cases

Common use cases that can be address with this architecture are:

  • Secure AI Model Hosting: Safely hosting and serving AI models within a private environment while exposing services for public interaction.

  • Hybrid Cloud Deployments: Integrating private and public cloud environments for seamless communication without compromising data privacy.

  • AI-Powered Business Applications: Deploying AI chatbots or other ML-driven services that need to interact with customers while keeping the underlying data and models secure.

  • Federated AI Training: Collaborating on AI model training across multiple secure environments, allowing data to stay localized while models are exposed via a unified interface.

2. The story behind this solution pattern

The increasing demand for AI-driven applications presents a unique challenge: how to securely deploy and operate AI models within environments that require strict data protection, while still exposing these models to public-facing services. This need became evident during the development of a local AI chatbot that would handle sensitive, proprietary data, requiring a solution where the model could be kept inside a protected environment.

Leveraging Red Hat Service Interconnect (RHSI), the team developed a pattern where AI models, served via InstructLab, are isolated within private infrastructure but exposed through secure connections into public OpenShift environments. The goal was to maintain strict control over the data and model security while providing the flexibility to scale and serve user requests externally.

This solution pattern was born from the necessity to balance security, performance, and accessibility, particularly for organizations looking to adopt hybrid cloud strategies. Using Red Hat Service Interconnect* to integrate the AI solution, the team enabled seamless communication between private and public environments without sacrificing data protection or operational efficiency.

This architecture ensures that companies can continue to innovate in AI and machine learning while adhering to compliance and security standards, making it ideal for industries such as healthcare, finance, and any domain where data security is paramount.

3. The Solution

Summary of the Solution

This solution pattern demonstrates how to securely deploy and serve a local AI chatbot using Red Hat Service Interconnect and InstructLab. The architecture enables the training and serving of AI models in a protected environment, ensuring sensitive data remains secure while still exposing the chatbot service to public users through an OpenShift environment.

Key components of the solution include:

  • InstructLab for managing and serving AI models within a secure, private infrastructure.

  • Red Hat Service Interconnect (RHSI) and Skupper to establish secure, seamless communication between isolated sites and public environments.

  • A Virtual Application Network (VAN) that securely connects two sites: a private site hosting the AI model and a public OpenShift site that exposes the chatbot service to external users.