Solution Patterns: Connect, Secure and Protect with Red Hat Connectivity Link

Architecture

1. Common Challenges addressed

With Hybrid Cloud becoming the defacto standard, Platform Engineering teams face increased complexity of setting up and running environments across multiple clusters and multiple regions.

There is a need to make it easy to connect, protect and manage services and infrastructure across the different environments to:

  • Provide a connectivity solution that leverages Gateway API as its foundation which enables platform engineers and application developers to collaborate to connect, secure, protect and observe their HTTP based APIs and infrastructure leveraging CRD based APIs rather than additional heavy weight platforms.

  • Provide a lightweight layer of API management focused tooling that provides an “API management lens” that compliments the core connectivity features.

2. Technology Stack

3. An in-depth look at the solution’s architecture

With growing demand for mobile based apps, Globex gears to support access to their core services from other channels/applications. In this solution we will look at a step-by-step approach to onboarding a new application and allowing to access the Globex services in a secure fashion.

architecture

  1. Globex’s development team built a Mobile Gateway as a single point of entry (a wrapper) in front of Globex’s core services. Ref: gateway-pattern

  2. The access to this mobile-gateway service endpoint needs to be secured and protected for connectivity from the Mobile App that is being built.

  3. To setup such a secure connectivity for a growing demand for access to core services, the Globex team adopts a Kubernetes native approach to application connectivity through Red Hat Connectivity Link.

  4. Connectivity Link allows for clear separation of concerns with regard to the various teams involved in setting up such a secure connectivity

  5. Platform Engineers workflow:
    The Platform Engineer has to setup a Gateway which will allow for secure connection to the backend service endpoints. To achieve this a number of components need to be setup.

    Click to view details about these components

    1. A Managed Zone needs to be setup within the DNS provider. E.g. managed.globex.com.
      Refer to the Getting started page to learn how to setup a Managed Zone.

    2. TLS issuer (a.k.a ClusterIssuer): Sets up a Certificate Issuer to create TLS certificates that are needed to secure communication. In this case, Globex uses Let’s Encrypt. This certificate is stored as a secret to be referenced by the Gateways.

    3. Gateway: create a new Gateway (using the Gateway API and Istio-based controllers) as an entrypoint for all requests to Globex’s system. The Gateway uses the TLS Certificates created by the TLS Issuer. Also a number of policies are setup to secure and protect the Gateway.

    4. TLS Policy: leverages the TLS-issuer/CertificateIssuer to set up TLS certificates for the listeners defined within the Gateway. Listeners define the hostname for the various incoming requests and is denoted with a wildcard hostname based on the root domain. These gateways can use a subdomain of the Managed Zone. E.g. *.managed.globex.com.

    5. Auth Policy: setup a zero-trust deny-all policy that result in a default 403 response for any unprotected endpoints.

    6. DNS Policy: provide DNS management by managing the lifecycle of DNS records to setup ingress connectivity using DNS to bring traffic to the Gateway.

    7. RateLimit Policy: set up a default artificially low global limit to further protect any endpoints exposed by this Gateway.

  6. Developer/Application owners workflow:
    Now that the Gateway is made available, the developers can now start onboarding their service endpoints by creating a HTTPRoute representing each endpoint. Developers can also override the Auth and RateLimit policies setup by the Platform Engineer to suit their specific authz/authn needs and non-functional requirements for rate limiting.

    Click to view details about these components

    1. HTTPRoute: this is part of the Gateway API. Setup a HTTPRoute by defining the parent Gateway that was setup by Platform Engineer. Define a hostname for the route so that requests can be sent to the correct route. Rules can be setup to send the requests to the right backend endpoint.

    2. AuthPolicy: Globex adopts OIDC for auth and the developer creates a new policy attached to the HTTPRoute. This policy overrides the deny-all policy created by the Platform Engineer.

    3. RateLimit Policy: the low-limits default RateLimit Policy created at the Gateway level is overridden by this policy to provide for a suitable limit based on non-functional requirements. Any changes can be made easily to the system by applying changes to this policy.

  7. Once the components and custom resources are in place, a Globex customer can login to the Globex Mobile application. The request will then be served securely through the Connectivity Link platform that has been setup.

  8. Suitable metrics can also be enabled which will be covered in the next version of this Solution Pattern.

4. About the Technology Stack

4.1. Gateway API

Ref: Gateway API

  • New standard for Ingress from k8s community

  • Gateway API is the core API that Kuadrant integrates with.

  • Represent L4/L7 routing and traffic management through generic common core API resources

  • API resources reflects the separation of responsibilities such as infra provider, PE or App Dev

Connectivity Link (Kuadrant) provides connectivity, security and service protection capabilities in the form of Kubernetes CRDs that implement the Gateway API concept of policy attachment. These policy APIs can target specific Gateway API resources such as Gateways and HTTPRoutes to extend their capabilities and configuration.

Click to view why Gateway API

  • Role Oriented - To allow cluster owners to designate how the infrastructure is used

  • Portable - This enables concepts & resources to be uniform across environments reducing complexity

  • Expressive - Provides built-in capabilities for header-based matching, traffic weighting & more

4.1.1. Gateway

  • describes how traffic can be translated to Services within the cluster

  • can express capabilities like HTTP header manipulation, traffic weighting & mirroring, TCP/UDP routing

  • may be attached to one or more Route references which serve to direct traffic for a subset of traffic to a specific service

4.1.2. HTTPRoute

  • enables advanced routing capabilities for Ingress.

  • specify routing behavior of HTTP requests from a Gateway listener to an API object, i.e. Service.

  • Each Route includes a way to reference the parent resources it wants to attach to.

4.2. OpenShift Service Mesh and Istio as Gateway provider

OpenShift Service Mesh is a supported Gateway API provider and Connectivity Link integrates with Istio (via WASM and Istio APIS) to provide service protection capabilities. It configures Envoy via the Istio control plane to enforce the applied policies and register components such as Authorino and Limitador.

Ref: Istio with Gateway API

Kuadrant has also been proven to work with Envoy Gateway as a Gateway API provider. This is to showcase that Kudrant is extensible and can work work with other Gateway providers too. To try this out check out this tutorial

4.3. Cert Manager

The cert-manager Operator for Red Hat OpenShift is a cluster-wide service that provides application certificate lifecycle management. It allows you to integrate with external certificate authorities and provides certificate provisioning, renewal, and retirement. In this solution pattern, we integrate with Let’s Encrypt.

If needed, one can create the certificates manually and inject them, rather than using cert-manager. You can create a secret my-cert (as an e.g.) and add that to a listener in the gateway. TLSPolicy is a very light API wrapper around cert-manager resources that makes the target, host and secret name obvious as already defined in the gateway listener.

4.4. Kuadrant

It enables platform engineers and application developers to easily connect, secure, and protect their services and infrastructure across multiple clusters. It exposes these capabilities in the form of powerful policy APIs that implement the Gateway API concept of policy attachment - with policies for TLS, DNS, application AuthN & AuthZ, and rate limiting. Additionally, Kuadrant offers observability templates to further support infrastructure management.

Simply put, Red Hat Connectivity Link

  • provides a bunch of policies (through Kuadrant APIs) - that can be applied on Gateway API resources viz. Gateway and HttpRoute…

  • so that you can connect, secure and protect all of your service endpoints

  • providing the Platform Engineers the right level of control

  • providing Developers the tools for a more fully rounded API mgmt feature set through Apicurio.

rhcl simplyput

4.5. Personas

  • Platform engineers

    • can deploy, manage, observe and protect services exposed via their gateways

  • confidently allow application developers to self-service knowing that endpoints are secured

  • Application developers

    • are allowed to self service and refine policies to their specific needs in order to protect exposed endpoints.

    • focus on what is important for them - their applications and code

4.6. Kuadrant' underlying components

Here is a view on how Kuadrant Policy APIs are applied on the Request flow on Gateway API

policies on requestflow

4.6.1. DNS Operator

DNS operator consumes DNSRecord resources that are configured via the DNSPolicy API and applies them into the targeted cloud DNS provider. AWS, Azure and Google DNS are our main targets.

4.6.2. Authorino Operator for Auth Policy

External authorization server fully manageable via Kubernetes Custom Resources. Supports JWT authentication, API key, mTLS, pattern-matching authz, OPA, K8s SA tokens, K8s RBAC, external metadata fetching, and more, with minimum to no coding at all, no rebuilding of your applications.
Ref: https://docs.kuadrant.io/0.8.0/authorino/

4.6.3. Limitador Operator for Ratelimits

  • Limitador is a generic rate-limiter and can be enabled using RateLimit APIs.

  • It support in-memory, Redis and disk data stores.

  • In this solution pattern, for a single cluster, we go with the built-in in-memory database;

  • for multi-cluster setup you can use a Redis service (or RocksDB)

4.7. Policies

Kuadrant at its heart, provides Gateway Policies (as described previously) for Kubernetes. To quote https://kuadrant.io/:

Gateways play a pivotal role in application connectivity. With Kuadrant, platform engineers and application developers can easily connect, secure and protect their services and infrastructure using its powerful policy APIs

4.7.1. Policy Attachement

Policy Attachment augments the behavior of an object to add additional settings that can’t be described within the spec of that object. A "Policy Attachment" is a specific type of resource that can affect specific settings across either one object (this is "Direct Policy Attachment"), or objects in a hierarchy (this is "Inherited Policy Attachment").
Ref: https://gateway-api.sigs.k8s.io/reference/policy-attachment/

4.7.2. Defaults and overrides

AuthPolicy and RateLimitPolicy can be attached to Gateways or to HTTPRoutes, with cascading effects through the hierarchy that result in one effective policy per gateway-route combination.

Ref: More about defaults and overrides

4.8. Red Hat build of Keycloak for OAuth

Red Hat build of Keycloak is a cloud-native Identity Access Management solution based on the popular open source Keycloak project

This Solution Pattern leverages Keycloak to setup authentication mechanism within the APIPolicy CR. An AuthPolicy can be setup either based on an API Key or Oauth, while OAuth is more preferrable.

Here is an excerpt of an AuthPolicy.

  rules:
    authentication:
      "keycloak-users":
        jwt:
          issuerUrl: https://sso.mykeycloak.example.com/realms/realmname
    response:
      success:
        filters:
          identity:
            json:
              properties:
                userid:
                  selector: auth.identity.sub

  • Note that the Keycloak’s Issuer URL is configured within the AuthPolicy, and also the userid from the JWT token is being used an identifier.

  • The mobile application used the same Keycloak realm for user authentication.

  • Once the user logs in, the JWT token from the app is then analysed for validity as pet the APIPolicy (this is done by by Authorino which is a lightweight Envoy external authorization server)