Solution Patterns: Connect, Secure and Protect with Red Hat Connectivity Link

Architecture

1. Common Challenges addressed

With Hybrid Cloud becoming the defacto standard, Platform Engineering teams face increased complexity of setting up and running environments across multiple clusters and multiple regions.

There is a need to make it easy to connect, protect and manage services and infrastructure across the different environments to:

  • Provide a connectivity solution that leverages Gateway API as its foundation which enables platform engineers and application developers to collaborate to connect, secure, protect and observe their HTTP based APIs and infrastructure leveraging CRD based APIs rather than additional heavy weight platforms.

  • Provide a lightweight layer of API management focused tooling that provides an “API management lens” that compliments the core connectivity features.

2. Technology Stack

3. An in-depth look at the solution’s architecture

With growing demand for mobile based apps, Globex gears to support access to their core services from other channels/applications. In this solution we will look at a step-by-step approach to onboarding a new application and allowing to access the Globex services in a secure fashion.

architecture

  1. Globex’s development team built a Mobile Gateway as a single point of entry (a wrapper) in front of Globex’s core services. Ref: gateway-pattern

  2. The access to this mobile-gateway service endpoint needs to be secured and protected for connectivity from the Mobile App that is being built.

  3. To setup such a secure connectivity for a growing demand for access to core services, the Globex team adopts a Kubernetes native approach to application connectivity through Red Hat Connectivity Link.

  4. Connectivity Link allows for clear separation of concerns with regard to the various teams involved in setting up such a secure connectivity

  5. Platform Engineers workflow:
    The Platform Engineer has to setup a Gateway which will allow for secure connection to the backend service endpoints. To achieve this a number of components need to be setup.

    Click to view details about these components

    1. A Managed Zone needs to be setup within the DNS provider. E.g. managed.globex.com.
      Refer to the Getting started page to learn how to setup a Managed Zone.

    2. TLS issuer (a.k.a ClusterIssuer): Sets up a Certificate Issuer to create TLS certificates that are needed to secure communication. In this case, Globex uses Let’s Encrypt. This certificate is stored as a secret to be referenced by the Gateways.

    3. Gateway: create a new Gateway (using the Gateway API and Istio-based controllers) as an entrypoint for all requests to Globex’s system. The Gateway uses the TLS Certificates created by the TLS Issuer. Also a number of policies are setup to secure and protect the Gateway.

    4. TLS Policy: leverages the TLS-issuer/CertificateIssuer to set up TLS certificates for the listeners defined within the Gateway. Listeners define the hostname for the various incoming requests and is denoted with a wildcard hostname based on the root domain. These gateways can use a subdomain of the Managed Zone. E.g. *.managed.globex.com.

    5. Auth Policy: setup a zero-trust deny-all policy that result in a default 403 response for any unprotected endpoints.

    6. DNS Policy: provide DNS management by managing the lifecycle of DNS records to setup ingress connectivity using DNS to bring traffic to the Gateway.

    7. RateLimit Policy: set up a default artificially low global limit to further protect any endpoints exposed by this Gateway.

  6. Developer/Application owners workflow:
    Now that the Gateway is made available, the developers can now start onboarding their service endpoints by creating a HTTPRoute representing each endpoint. Developers can also override the Auth and RateLimit policies setup by the Platform Engineer to suit their specific authz/authn needs and non-functional requirements for rate limiting.

    Click to view details about these components

    1. HTTPRoute: this is part of the Gateway API. Setup a HTTPRoute by defining the parent Gateway that was setup by Platform Engineer. Define a hostname for the route so that requests can be sent to the correct route. Rules can be setup to send the requests to the right backend endpoint.

    2. AuthPolicy: Globex adopts OIDC for auth and the developer creates a new policy attached to the HTTPRoute. This policy overrides the deny-all policy created by the Platform Engineer.

    3. RateLimit Policy: the low-limits default RateLimit Policy created at the Gateway level is overridden by this policy to provide for a suitable limit based on non-functional requirements. Any changes can be made easily to the system by applying changes to this policy.

  7. Once the components and custom resources are in place, a Globex customer can login to the Globex Mobile application. The request will then be served securely through the Connectivity Link platform that has been setup.

  8. Suitable metrics can also be enabled which will be covered in the next version of this Solution Pattern.

4. About the Technology Stack

4.1. What is Gateway API

Gateway APIThis is the new standard for Ingress from the Kubernetes community.

Gateway API is a relatively new Kubernetes based API focused on traffic routing and is often referred to as the next generation of Ingress on kube.

Used to define the Gateways and Routing rules for requests entering those gateways. Our supported provider is Istio via OpenShift Service Mesh.

Connectivity Link (Kuadrant) provides connectivity, security and service protection capabilities in the form of Kubernetes CRDs that implement the Gateway API concept of policy attachment. These policy APIs can target specific Gateway API resources such as Gateways and HTTPRoutes to extend their capabilities and configuration.

4.1.1. Istio as Gateway provider

Connectivity Link/Kuadrant’s focus is on HTTP traffic and Istio/OpenShift Service Mesh as a supported Gateway API implementation. Istio is the Gateway API provider that Connectivity Link integrates with (via WASM and Istio APIS) to provide service protection capabilities. It configures Envoy via the Istio control plane to enforce the applied policies and register components such as Authorino and Limitador.+ Ref: https://istio.io/latest/docs/tasks/traffic-management/ingress/gateway-api/ [Istio with Gateway API]

4.1.2. HTTPRoute

HTTPRoute enables advanced routing capabilities for Ingress. It is a Gateway API type for specifying routing behavior of HTTP requests from a Gateway listener to an API object, i.e. Service.
Ref: https://gateway-api.sigs.k8s.io/api-types/httproute/

4.2. What is Kuadrant

It enables platform engineers and application developers to easily connect, secure, and protect their services and infrastructure across multiple clusters with policies for TLS, DNS, application authentication & authorization, and rate limiting. Additionally, Kuadrant offers observability templates to further support infrastructure management.
Ref: https://docs.kuadrant.io

4.3. Kuadrant: list of underlying components

4.3.1. DNS Operator

DNS operator consumes DNSRecord resources that are configured via the DNSPolicy API and applies them into the targeted cloud DNS provider. AWS, Azure and Google DNS are our main targets.

4.3.2. Cert Manager for TLS Policy

Manages TLS certificates for our components and for the Gateways. Consumes Certificate resources created by the Kuadrant operator in response to the TLSPolicy.

4.3.3. Authorino Operator for Auth Policy

External authorization server fully manageable via Kubernetes Custom Resources. Supports JWT authentication, API key, mTLS, pattern-matching authz, OPA, K8s SA tokens, K8s RBAC, external metadata fetching, and more, with minimum to no coding at all, no rebuilding of your applications.
Ref: https://docs.kuadrant.io/0.8.0/authorino/

4.3.4. Limitador Operator for Ratelimits

Limitador is a generic rate-limiter and can be enabled using RateLimit APIs.

4.4. Policies

Kuadrant at its heart, provides Gateway Policies for Kubernetes. To quote https://kuadrant.io/:

Gateways play a pivotal role in application connectivity. With Kuadrant, platform engineers and application developers can easily connect, secure and protect their services and infrastructure using its powerful policy APIs

4.4.1. Policy Attachement

Policy Attachment augments the behavior of an object to add additional settings that can’t be described within the spec of that object. A "Policy Attachment" is a specific type of resource that can affect specific settings across either one object (this is "Direct Policy Attachment"), or objects in a hierarchy (this is "Inherited Policy Attachment").
Ref: https://gateway-api.sigs.k8s.io/reference/policy-attachment/

4.4.2. Defaults and overrides

AuthPolicy and RateLimitPolicy can be attached to Gateways or to HTTPRoutes, with cascading effects through the hierarchy that result in one effective policy per gateway-route combination.

Ref: More about defaults and overrides

4.5. Red Hat build of Keycloak for OAuth

Red Hat build of Keycloak is a cloud-native Identity Access Management solution based on the popular open source Keycloak project

This Solution Pattern leverages Keycloak to setup authentication mechanism within the APIPolicy CR. An AuthPolicy can be setup either based on an API Key or Oauth, while OAuth is more preferrable.

Here is an excerpt of an AuthPolicy.

  rules:
    authentication:
      "keycloak-users":
        jwt:
          issuerUrl: https://sso.mykeycloak.example.com/realms/realmname
    response:
      success:
        dynamicMetadata:
          identity:
            json:
              properties:
                userid:
                  selector: auth.identity.sub

  • Note that the Keycloak’s Issuer URL is configured within the AuthPolicy, and also the userid from the JWT token is being used an identifier.

  • The mobile application used the same Keycloak realm for user authentication.

  • Once the user logs in, the JWT token from the app is then analysed for validity as pet the APIPolicy (this is done by by Authorino which is a lightweight Envoy external authorization server)