The New Way to Expose APIs in Kubernetes Cluster

This year’s KubeCon was held from May 16th till May 20th in lovely Valencia, located on Spain’s east coast. Unfortunately, I could not be there in person to enjoy the wide sandy beaches, striking architecture, buzzing food scene, and culture. As a consolation, I did have a look at some of the newer options in Kubernetes as they relate to APIs. One of which is the Gateway API.

Ingress is one of the most widely used resources across Kubernetes. The component is responsible for both the infrastructure and application, and helps to expose your applications and services to the outside world. However, Kubernetes networking technologies have evolved significantly and many newer use cases quickly exposed the limitations of Ingress.

The Ingress API did not provide for the advanced load-balancing capabilities users wanted to define and was fairly impractical for users to manage. Vendors tried to address these additional use cases by using annotations in order to extend the platform but this led to inconsistent implementations between providers.

Hence the creation of the Gateway API, a collection of new resources and APIs which improve and will potentially supersede Ingress in the future.

Gateway API divides Ingress functionality into 3 components each responsible for its own part as detailed below. Also, the Gateway API adds additional valuable and useful features. A lot of downstream implementers and integrators of Gateway API have public previews available and are iterating quickly. Next to this Envoy has also announced their Envoy Gateway to standardize Kubernetes Ingress.

The Old Way – Ingress

In order to expose APIs with Ingress using security best practices, you should use a service that connects into ingress which then connects into load balancer. That means Ingress includes both ingress controller details (configured by the platform admin, most likely devops) and app routing (configuring by the developer).

The New Way –  Gateway API

Architecture

GatewayClass -> Gateway -> [*]Route (see picture above instead)

The GatewayClass is provided by the platform/infrastructure provider for example Istio or Google Cloud.

The Gateway is an instance of the gatewayClass defined by the platform/cluster admin that binds the gateway into LoadBalancer.

A *Route is meant to bind the route to a Gateway (more than one route can be configured to one gateway). Available in TCP/IP layers:

• HTTPRoute – Can route HTTP and HTTPS
• TLSRoute – routes TLS and routes based on parameters in the the TLS layer and SNI (Server Name Indication)
• TCPRoute – Routes based on TCP destination port
• UDPRoute – Routes based on UDP destination port

Advantages

Advantages using the new way:

Gateway API adds new features:

• HTTP header-based matching
• HTTP header manipulation
• Weighted traffic splitting
• Traffic mirroring
• Role-oriented resource model

Gateway API Also enables extensions:

• Routing to other protocols
• Arbitrary backend CRDs such as buckets, functions, etc.
• Custom parameters and configurations (LB algorithms,  custom matching)
• The new way decouples the infrastructure from the application

Outlook Going Forward

The main benefit of the new Gateway API is that Ingress is now essentially split into two separate functions, one for the Gateway and one for the Routes to the back-end. This gives operators the ability to change gateways but keep the routing configuration in place.

This also means that for day 2 operations, the platform owners can still own the selection and deployment of the Gateways. But developers and operations have the freedom to update routes to back-end services they need to expose. This adds more RBAC options and raises the overall security posture.

The need for better security was recently demonstrated by researchers with the Shadowserver Foundation who have discovered more than 380,000 open Kubernetes API servers exposed on the Internet. That represents 84% of all global Kubernetes API instances observable online.

Finally, the Gateway API provides support for more sophisticated LB features like weighted traffic splitting and HTTP header-based matching and manipulation. The feature is still in development and many vendors have started providing experimental support so we expect steady progress of the Gateway API going forward.