Cloud Tips

Traefik Ingress for Eks using Terraform

December 19, 2023

amazon_eks_logo

In this article, we delve into the dynamic world of Traefik, a cutting-edge ingress controller, and unveil how to harness its power seamlessly within an EKS environment using Terraform. Learn how this dynamic duo can effortlessly handle ingress traffic, streamline routing

Link to Github code repo here

Prerequisites:

  1. AWS Account: You must have an active AWS account. If you don't have one, you can sign up for an AWS account on the AWS website. You can create it here
  2. IAM User or Role: Create an IAM (Identity and Access Management) user or role in your AWS account with the necessary permissions to create and manage EKS clusters. At a minimum, the user or role should have permissions to create EKS clusters, EC2 instances, VPCs, and related resources.
  3. AWS CLI: Install and configure the AWS Command Line Interface (CLI) on your local machine. You'll use the AWS CLI to interact with your AWS account and configure your AWS credentials. You can download it here
  4. Terraform Installed: Install Terraform on your local machine. You can download Terraform from the official Terraform website and follow the installation instructions for your operating system here

Terraform

We would use a sample Terraform EKS code, created in a previous article, you can find more detailed description here

Since we would use helm to install Traefik application to created cluster in terraform let's add the following to provider.tf:

terraform {
  required_providers {
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = ">= 2.16.1"
    }
    helm = {
      source  = "hashicorp/helm"
      version = ">= 2.8.0"
    }
  }
}
  • This block declares the required Terraform providers and their versions. It specifies that the kubernetes provider should be at least version 2.16.1, and the helm provider should be at least version 2.8.0.
locals {
  eks_endpoint       = module.eks.cluster_endpoint
  eks_ca_certificate = module.eks.cluster_certificate_authority_data
}
  • The locals block defines local variables for the EKS cluster's endpoint (eks_endpoint) and the cluster's CA certificate (eks_ca_certificate). These variables are sourced from the outputs of an EKS module, we would need it later
provider "kubernetes" {
  host  = local.eks_endpoint
  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    args        = ["eks", "get-token", "--cluster-name", local.cluster_name]
    command     = "aws"
  }
  cluster_ca_certificate = base64decode(
    local.eks_ca_certificate
  )
}
  • This block configures the kubernetes provider, specifying the EKS cluster's endpoint as the host. It employs the aws CLI to authenticate, obtaining the required credentials using the eks get-token command. The cluster's CA certificate is base64-decoded for secure communication.
provider "helm" {
  kubernetes {
    host                   = local.eks_endpoint
    cluster_ca_certificate = base64decode(
      local.eks_ca_certificate
    )
    exec {
      api_version = "client.authentication.k8s.io/v1beta1"
      args        = ["eks", "get-token", "--cluster-name", local.cluster_name]
      command     = "aws"
    }
  }
}
  • The helm provider block configures Helm to interact with the EKS cluster. It specifies the EKS cluster's endpoint and CA certificate, similar to the Kubernetes provider. The aws CLI is used for authentication, obtaining the required credentials with the eks get-token command

Traefik.tf

To make things clear we would create a separate file for Traefik install - traefik.tf:

resource "helm_release" "traefik" {
  count            = var.enable_traefik_helm_release ? 1 : 0
  name             = var.traefik_helm_release_name
  namespace        = var.traefik_k8s_namespace
  repository       = var.traefik_helm_repo
  chart            = var.traefik_helm_chart
  version          = var.traefik_helm_chart_version
  timeout          = var.traefik_helm_chart_timeout_seconds
  create_namespace = true
}
  • Count attribute is conditional, creating the Traefik Helm release only if the variable enable_traefik_helm_release is set to true
  • Name and Namespace provides a customizable way to name the Traefik installation and define the Kubernetes namespace
  • repository, chart and version - defines the Helm chart details, including the repository URL, the chart name, and the desired version to deploy.
  • timeout - specifies the maximum time (in seconds) Terraform waits for the Helm chart to be deployed before timing out. It allows customization of the deployment timeout duration.
  • create_namespace - Indicates that Terraform should create the Kubernetes namespace specified in var.traefik_k8s_namespace if it does not already exist. This ensures the designated namespace is available for the ArgoCD deployment.

Variables.tf

Since we using variables let's define them in variables.tf:

variable "enable_traefik_helm_release" {
  type        = bool
  default     = true
  description = "Enable/disable Traefik Helm chart deployment on EKS"
}

variable "traefik_helm_repo" {
  type        = string
  default     = "https://traefik.github.io/charts"
  description = "Traefik Helm chart repository URL"
}

variable "traefik_helm_chart" {
  type        = string
  default     = "traefik"
  description = "Traefik Helm chart name"
}

variable "traefik_helm_release_name" {
  type        = string
  default     = "traefik"
  description = "Traefik Helm release name"
}

variable "traefik_helm_chart_version" {
  type        = string
  default     = "20.8.0"
  description = "Traefik Helm chart version to deploy"
}
variable "traefik_helm_chart_timeout_seconds" {
  type        = number
  default     = 300
  description = "Timeout value for Helm chart install/upgrade operations"
}

variable "traefik_k8s_namespace" {
  type        = string
  default     = "traefik"
  description = "Kubernetes namespace to use for the Traefik Helm release"
}

Deployment

In order to initialize terraform and download modules run:

`terraform init`

You can also check which resources terraform is planning to create by running:

terraform plan

To provision resources run:

terraform apply

Testing

After Terraform applied you should see similar output:

Apply complete! Resources: 61 added, 0 changed, 0 destroyed.

Outputs:

connect_to_eks = "aws eks --region <YOUR_REGION> update-kubeconfig --name <CLUSTER_NAME> --profile default"
endpoint = "<CLUSTER_ENDPOINT>"

Execute the command from connect_to_eks output in order to generate kubeconfig file:

aws eks --region <YOUR_REGION> update-kubeconfig --name <CLUSTER_NAME> --profile default

Verify conectivity to the cluster with kubectl:

kubectl get no

You should see list of nodes:

NAME                        STATUS   ROLES    AGE   VERSION
ip-10-0-1-9.ec2.internal    Ready    <none>   33m   v1.27.7-eks-e71965b
ip-10-0-3-76.ec2.internal   Ready    <none>   34m   v1.27.7-eks-e71965b

Now we can check if Traefik is up and running:

kubectl get po --namespace traefik
NAME                       READY   STATUS    RESTARTS   AGE
traefik-84cb5c7444-k9cp8   1/1     Running   0          12m

Check if your loadbalancer is ready to use, you should see similar output. Copy value of your external IP, we will use it to expose our test app:

kubectl get svc --namespace traefik
NAME      TYPE           CLUSTER-IP      EXTERNAL-IP                                                               PORT(S)                      AGE
traefik   LoadBalancer   172.20.14.232   ac304e54c6f0d4aaeb137ba0f9d15d26-1652635264.us-east-1.elb.amazonaws.com   80:31550/TCP,443:32735/TCP   13m

Now let's deploy test application to see if it working corectly

We would deploy basic nginx application. First copy it locally to your PC:

wget https://raw.githubusercontent.com/cloudtipss/Traefik-Eks/main/testapp/test.yaml

File contains Deployment, Service and Ingress. Replace host value in a Service Block with external IP from a previous step

Now you can apply it, run the following command from a folder you copied test.yaml:

kubectl apply -f ./test.yaml

deployment.apps/nginx created
service/nginx created
ingress.networking.k8s.io/nginx created

Now let's verify our app is running and listening on a host we provided:

curl -k https://ac304e54c6f0d4aaeb137ba0f9d15d26-1652635264.us-east-1.elb.amazonaws.com/

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

As you can see Nginx application is up and running!

You can find source code in our Github repo