How ExternalDNS works with Kubernetes AWS ALB Ingress and Helmfile

DevOps Engineers are responsible for deploying Helm charts for Kubernetes applications on different environments including AWS. However, after deploying helm charts, we need to manually add DNS records in the AWS Route 53 service to start the application. Adding such records manually is tedious and we can automate this process using ExternalDNS.

What is ExternalDNS?

ExternalDNS is Kubernetes cluster internal DNS server. ExternalDNS allows you to control DNS records dynamically using the Kubernetes services and Ingresses with DNS providers for e.g., AWS Route 53 service.

What is a HelmFile?

HelmFile allows you to declare the definition of all Kubernetes clusters in a single YAML file by bundling multiple helm releases i.e., multiple helm charts in a single YAML file. It keeps a directory of chart value files with version control. It also allows you to build a release specification depending on the type of environment (develop, test, production). More information about the helmfile can be found here.

Deploy External DNS on Kubernetes Cluster Using Helmfile (AWS EKS)

Prerequisites:

Create an IAM policy and Role which allows Kubernetes pods to assume the AWS IAM roles.

This policy allows ExternalDNS to update the Route53 Resource Record Sets and Hosted Zones. You can customize this policy as per your requirement.

{ 
  "Version": "2012-10-17", 
  "Statement": [ 
    { 
      "Effect": "Allow", 
      "Action": [ 
        "route53:ChangeResourceRecordSets" 
      ], 
      "Resource": [ 
        "arn:aws:route53:::hostedzone/*" 
      ] 
    }, 
    { 
      "Effect": "Allow", 
      "Action": [ 
        "route53:ListHostedZones", 
        "route53:ListResourceRecordSets" 
      ], 
      "Resource": [ 
        "*" 
      ] 
    } 
  ] 
}

Then, create an IAM Role which is assumed by the ExternalDNS Pod. After creating the role, please attach the IAM Policy to the IAM role created earlier.

Please refer to the appropriate documentation if you are using kiam or Kube2iam.

Deploy ExternalDNS on AWS EKS using helmfile

Helm chart for ExternalDNS is available on Github. We are modifying the chart and converting it into a helmfile.

Create the helmfile named as externaldns.yaml

In the podAnnotations section, add the IAM role created earlier.

# helmfile -f externaldns.yaml diff 
repositories: 
- name: external-dns 
  url: https://kubernetes-sigs.github.io/external-dns/ 
 
 
releases: 
- name: external-dns 
  namespace: kube-system 
  chart: external-dns/external-dns 
  version: 1.7.0 
  values: 
  - podAnnotations: 
      iam.amazonaws.com/role: “<IAM role ARN>” 
  wait: true 
  timeout: 120

You can modify values.yaml in the helmchart according to your requirements by adding parameters for ExternalDNS,
sources:

• – service
• – ingress

domain-filter: test.devops.com # will make ExternalDNS see only the hosted zones matching provided domain, omit to process all available hosted zones.

provider: aws

policy: upsert-only # would prevent ExternalDNS from deleting any records, omit to enable full synchronization.
aws-zone-type:public # public or private hosted zones.

Run or test the syntax using diff command,

helmfile -f externaldns.yaml diff

If satisfied with the diff, run a sync or apply,

helmfile -f externaldns.yaml apply

Once deployed check if the pod is running,

kubectl get pods -n kube-system

Deploy a Kubernetes RabbitMQ application using helm file

In this example, we will deploy a RabbitMQ application and see how ExternalDNS works with ALB ingress objects. Once deployed it will dynamically add DNS records based on the hostname and ALB ingress.

The Helm chart for RabbitMQ is available on Github. We are modifying the chart and converting it into a helmfile.

Create the helmfile named rabbitmq.yaml

# helmfile -f rabbitmq.yaml diff 
repositories: 
- name: bitnami 
  url: https://charts.bitnami.com/bitnami 
 
 
releases: 
- name: rabbitmq 
  namespace: devops 
  chart: bitnami/rabbitmq 
  version: 8.24.12 
  values: 
  - auth: 
      username: “rabbitmq” 
      password: “rabbitmq” 
      erlangCookie: “ajhdkhhihkds” 
    persistence: 
      enabled: true 
    service: 
      type: NodePort 
    ingress: 
      enabled: true 
      path: /* 
      hostname: rabbitmq.devops.test.com 
      annotations: 
        kubernetes.io/ingress.class: alb 
        external-dns.alpha.kubernetes.io/hostname: rabbitmq.devops.test.com 
        alb.ingress.kubernetes.io/actions.response-404: | 
          {"Type":"fixed-response","FixedResponseConfig":{"ContentType":"text/plain","StatusCode":"404","MessageBody":"devops - 404 Page not found"}} 
        alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' 
        alb.ingress.kubernetes.io/certificate-arn: “arn:aws:acm:us-west-2:738379792:certificate/97394d-72255-6fd1b-a59e-79b9a80w42451” 
        alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]' 
        alb.ingress.kubernetes.io/load-balancer-attributes: access_logs.s3.enabled=true, access_logs.s3.bucket=devops-elb-logs,access_logs.s3.prefix=devops 
        alb.ingress.kubernetes.io/scheme: internet-facing 
        alb.ingress.kubernetes.io/security-groups: sg-08043e2c21117694234 
        alb.ingress.kubernetes.io/subnets: subnet-b5f64bff,subnet-431e40,subnet-564e90,subnet-58863b8 
        alb.ingress.kubernetes.io/group.name: devops 
  wait: true 
  timeout: 600

In podAnnotations please add external DNS annotation external-dns.alpha.kubernetes.io/hostname to create a record for the resulting ALB.

The above Kubernetes pod will create an (IPv4) ALB in AWS which will forward traffic to the RabbitMQ application.

Run or test the syntax using diff command:-

helmfile -f rabbitmq.yaml diff

If satisfied with the diff, run a sync or apply.

helmfile -f rabbitmq.yaml apply

Once it is deployed check the pod is running.

kubectl get pods

If the source is ingress, then ExternalDNS will create DNS records based on the hosts specified in the ingress objects. The above example will result in one alias record being created i.e rabbitmq.devops.test.com which is the alias of ALB that is associated with the ingress object.

About the author

Suraj Kamble is a DevOps engineer at Excellarate. He has expertise in tools and technologies such as Docker, Kubernetes, terraform, Jenkins, CloudFormation, Gitlab and Bitbucket CICD, Sumologic, Datadog, Grafana and Prometheus. Suraj holds a bachelor’s degree in computer engineering from the Savitribai Phule Pune University.