Traefik


  • Traefik is a cloud-hybrid reverse proxy and load balancer that makes deploying, configuring and integrating infrastructure components easy and automatic.

Install

  • Docker Compose
    • There should be an acme.json file that you create and pass through the docker with the permission of chmod 600.
    • Furthermore, there are two more files that you will have to configure and pass through before launching the traefik container. We provided them in the #config section below.
    • # Original written by Techo Tim
      version: '3'
      services:
      traefik:
      image: traefik:latest
      # - Latest image of traefik.
      container_name: traefik
      # - Container Name for our cluster.
      restart: unless-stopped
      security_opt:
      - no-new-privileges:true
      networks:
      # - Docker Network Name: proxy
      - proxy
      ports:
      - 80:80
      - 443:443
      environment:
      # - Below will be the cloudflare information
      - CF_API_EMAIL=user@example.com
      - CF_DNS_API_TOKEN=YOU_API_TOKEN
      # - You may need to append the environments with a _FILE, as it resolved the issue at https://github.com/KBVE/kbve.com/issues/208
      # - CF_API_KEY=YOU_API_KEY
      # be sure to use the correct one depending on if you are using a token or key
      volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      # - Updated locations for the traefik volumes.
      # - Replace `altoids` with the username.
      - /home/altoids/traefik/data/traefik.yml:/traefik.yml:ro
      - /home/altoids/traefik/data/acme.json:/acme.json
      - /home/altoids/traefik/data/config.yml:/config.yml:ro
      labels:
      # - Replace kbve.com with your domain
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.entrypoints=http"
      - "traefik.http.routers.traefik.rule=Host(`traefik-dashboard.local.kbve.com`)"
      - "traefik.http.middlewares.traefik-auth.basicauth.users=USER:BASIC_AUTH_PASSWORD"
      - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
      - "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
      - "traefik.http.routers.traefik-secure.entrypoints=https"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik-dashboard.local.kbve.com`)"
      - "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
      - "traefik.http.routers.traefik-secure.tls.domains[0].main=local.kbve.com"
      - "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.local.kbve.com"
      - "traefik.http.routers.traefik-secure.service=api@internal"
      networks:
      proxy:
      external: true
      view raw docker-compose.yml delivered with ❤ by EmGithub
      This is a docker compose for traefik.

Config

  • Traefik.yml Example
    • # Original written by Techno Tim
      api:
      dashboard: true
      debug: true
      entryPoints:
      http:
      address: ":80"
      http:
      redirections:
      entryPoint:
      to: https
      scheme: https
      https:
      address: ":443"
      serversTransport:
      insecureSkipVerify: true
      providers:
      docker:
      endpoint: "unix:///var/run/docker.sock"
      exposedByDefault: false
      file:
      filename: /config.yml
      certificatesResolvers:
      cloudflare:
      acme:
      email: you@example.com
      storage: acme.json
      dnsChallenge:
      provider: cloudflare
      resolvers:
      - "1.1.1.1:53"
      - "1.0.0.1:53"
      view raw traefik.yml delivered with ❤ by EmGithub
      This the primary config for our traefik.yml
  • Config.yml Router Example
    • # Original written by Techno Tim
      http:
      #region routers
      routers:
      proxmox:
      entryPoints:
      - "https"
      rule: "Host(`proxmox.local.kbve.com`)"
      middlewares:
      - default-headers
      - https-redirectscheme
      tls: {}
      service: proxmox
      pihole:
      entryPoints:
      - "https"
      rule: "Host(`pihole.local.kbve.com`)"
      middlewares:
      - default-headers
      - addprefix-pihole
      - https-redirectscheme
      tls: {}
      service: pihole
      homebridge:
      entryPoints:
      - "https"
      rule: "Host(`homebridge.local.kbve.com`)"
      middlewares:
      - default-headers
      - https-redirectscheme
      tls: {}
      service: homebridge
      homeassistant:
      # For Homeassistant config, check: https://www.home-assistant.io/integrations/http/#reverse-proxies
      # This relies on Homeassistant using http. No certs are needed in the Homeassistant config.
      entryPoints:
      - "https"
      rule: "Host(`homeassistant.local.kbve.com`)"
      middlewares:
      - default-headers
      - https-redirectscheme
      tls: {}
      service: homeassistant
      syncthing:
      entryPoints:
      - "https"
      rule: "Host(`syncthing.local.kbve.com`)"
      middlewares:
      - default-headers
      - https-redirectscheme
      tls: {}
      service: syncthing
      truenas:
      entryPoints:
      - "https"
      rule: "Host(`truenas.local.kbve.com`)"
      middlewares:
      - default-headers
      - https-redirectscheme
      tls: {}
      service: truenas
      plex:
      entryPoints:
      - "https"
      rule: "Host(`plex.local.kbve.com`)"
      middlewares:
      - default-headers
      - https-redirectscheme
      tls: {}
      service: plex
      minio:
      entryPoints:
      - "https"
      rule: "Host(`minio.local.kbve.com`)"
      middlewares:
      - default-headers
      - https-redirectscheme
      tls: {}
      service: minio
      rancher:
      entryPoints:
      - "https"
      rule: "Host(`rancher.local.kbve.com`)"
      middlewares:
      - default-headers
      - https-redirectscheme
      tls: {}
      service: rancher
      idrac:
      entryPoints:
      - "https"
      rule: "Host(`idrac.local.kbve.com`)"
      middlewares:
      - idrac
      - https-redirectscheme
      tls: {}
      service: idrac
      idrac-console:
      entryPoints:
      - "idrac" # REQUIRED for iDRAC virtual console: Create a new TCP entry point in traefik on port 5900
      rule: "Host(`idrac.local.kbve.com`)"
      middlewares:
      - idrac
      - https-redirectscheme
      tls: {}
      service: idrac-console
      opnsense:
      entryPoints:
      - "https"
      rule: "Host(`opnsense.local.kbve.com`)"
      middlewares:
      - default-headers
      - https-redirectscheme
      tls: {}
      service: opnsense
      pterodactyl:
      entryPoints:
      - "https"
      rule: "Host(`pterodactyl.local.kbve.com`)"
      middlewares:
      - default-headers
      - https-redirectscheme
      tls: {}
      service: pterodactyl
      #endregion
      #region services
      services:
      proxmox:
      loadBalancer:
      servers:
      - url: "https://192.168.0.100:8006"
      passHostHeader: true
      pihole:
      loadBalancer:
      servers:
      - url: "http://192.168.0.101:80"
      passHostHeader: true
      homebridge:
      loadBalancer:
      servers:
      - url: "http://192.168.0.102:10999"
      passHostHeader: true
      homeassistant:
      loadBalancer:
      servers:
      - url: "http://192.168.0.102:10999"
      passHostHeader: true
      syncthing:
      loadBalancer:
      servers:
      - url: "https://192.168.0.103:8384"
      passHostHeader: true
      truenas:
      loadBalancer:
      servers:
      - url: "https://192.168.0.104"
      passHostHeader: true
      plex:
      loadBalancer:
      servers:
      - url: "https://192.168.0.105:32400"
      passHostHeader: true
      minio:
      loadBalancer:
      servers:
      - url: "https://192.168.0.106:9000/"
      passHostHeader: true
      rancher:
      loadBalancer:
      servers:
      - url: "https://192.168.0.107"
      passHostHeader: true
      idrac:
      loadBalancer:
      servers:
      - url: "https://192.168.0.108"
      passHostHeader: true
      idrac-console:
      loadBalancer:
      servers:
      - url: "https://192.168.0.108:5900"
      passHostHeader: true
      opnsense:
      loadBalancer:
      servers:
      - url: "https://192.168.0.109"
      passHostHeader: true
      pterodactyl:
      loadBalancer:
      servers:
      - url: "http://192.168.0.110:80"
      passHostHeader: true
      #endregion
      middlewares:
      addprefix-pihole:
      addPrefix:
      prefix: "/admin"
      https-redirectscheme:
      redirectScheme:
      scheme: https
      permanent: true
      default-headers:
      headers:
      frameDeny: true
      browserXssFilter: true
      contentTypeNosniff: true
      forceSTSHeader: true
      stsIncludeSubdomains: true
      stsPreload: true
      stsSeconds: 15552000
      customFrameOptionsValue: SAMEORIGIN
      customRequestHeaders:
      X-Forwarded-Proto: https
      idrac:
      headers:
      frameDeny: true
      browserXssFilter: true
      forceSTSHeader: true
      stsIncludeSubdomains: true
      stsSeconds: 15552000
      customFrameOptionsValue: SAMEORIGIN
      customRequestHeaders:
      X-Forwarded-Proto: https
      default-whitelist:
      ipWhiteList:
      sourceRange:
      - "10.0.0.0/8"
      - "192.168.0.0/16"
      - "172.16.0.0/12"
      secured:
      chain:
      middlewares:
      - default-whitelist
      - default-headers
      view raw config.yml delivered with ❤ by EmGithub
      This the router config for our reverse proxy. Written by Techo Tim originaly and modified by our team.

Kubernetes

  • Patching Traefik on k3s cluster

    • We want to find the instance of where traefik is running. Running sudo kubectl get all -o wide --all-namespaces should display all your containers, look for traefik.

    • Patch

      • sudo kubectl patch svc traefik -n kube-system -p '{"spec":{"externalTrafficPolicy":"Cluster"}}'`
    • std output should be service/traefik patched

  • Helm Charts

    • helm repo add traefik https://helm.traefik.io/traefik
      • Sucess: std output should be

        • "traefik" has been added to your repositories
    • helm repo update
  • Traefik Middleware for Kubernetes

    • Middleware kind should be isolated for performance and security reasons.
      • Auth - Kind: Middleware
        • Example:

          • apiVersion: traefik.containo.us/v1alpha1
            kind: Middleware
            metadata:
              name: longhorn-auth
              namespace: longhorn-system
            spec:
              basicAuth:
                secret: authsecret
            • The middleware should be saved as a yaml / yml file and applied using kubectl.
      • Auth - Kind: Ingress
        • Calling the longhorn-auth in the Ingress via annotations:
          • Example:

            •           
                apiVersion: networking.k8s.io/v1
                kind: Ingress
                metadata:
                  name: longhorn-ing-traefik
                  namespace: longhorn-system
                  annotations:
                    externalTrafficPolicy: Local 
                    kubernetes.io/ingress.class: traefik
                    traefik.ingress.kubernetes.io/router.middlewares: longhorn-system-longhorn-auth@kubernetescrd
                    ingress.kubernetes.io/whitelist-x-forwarded-for: "true"
                    
                spec:
                  rules:
                  - host: "x.kbve.com"
                    http:
                      paths:
                      - path: /
                        pathType: Prefix
                        backend:
                          service:
                            name: longhorn-service-provider
                            port:
                              number: 8000
              
              
            • In our PoC above, we see that the middleware is referenced as:

                  traefik.ingress.kubernetes.io/router.middlewares: longhorn-system-longhorn-auth@kubernetescrd

              Its important to note the namespace of the middleware, longhorn-system , before calling the middleware’s name. This is to let the crd know where the middleware is located.


Notes

According to the notes on Traefik & Kubernetes we first need to install the Resource Definitions and RBAC into kubectl by running the following commands:

# Install Traefik Resource Definitions:
kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v2.8/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml

# Install RBAC for Traefik:
kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v2.8/docs/content/reference/dynamic-configuration/kubernetes-crd-rbac.yml

After this installation, we’ll have a set of Custom Resource Definitions which should have the following benefits:

  • The usage of name and namespace to refer to another Kubernetes resource.
  • The usage of secret for sensitive data (TLS certificates and credentials).
  • The structure of the configuration.
  • The requirement to declare all the definitions.

See the list of CRDs in the dedicated routing section.

The biggest thing we need from this is the ability to add the BasicAuth plugin. This plugin (which is what we tried to reference before with the auth@file line) uses an htpasswd password to block incoming traffic to the pod.

This will require setting up an IngressRoute (which is a specific Kubernetes resource added by the Traefik Resource Definitions) with settings to specify what the middlewares are. Find more info on the Traefik Middlewares Here


Cloudflare

These are notes on integrating Cloudflare with Traefik, including automating some of the actions so that you may not have to repeat them.

Acme Docs

Official Docs

Access the API Tokens directly from Cloudflare Profile

Common environmental variable names and their purpose:

  • CF_API_EMAIL - The Cloudflare account holder’s email.
  • CF_API_KEY - The Cloudflare API key.
  • CF_DNS_API_TOKEN - The API token with DNS:Edit permission.
  • CF_ZONE_API_TOKEN - The API token with Zone:Read permission.