Kube Endpoints and Reverse-Proxying Non-Kube Services

For a while now, I’ve been using my hybrid-cloud K3s nodes to reverse proxy into a few services that are not running in my Kubernetes cluster. Originally, I planned on proxying the traffic I wanted in NGINX at the edge, but then it would be nearly impossible to pass through to Traefik without a lot of manual work and effort. I ended up doing it in a reasonably hacky way, but ultimately decided to move away from that approach due to maintenance and re-deployability concerns. Everything I needed to learn to get it working was found on a blog post from Eleven Labs, and I extended that information to get it working on Trafik running on Kubernetes. Functional, but definitely hacky.

Over the 2021-2022 holidays, I stumbled into a very interesting StackOverflow post. TL;DR: They recommended creating an Endpoint Object and just pointing the addresses to an external IP.

Well…that’s kind of a neat idea. Let’s see if we can get that approach working! First, let’s look over how the network traffic should be flowing.

Network flow

From the public internet, this is effectively how traffic makes it way from the user over the public internet to the service running on a VM host in my homelab.

Migrating over wiki.danmanners.com

Okay, so if I wanted to update things from A to B, it should probably look something like this:

Original Code

[http]
    [http.middlewares]
        [http.middlewares.http-https-redirectscheme.redirectScheme]
            scheme = "https"
            permanent = true
    [http.services]
    [http.services.wikijs]
        [http.services.wikijs.loadBalancer]
            [[http.services.wikijs.loadBalancer.servers]]
                url = "http://10.45.0.32:80/"

The original code to reference can be found here, here, and here.

New Code

---
apiVersion: v1
kind: Service
metadata:
  name: wikijs
  namespace: kube-system
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  clusterIP: None
  type: ClusterIP
---
apiVersion: v1
kind: Endpoints
metadata:
  name: wikijs
  namespace: kube-system
subsets:
- addresses:
  - ip: 10.45.0.32
  ports:
  - name: http
    port: 80
    protocol: TCP
---
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
  name: wikijs
  namespace: kube-system
  labels:
    app.kubernetes.io/instance: traefik
spec:
  weighted:
    services:
      - name: wikijs
        passHostHeader: true
        port: 80
        scheme: http

The full code for my homelab with Traefik can be viewed here

Does it work?

Short answer: Yes!

Longer answer: It definitely works, and latency from my house, over the internet, and back seems to be pretty consistently under 12ms. Not bad at all, especially considering how many hops there are!

Are there other possible challenges or issues I’m not aware of today? Absolutely. Is there perhaps an even better way to do things? Very well may be! Does this work quite easily, AND allow me to easily manage everything through GitOps? Yes, absolutely.

Questions? Thoughts?

Feel free to ping me at daniel.a.manners@gmail.com. If something I wrote isn’t clear, feel free to ask me a question or tell me to update it!