I couldn't find a documented way to do this in either the Traefik nor the K3S documentation so I'm going to detail the steps here.

Traefik has a very annoying 60 second timeout on reads, which means that any request that takes longer than this (e.g a file upload) will be automatically cancelled. This is a huge issue for anything other than basic web content.

The Traefik Helm Chart contains the following stanza which you'd think would allow for the read timeout to be increased:

ports:
  web:
    transport:
      respondingTimeouts:
        readTimeout:
        writeTimeout:
        idleTimeout:
  websecure:
    transport:
      respondingTimeouts:
        readTimeout:
        writeTimeout:
        idleTimeout:

However, this is a total red herring - setting these values has no effect at all! The chart does not consume them and no changes will be made to the Traefik deployment.

Instead, the timeout configuration must be passed via the `additionalArguments` parameter:

additionalArguments:
    - "--entryPoints.web.transport.respondingTimeouts.readTimeout=3600s"
    - "--entryPoints.web.transport.respondingTimeouts.writeTimeout=3600s"
    - "--entryPoints.web.transport.respondingTimeouts.idleTimeout=3600s"
    - "--entryPoints.websecure.transport.respondingTimeouts.readTimeout=3600s"
    - "--entryPoints.websecure.transport.respondingTimeouts.writeTimeout=3600s"
    - "--entryPoints.websecure.transport.respondingTimeouts.idleTimeout=3600s"

To apply this in K3S, this must be added to a HelmChartConfig CRD and applied to the cluster:

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    additionalArguments:
      - "--entryPoints.web.transport.respondingTimeouts.readTimeout=3600s"
      - "--entryPoints.web.transport.respondingTimeouts.writeTimeout=3600s"
      - "--entryPoints.web.transport.respondingTimeouts.idleTimeout=3600s"
      - "--entryPoints.websecure.transport.respondingTimeouts.readTimeout=3600s"
      - "--entryPoints.websecure.transport.respondingTimeouts.writeTimeout=3600s"
      - "--entryPoints.websecure.transport.respondingTimeouts.idleTimeout=3600s"

If done correctly, Traefik should now restart and apply the correct timeout configuration. I've set my timeout to 3600s (1 hour) in this example.

To persist these changes across restarts, the HelmChartConfig manifest above will need to be added to a file named "traefik-config.yaml" and placed in the "/var/lib/rancher/k3s/server/manifests" folder on the K3S node.

Working this out took several hours of debugging, even with the assistance of ChatGPT. Maybe I should have stuck to the more widely supported NGINX Ingress Controller instead....

Submitted by admin on Sun, 10/20/2024 - 22:25