Trident: Trident node pod gets stuck in CrashLoopBackoff when the port conflicts with ephemeral ports used on the host.

[tksm]

Describe the bug

Trident node pod gets stuck in CrashLoopBackoff when the port conflicts with ephemeral ports used on the host.

Startup, liveness, and readiness probes for Trident node pods were introduced in Trident v21.04.0. The port for the probe is currently fixed as 34572 (changed to 51199 later). Since Trident node pods use hostNetwork: true and the port (34572, 51199) is in the range of ephemeral ports in many Linux Kernels (32768-60999), there is a certain probability of a port conflict that causes CrashLoopBackoff.

Environment

• Trident version: v21.04.0
• Trident installation flags used: silenceAutosupport: true (Trident Operator)
• Container runtime: Docker 20.10.2
• Kubernetes version: v1.20.1
• Kubernetes orchestrator: Kubernetes
• Kubernetes enabled feature gates:
• OS: Ubuntu 18.04.5
• NetApp backend types: ONTAP AFF 9.1P14
• Other:

To Reproduce

Steps to reproduce the behavior:

• Use 34572 port as a source port on the node before trident installation
  
  
  
  • e.g. nc -p 34572 localhost 22
  
  
• Install trident
• Trident node pod on the node will get stuck in CrashLoopBackoff

Expected behavior

Trident node pod should not fail for a non-essential reason.

In our case, making the probe port configurable will fix the problem.

Additional context

[gnarl]

This issue was fixed with commit 133da74 which is available in the Trident 21.04.1 release.