0

I administrate kubernetes clusters running software that is not always perfect. Some have memory leaks, some have cpu leaks, some have both.

I would like to automatically kill the leaking pods. The ones who leak memory are easy, I just put a memory limit and they die eventually when they reach the limit. However CPU seems more tricky. I do not want to put a CPU limit, that will just make them slower and not kill them.

These days I visit regularly the grafana showing the prometheus cpu charts, and once in a while I can spot a pod stuck in an infinite loop, or a pod increasing its CPU usage steadily. I then kill them manually.

I believe no such feature is standard in kubernetes, but I wonder if a plugin exists, or a simple hack.

Improve this question
9
  • 1
    check this link
    – asktyagi
    Jan 2 at 14:11
  • 1
    I believe answer is in discussion itself, you can use probes to find container or application health and kubelet will kill it for you based on your policies.
    – asktyagi
    Jan 2 at 14:56
  • 1
    Do you have any say as to what resides in the pod images either during the build or with a base image? My guess is you're going to need to implement a liveness probe via a command to check CPU usage and let it die if it goes above a threshold.
    – Matt W
    Jan 2 at 23:42
  • 1
    Please check this.
    – Abhijith Chitrapu
    Jan 3 at 11:04
  • 1
    As per my knowledge we cannot kill a Pod automatically apart from resource management.
    – Abhijith Chitrapu
    Jan 4 at 5:31

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .