Ingress-nginx: Nginx child reaping broken

[n1koo]

I think https://github.com/kubernetes/ingress/pull/577 broke master.

I'm seeing

W0412 13:19:02.827411       1 template.go:113] unexpected error cleaning template: wait: no child processes
W0412 13:19:02.831736       1 queue.go:94] requeuing gke-tierstaging-us-east1-default-pool-59d9c8bc-t03k, err
-------------------------------------------------------------------------------
Error: waitid: no child processes
nginx: the configuration file /tmp/nginx-cfg258743890 syntax is ok
2017/04/12 13:19:02 [emerg] 395#395: no "events" section in configuration
nginx: [emerg] no "events" section in configuration
nginx: configuration file /tmp/nginx-cfg258743890 test failed

With this patch, but without it totally works. Controllers end up in a crashloop due to this.

I think the problem is that as we spawn things like grep they finish waster than we can monitor them. Happy to supply more information as needed, but this is pretty easily reproducible with legit templates being used

[aledbf]

@n1koo please test if the image quay.io/aledbf/nginx-ingress-controller:0.88 solves the issue

[agustincastanio]

Hi, I am using gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.4 and still getting:
~~
Error: exit status 1
nginx: the configuration file /tmp/nginx-cfg262860016 syntax is ok
2017/04/27 19:49:09 [emerg] 10749#10749: no "events" section in configuration
nginx: [emerg] no "events" section in configuration
nginx: configuration file /tmp/nginx-cfg262860016 test failed
~~

[aledbf]

@agustincastanio this is one of the reasons why we need to release beta.5 asap
Please test the image quay.io/aledbf/nginx-ingress-controller:0.106

[Malet]

2017/05/04 10:55:38 [notice] 16188#16188: signal process started
2017/05/04 10:55:38 [alert] 16188#16188: kill(12, 1) failed (3: No such process)
nginx: [alert] kill(12, 1) failed (3: No such process)

Getting this with beta.4, new routes failing to be loaded as a result

[aledbf]

@Malet please update to beta.5

[Malet]

@aledbf switched to beta.5 now to see if that fixes it, quay.io/aledbf/nginx-ingress-controller:0.106 did not fix the issue though

[agustincastanio]

I have been using beta.5 for five days without this issue.
Good job!

[fortytw2]

I'm currently having the exact same issue as @Malet on beta.5

2017/05/05 12:40:08 [notice] 11013#11013: signal process started
2017/05/05 12:40:08 [alert] 11013#11013: kill(22, 1) failed (3: No such process)
nginx: [alert] kill(22, 1) failed (3: No such process)
W0505 12:40:09.258753       1 queue.go:88] requeuing secret-ns/svc-name, err wait: no child processes

EDIT: but I think this is the wrong issue to report? apologies

[Malet]

Still getting this issue on beta.5 also, might have a few more ingress rules than most though

kubectl get ingress | wc -l
     173

[edbergavera]

We had the same exact issue:

2017-05-18T03:03:16.185433133Z 2017/05/18 03:03:16 [emerg] 21#21: could not build server_names_hash, you should increase server_names_hash_bucket_size: 64
2017-05-18T03:03:16.185436987Z nginx: [emerg] could not build server_names_hash, you should increase server_names_hash_bucket_size: 64
2017-05-18T03:03:16.185440281Z nginx: configuration file /tmp/nginx-cfg421426481 test failed

It was fixed by gcr.io/google-containers/nginx-ingress-controller:0.9.0-beta.5

Cheers!

[yawboateng]

running beta 5 and im seeing similar issue:

E0520 16:24:28.489907       1 controller.go:416] unexpected failure restarting the backend: 
2017/05/20 16:24:28 [notice] 32168#32168: signal process started
2017/05/20 16:24:28 [alert] 32168#32168: kill(19, 1) failed (3: No such process)
nginx: [alert] kill(19, 1) failed (3: No such process)
W0520 16:24:28.489942       1 queue.go:88] requeuing monitoring/kube-dashboard, err exit status 1

[jeffrichie]

I'm on 0.9.0-beta.11 and the same issue, FYI.

[aledbf]

@jeffrichiefs please update to 0.9.0-beta.15. We changed how we monitor the nginx process