Kubernetes-ingress: Kubernetes on private network

[fattahmahran]

I installed fabric8 on top of kubernetes1.4.7 and I can't reach the URL for fabric8. I checked and I found the below error. I have kubernetes cluster with 3 servers. If you could figureout the error or guide me how to increase log level so I can get more details about the error.

I'm running kubernetes behind a proxy and all system is using this proxy. how can I configure ingress to use this proxy? I'm not sure if this is the issue.

root@kuber-master:~# kubectl -n fabric8-system logs -f ingress-nginx-2015555637-0obbr
I1216 18:48:12.848439 1 nginx.go:240] Writing NGINX conf to /etc/nginx/nginx.conf

user nginx;
worker_processes 1;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
worker_connections 1024;
}

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

access_log  /var/log/nginx/access.log  main;

sendfile        on;
#tcp_nopush     on;

keepalive_timeout  65;

#gzip  on;

server_names_hash_max_size 512;


map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

include /etc/nginx/conf.d/*.conf;

}
I1216 18:48:12.849306 1 nginx.go:258] The main NGINX configuration file had been updated
I1216 18:48:12.849578 1 nginx.go:213] executing nginx
F1216 18:48:12.876396 1 nginx.go:196] Failed to start nginx

---

root@kuber-master:~# kubectl -n fabric8-system describe pod ingress-nginx-2015555637-0obbr
Name: ingress-nginx-2015555637-0obbr
Namespace: fabric8-system
Node: 10.103.12.20/10.103.12.20
Start Time: Fri, 16 Dec 2016 13:46:44 -0500
Labels: group=io.fabric8.devops.apps
pod-template-hash=2015555637
project=ingress-nginx
provider=fabric8
version=2.2.298
Status: Running
IP: 172.16.97.5
Controllers: ReplicaSet/ingress-nginx-2015555637
Containers:
nginx-ingress:
Container ID: docker://99835d6dc4bf725edcaefd2e18adcb6adf8c61e56f038033bf8825a0c5023b23
Image: nginxdemos/nginx-ingress:0.3.1
Image ID: docker://sha256:3773d84614b893eb2577d64286bca8bd21eeaf5967d9124638b8a105909f0166
Ports: 80/TCP, 443/TCP
Args:
-v=3
-nginx-configmaps=fabric8-system/nginx-config
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 255
Started: Fri, 16 Dec 2016 13:52:26 -0500
Finished: Fri, 16 Dec 2016 13:52:26 -0500
Ready: False
Restart Count: 6
Volume Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-f7z19 (ro)
Environment Variables:
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
default-token-f7z19:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-f7z19
QoS Class: BestEffort
Tolerations:
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
10m 10m 1 {default-scheduler } Normal Scheduled Successfully assigned ingress-nginx-2015555637-0obbr to 10.103.12.20
10m 10m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Created Created container with docker id 47ae0861f0ff; Security:[seccomp=unconfined]
10m 10m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Started Started container with docker id 47ae0861f0ff
10m 10m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Created Created container with docker id 5db192acf5ee; Security:[seccomp=unconfined]
10m 10m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Started Started container with docker id 5db192acf5ee
10m 10m 2 {kubelet 10.103.12.20} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "nginx-ingress" with CrashLoopBackOff: "Back-off 10s restarting failed container=nginx-ingress pod=ingress-nginx-2015555637-0obbr_fabric8-system(fd94c9d3-c3bf-11e6-b3d9-0050560116bd)"

10m 10m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Created Created container with docker id e915dcf51f5b; Security:[seccomp=unconfined]
10m 10m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Started Started container with docker id e915dcf51f5b
10m 9m 2 {kubelet 10.103.12.20} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "nginx-ingress" with CrashLoopBackOff: "Back-off 20s restarting failed container=nginx-ingress pod=ingress-nginx-2015555637-0obbr_fabric8-system(fd94c9d3-c3bf-11e6-b3d9-0050560116bd)"

9m 9m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Started Started container with docker id 15e4f22a796c
9m 9m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Created Created container with docker id 15e4f22a796c; Security:[seccomp=unconfined]
9m 9m 3 {kubelet 10.103.12.20} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "nginx-ingress" with CrashLoopBackOff: "Back-off 40s restarting failed container=nginx-ingress pod=ingress-nginx-2015555637-0obbr_fabric8-system(fd94c9d3-c3bf-11e6-b3d9-0050560116bd)"

8m 8m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Created Created container with docker id 8bb3380e09ab; Security:[seccomp=unconfined]
8m 8m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Started Started container with docker id 8bb3380e09ab
8m 7m 7 {kubelet 10.103.12.20} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "nginx-ingress" with CrashLoopBackOff: "Back-off 1m20s restarting failed container=nginx-ingress pod=ingress-nginx-2015555637-0obbr_fabric8-system(fd94c9d3-c3bf-11e6-b3d9-0050560116bd)"

7m 7m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Created Created container with docker id c1ab9788399b; Security:[seccomp=unconfined]
7m 7m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Started Started container with docker id c1ab9788399b
7m 4m 13 {kubelet 10.103.12.20} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "nginx-ingress" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=nginx-ingress pod=ingress-nginx-2015555637-0obbr_fabric8-system(fd94c9d3-c3bf-11e6-b3d9-0050560116bd)"

10m 4m 7 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Pulled Container image "nginxdemos/nginx-ingress:0.3.1" already present on machine
4m 4m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Created Created container with docker id 99835d6dc4bf; Security:[seccomp=unconfined]
4m 4m 1 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Normal Started Started container with docker id 99835d6dc4bf
10m 11s 48 {kubelet 10.103.12.20} spec.containers{nginx-ingress} Warning BackOff Back-off restarting failed docker container
4m 11s 21 {kubelet 10.103.12.20} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "nginx-ingress" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=nginx-ingress pod=ingress-nginx-2015555637-0obbr_fabric8-system(fd94c9d3-c3bf-11e6-b3d9-0050560116bd)"

[pleshakov]

@abdoo
My apologies for the delayed response. From my understanding, this file is used to deploy the controller https://github.com/fabric8io/fabric8-devops/blob/master/ingress-nginx/src/main/fabric8/deployment.yml As you can see, ports 80 and 443 on the container are mapped to ports 80 and 443 on the host machine. NGINX can't start probably because one or both of those ports are taken by some other application or a container running on that machine.

Unfortunately, we don't report the exact error when NGINX isn't able start. This will be fixed.

[fattahmahran]

Thanks for the response.

I try to check what's using these ports. I have another question related to
this. my servers are running on private Env. so I need proxy to connect to
Internet. I configured all applications on these servers to use this proxy
(docker, curl..etc). does NGINX needs to be configured to use this proxy as
well?

Thanks,
best regards,

On Mon, Dec 19, 2016 at 8:42 AM, Michael Pleshakov <[email protected]

wrote:

@abdoo https://github.com/abdoo
My apologies for the delayed response. From my understanding, this file is
used to deploy the controller https://github.com/fabric8io/
fabric8-devops/blob/master/ingress-nginx/src/main/fabric8/deployment.yml
As you can see, ports 80 and 443 on the container are mapped to ports 80
and 443 on the host machine. NGINX can't start probably because one or both
of those ports are taken by some other application or a container running
on that machine.

Unfortunately, we don't report the exact error when NGINX isn't able
start. This will be fixed.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-267966566,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhz4_l7jwLtoIwt1o-lmUuZtM28WWJks5rJonIgaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[fattahmahran]

also I checked these two ports 80 and 443 and they not being used on the
server.

root@kuber-master:~# telnet 0 80
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
Connection closed by foreign host.
root@kuber-master:~# telnet 0 443
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
Connection closed by foreign host.
root@kuber-master:~#

On Mon, Dec 19, 2016 at 10:21 AM, Abd El-Fattah Mahran <
[email protected]> wrote:

Thanks for the response.

I try to check what's using these ports. I have another question related
to this. my servers are running on private Env. so I need proxy to connect
to Internet. I configured all applications on these servers to use this
proxy (docker, curl..etc). does NGINX needs to be configured to use this
proxy as well?

Thanks,
best regards,

On Mon, Dec 19, 2016 at 8:42 AM, Michael Pleshakov <
[email protected]> wrote:

@abdoo https://github.com/abdoo
My apologies for the delayed response. From my understanding, this file
is used to deploy the controller https://github.com/fabric8io/f
abric8-devops/blob/master/ingress-nginx/src/main/fabric8/deployment.yml
As you can see, ports 80 and 443 on the container are mapped to ports 80
and 443 on the host machine. NGINX can't start probably because one or both
of those ports are taken by some other application or a container running
on that machine.

Unfortunately, we don't report the exact error when NGINX isn't able
start. This will be fixed.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-267966566,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhz4_l7jwLtoIwt1o-lmUuZtM28WWJks5rJonIgaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[pleshakov]

@abdoo

I try to check what's using these ports. I have another question related to
this. my servers are running on private Env. so I need proxy to connect to
Internet. I configured all applications on these servers to use this proxy
(docker, curl..etc). does NGINX needs to be configured to use this proxy as
well?

NGINX doesn't require outbound Internet connectivity so there's no need to configure it for NGINX.

also I checked these two ports 80 and 443 and they not being used on the
server.

I think it's the issue when your Kubernetes environment doesn't allow to create containers that expose privileged ports on the host. Could you try to add

   securityContext:
      privileged: true

to the NGINX deployment yaml file and then redeploy the controller, similar to https://github.com/nginxinc/kubernetes-ingress/pull/53/files ?

[r4j4h]

Incase it is not from the security context, are you using CNI for your cluster? If so the hostPort entries are ignored right now by CNI implementations (see this issue). If that is indeed your problem, try hostNetworking: true to make the container use the host's network instead of just the ports. This works in the meantime over CNI.

[fattahmahran]

Hi Michael,

this is the information I have from the container

spec:
containers:
- args:
- -v=3
- -nginx-configmaps=fabric8-system/nginx-config
image: nginxdemos/nginx-ingress:0.3.1
imagePullPolicy: IfNotPresent
name: nginx-ingress
ports:
- containerPort: 80
hostPort: 80
name: http
protocol: TCP
- containerPort: 443
hostPort: 443
name: https
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
dnsPolicy: ClusterFirst
nodeSelector:
fabric8.io/externalIP: "true"
restartPolicy: Always
securityContext: {}
terminationGracePeriodSeconds: 30

I tried to add the information like that
spec:
containers:
- args:
- -v=3
- -nginx-configmaps=fabric8-system/nginx-config
image: nginxdemos/nginx-ingress:0.3.1
imagePullPolicy: IfNotPresent
name: nginx-ingress
ports:
- containerPort: 80
hostPort: 80
name: http
protocol: TCP
securityContext:
privileged: true
- containerPort: 443
hostPort: 443
name: https
protocol: TCP
securityContext:
privileged: true
resources: {}
terminationMessagePath: /dev/termination-log
dnsPolicy: ClusterFirst
nodeSelector:
fabric8.io/externalIP: "true"
restartPolicy: Always
securityContext: {}
terminationGracePeriodSeconds: 30

I got this error

A copy of your changes has been stored to "/tmp/kubectl-edit-fplln.yaml"
found invalid field securityContext for v1.ContainerPort
found invalid field securityContext for v1.ContainerPort
found invalid field privileged for v1.ContainerPort

Can you guide is this is the correct place to add?

Thanks,
best regards,

On Mon, Dec 19, 2016 at 12:07 PM, Michael Pleshakov <
[email protected]> wrote:

@abdoo https://github.com/abdoo

I try to check what's using these ports. I have another question related to
this. my servers are running on private Env. so I need proxy to connect to
Internet. I configured all applications on these servers to use this proxy
(docker, curl..etc). does NGINX needs to be configured to use this proxy as
well?

NGINX doesn't require outbound Internet connectivity so there's no need to
configure it for NGINX.

also I checked these two ports 80 and 443 and they not being used on the
server.

I think it's the issue when your Kubernetes environment doesn't allow to
create containers that expose privileged ports on the host. Could you try
to add

securityContext:
privileged: true

to the NGINX deployment yaml file and then redeploy the controller,
similar to https://github.com/nginxinc/kubernetes-ingress/pull/53/files ?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-268019894,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhzz3yA2pWY2LQWr8UyCLDdYLAwRRWks5rJrm_gaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[fattahmahran]

hi Jasmine,

I'm not sure if Im using it or not. I just checked ubuntu/util.sh and I
found this code.

function create-kubelet-opts() {
if [ -n "$7" ] ; then
cni_opts=" --network-plugin=cni --network-plugin-dir=/etc/cni/net.d"
else
cni_opts=""
fi
cat < KUBELET_OPTS="\
--hostname-override=${1} \
--api-servers=http://${2}:8080 \
--logtostderr=true \
--cluster-dns=${3} \
--cluster-domain=${4} \
--config=${5} \
--allow-privileged=true
$cni_opts"
EOF
}

I'm not sure which condition happened?

On Mon, Dec 19, 2016 at 1:38 PM, Jasmine Hegman [e-mail]
wrote:

Incase it is not from the security context, are you using CNI for your
cluster? If so the hostPort entries are ignored right now by CNI
implementations (see this issue
https://github.com/kubernetes/kubernetes/issues/31307#issuecomment-267468950).
If that is indeed your problem, try hostNetworking: true to make the
container use the host's network instead of just the ports. This works in
the meantime over CNI.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-268042486,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhz_4Ow370rzAtjDOfzCEm_rCSPHo8ks5rJs80gaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[pleshakov]

@abdoo
securityContext must be put one level up into the Container spec, not the Port spec.

[fattahmahran]

Thanks a million Michael Pod is Running now :-)

I1219 19:29:14.620935 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:14.620984 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:14.621007 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:14.645287 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:14.645334 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:14.645344 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager
I1219 19:29:16.634414 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:16.634486 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:16.634498 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:16.657874 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:16.657916 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:16.657926 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager
I1219 19:29:18.642477 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:18.642554 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:18.642575 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:18.687874 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:18.687969 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:18.687992 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager
I1219 19:29:20.651936 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:20.652033 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:20.652053 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:20.699236 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:20.699307 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:20.699325 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager
I1219 19:29:22.668951 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:22.669028 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:22.669039 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:22.710568 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:22.710606 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:22.710616 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager
I1219 19:29:24.679679 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:24.679790 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:24.679962 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:24.722250 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:24.722304 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:24.722338 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager

On Mon, Dec 19, 2016 at 2:14 PM, Michael Pleshakov <[email protected]

wrote:

@abdoo https://github.com/abdoo
securityContext must be put one level up into the Container spec, not the
Port spec.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-268051564,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhz4Oa7BsSPdEV0BgXgGeE73oqh-XSks5rJteigaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[fattahmahran]

Hi Michael,

one more thing, now when I'm trying to access
http://fabric8.default.10.103.12.20.nip.io/

it returns 502 bad gateway (nginx/1.11.4)

On Mon, Dec 19, 2016 at 2:33 PM, Abd El-Fattah Mahran <
[email protected]> wrote:

Thanks a million Michael Pod is Running now :-)

I1219 19:29:14.620935 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:14.620984 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:14.621007 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:14.645287 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:14.645334 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:14.645344 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager
I1219 19:29:16.634414 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:16.634486 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:16.634498 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:16.657874 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:16.657916 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:16.657926 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager
I1219 19:29:18.642477 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:18.642554 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:18.642575 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:18.687874 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:18.687969 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:18.687992 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager
I1219 19:29:20.651936 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:20.652033 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:20.652053 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:20.699236 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:20.699307 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:20.699325 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager
I1219 19:29:22.668951 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:22.669028 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:22.669039 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:22.710568 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:22.710606 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:22.710616 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager
I1219 19:29:24.679679 1 controller.go:154] Endpoints kube-scheduler
changed, syncing
I1219 19:29:24.679790 1 utils.go:70] Syncing
kube-system/kube-scheduler
I1219 19:29:24.679962 1 controller.go:275] Syncing endpoints
kube-system/kube-scheduler
I1219 19:29:24.722250 1 controller.go:154] Endpoints
kube-controller-manager changed, syncing
I1219 19:29:24.722304 1 utils.go:70] Syncing
kube-system/kube-controller-manager
I1219 19:29:24.722338 1 controller.go:275] Syncing endpoints
kube-system/kube-controller-manager

On Mon, Dec 19, 2016 at 2:14 PM, Michael Pleshakov <
[email protected]> wrote:

@abdoo https://github.com/abdoo
securityContext must be put one level up into the Container spec, not
the Port spec.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-268051564,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhz4Oa7BsSPdEV0BgXgGeE73oqh-XSks5rJteigaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[pleshakov]

@abdoo

NGINX returns the bad gateway error when there is no healthy upstream servers available. Please double check that the containers of the service you are exposing via NGINX are successfully created and running properly. Checking the controller logs also night help to determine the problem.

Additionally, I suggest to try to use the latest version of the controller, which is nginxdemos/nginx-ingress:0.6.0

[fattahmahran]

hi Michael,

all services are running normally.

root@kuber-master:~# kubectl get pods
NAME READY STATUS RESTARTS
AGE
configmapcontroller-3703242456-s914e 1/1 Running 0 5h
exposecontroller-1115949590-fuxxw 1/1 Running 0 5h
fabric8-3192130638-0fue6 2/2 Running 0 5h
fabric8-docker-registry-1981087684-rr84r 1/1 Running 0 5h
fabric8-forge-3730694965-ldqb6 1/1 Running 0 5h
gogs-1853259817-dxkd0 1/1 Running 0 5h
jenkins-1661962136-9ugls 1/1 Running 0 5h
nexus-3628846063-ejfcx 1/1 Running 0 5h
root@kuber-master:~# kubectl logs -f fabric8-3192130638-0fue6
Error from server: a container name must be specified for pod
fabric8-3192130638-0fue6, choose one of: [fabric8 jenkinshift]
root@kuber-master:~# kubectl logs -f fabric8-3192130638-0fue6 fabric8
2016/12/19 20:59:01 Creating config file:
/site/osconsole/config.basic.js.tmpl => /site/osconsole/config.js
2016/12/19 20:59:01
2016/12/19 20:59:52 Creating service proxy: /k8s/ =>
https://192.168.3.1:443/
2016/12/19 20:59:52 Listening on :9090
2016/12/19 20:59:52
^C
root@kuber-master:~# kubectl logs -f fabric8-3192130638-0fue6 jenkinshift
2016-12-19 20:59:02.092742 I | Invoking Jenkins on URL http://jenkins/
2016-12-19 20:59:02.092919 I | About to query Kubernetes
2016-12-19 20:59:02.133939 I | Using namespace default
2016-12-19 20:59:02.134879 I | jenkinshift start listening on localhost:9191
^C
root@kuber-master:~# kubectl logs -f gogs-1853259817-dxkd0
2016/12/19 20:58:12 Creating admin user: true
2016/12/19 20:58:12 [T] Custom path: /opt/gogs/custom
2016/12/19 20:58:12 [T] Log path: /app/gogs/data/logs
2016/12/19 20:58:12 [I] Gogs: Go Git Service
2016/12/19 20:58:15 Custom path: /opt/gogs/custom
2016/12/19 20:58:15 Admin account created: gogsadmin
2016/12/19 20:58:15 [T] Custom path: /opt/gogs/custom
2016/12/19 20:58:15 [T] Log path: /app/gogs/data/logs
2016/12/19 20:58:15 [I] Gogs: Go Git Service 0.9.97.0901
^C

I tried to use 0.6.0 and it didn't work

I1220 02:31:54.941108 1 main.go:37] Starting NGINX Ingress controller
Version 0.6.0
I1220 02:31:54.942405 1 nginx.go:251] Writing NGINX conf to
/etc/nginx/nginx.conf

user nginx;
worker_processes 1;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
worker_connections 1024;
}

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

log_format  main  '$remote_addr - $remote_user [$time_local] "$request"

'
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;

sendfile        on;
#tcp_nopush     on;

keepalive_timeout  65;

#gzip  on;

server_names_hash_max_size 512;


map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

include /etc/nginx/conf.d/*.conf;

}
I1220 02:31:54.943303 1 nginx.go:269] The main NGINX configuration
file had been updated
I1220 02:31:54.943508 1 nginx.go:224] executing nginx
F1220 02:31:54.959240 1 nginx.go:207] Failed to start nginx

spec:
containers:
- args:
- -v=3
- -nginx-configmaps=fabric8-system/nginx-config
image: nginxdemos/nginx-ingress:0.6.0
imagePullPolicy: IfNotPresent
name: nginx-ingress
ports:
- containerPort: 80
hostPort: 80
name: http
protocol: TCP
- containerPort: 443
hostPort: 443
name: https
protocol: TCP
resources: {}
securityContext:
privileged: true
terminationMessagePath: /dev/termination-log
dnsPolicy: ClusterFirst
nodeSelector:
fabric8.io/externalIP: "true"
restartPolicy: Always
securityContext: {}

Thanks,
best regards,

On Mon, Dec 19, 2016 at 4:32 PM, Michael Pleshakov <[email protected]

wrote:

@abdoo https://github.com/abdoo

NGINX returns the bad gateway error when there is no healthy upstream
servers available. Please double check that the containers of the service
you are exposing via NGINX are successfully created and running properly.
Checking the controller logs also night help to determine the problem.

Additionally, I suggest to try to use the latest version of the
controller, which is nginxdemos/nginx-ingress:0.6.0

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-268084684,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhz30UqpXJ9KOxj737SnTr0qV6ksxXks5rJvfUgaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[fattahmahran]

root@kuber-master:~# kubectl get services
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
fabric8 192.168.3.182 80/TCP 5m
fabric8-docker-registry 192.168.3.113 80/TCP 5m
fabric8-forge 192.168.3.134 80/TCP 5m
gogs 192.168.3.252 80/TCP 5m
gogs-ssh 192.168.3.123 22/TCP 5m
jenkins 192.168.3.194 80/TCP 5m
jenkins-jnlp 192.168.3.32 50000/TCP 5m
jenkinshift 192.168.3.8 80/TCP 5m
kubernetes 192.168.3.1 443/TCP 3d
mongo 192.168.3.201 27017/TCP 3d
nexus 192.168.3.111 80/TCP 5m
root@kuber-master:~#
root@kuber-master:~# kubectl get endpoints
NAME ENDPOINTS AGE
fabric8 172.16.87.3:9090 5m
fabric8-docker-registry 172.16.49.4:5000 5m
fabric8-forge 172.16.97.4:8080 5m
gogs 172.16.49.6:3000 5m
gogs-ssh 172.16.49.6:2222 5m
jenkins 5m
jenkins-jnlp 5m
jenkinshift 172.16.87.3:9191 5m
kubernetes 10.103.12.20:6443 3d
mongo 3d
nexus 172.16.49.2:8081 5m
root@kuber-master:~#

On Mon, Dec 19, 2016 at 10:02 PM, Abd El-Fattah Mahran <
[email protected]> wrote:

hi Michael,

all services are running normally.

root@kuber-master:~# kubectl get pods
NAME READY STATUS RESTARTS
AGE
configmapcontroller-3703242456-s914e 1/1 Running 0
5h
exposecontroller-1115949590-fuxxw 1/1 Running 0
5h
fabric8-3192130638-0fue6 2/2 Running 0
5h
fabric8-docker-registry-1981087684-rr84r 1/1 Running 0
5h
fabric8-forge-3730694965-ldqb6 1/1 Running 0
5h
gogs-1853259817-dxkd0 1/1 Running 0
5h
jenkins-1661962136-9ugls 1/1 Running 0
5h
nexus-3628846063-ejfcx 1/1 Running 0
5h
root@kuber-master:~# kubectl logs -f fabric8-3192130638-0fue6
Error from server: a container name must be specified for pod
fabric8-3192130638-0fue6, choose one of: [fabric8 jenkinshift]
root@kuber-master:~# kubectl logs -f fabric8-3192130638-0fue6 fabric8
2016/12/19 20:59:01 Creating config file: /site/osconsole/config.basic.js.tmpl
=> /site/osconsole/config.js
2016/12/19 20:59:01
2016/12/19 20:59:52 Creating service proxy: /k8s/ =>
https://192.168.3.1:443/
2016/12/19 20:59:52 Listening on :9090
2016/12/19 20:59:52
^C
root@kuber-master:~# kubectl logs -f fabric8-3192130638-0fue6 jenkinshift
2016-12-19 20:59:02.092742 I | Invoking Jenkins on URL http://jenkins/
2016-12-19 20:59:02.092919 I | About to query Kubernetes
2016-12-19 20:59:02.133939 I | Using namespace default
2016-12-19 20:59:02.134879 I | jenkinshift start listening on
localhost:9191
^C
root@kuber-master:~# kubectl logs -f gogs-1853259817-dxkd0
2016/12/19 20:58:12 Creating admin user: true
2016/12/19 20:58:12 [T] Custom path: /opt/gogs/custom
2016/12/19 20:58:12 [T] Log path: /app/gogs/data/logs
2016/12/19 20:58:12 [I] Gogs: Go Git Service
2016/12/19 20:58:15 Custom path: /opt/gogs/custom
2016/12/19 20:58:15 Admin account created: gogsadmin
2016/12/19 20:58:15 [T] Custom path: /opt/gogs/custom
2016/12/19 20:58:15 [T] Log path: /app/gogs/data/logs
2016/12/19 20:58:15 [I] Gogs: Go Git Service 0.9.97.0901
^C

I tried to use 0.6.0 and it didn't work

I1220 02:31:54.941108 1 main.go:37] Starting NGINX Ingress
controller Version 0.6.0
I1220 02:31:54.942405 1 nginx.go:251] Writing NGINX conf to
/etc/nginx/nginx.conf

user nginx;
worker_processes 1;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
worker_connections 1024;
}

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

log_format  main  '$remote_addr - $remote_user [$time_local]

"$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;

sendfile        on;
#tcp_nopush     on;

keepalive_timeout  65;

#gzip  on;

server_names_hash_max_size 512;


map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

include /etc/nginx/conf.d/*.conf;

}
I1220 02:31:54.943303 1 nginx.go:269] The main NGINX configuration
file had been updated
I1220 02:31:54.943508 1 nginx.go:224] executing nginx
F1220 02:31:54.959240 1 nginx.go:207] Failed to start nginx

spec:
containers:
- args:
- -v=3
- -nginx-configmaps=fabric8-system/nginx-config
image: nginxdemos/nginx-ingress:0.6.0
imagePullPolicy: IfNotPresent
name: nginx-ingress
ports:
- containerPort: 80
hostPort: 80
name: http
protocol: TCP
- containerPort: 443
hostPort: 443
name: https
protocol: TCP
resources: {}
securityContext:
privileged: true
terminationMessagePath: /dev/termination-log
dnsPolicy: ClusterFirst
nodeSelector:
fabric8.io/externalIP: "true"
restartPolicy: Always
securityContext: {}

Thanks,
best regards,

On Mon, Dec 19, 2016 at 4:32 PM, Michael Pleshakov <
[email protected]> wrote:

@abdoo https://github.com/abdoo

NGINX returns the bad gateway error when there is no healthy upstream
servers available. Please double check that the containers of the service
you are exposing via NGINX are successfully created and running properly.
Checking the controller logs also night help to determine the problem.

Additionally, I suggest to try to use the latest version of the
controller, which is nginxdemos/nginx-ingress:0.6.0

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-268084684,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhz30UqpXJ9KOxj737SnTr0qV6ksxXks5rJvfUgaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[pleshakov]

@abdoo
Could you also list Ingress resources?

[fattahmahran]

Hi Michael,

root@kuber-master:~# kubectl get ing
NAME HOSTS
ADDRESS PORTS AGE
fabric8 fabric8.default.10.103.12.20.nip.io
80 16h
fabric8-docker-registry
fabric8-docker-registry.default.10.103.12.20.nip.io 80
16h
fabric8-forge fabric8-forge.default.10.103.12.20.nip.io
80 16h
gogs gogs.default.10.103.12.20.nip.io
80 16h
gogs-ssh gogs-ssh.default.10.103.12.20.nip.io
80 16h
jenkins jenkins.default.10.103.12.20.nip.io
80 16h
jenkins-jnlp jenkins-jnlp.default.10.103.12.20.nip.io
80 16h
jenkinshift jenkinshift.default.10.103.12.20.nip.io
80 16h
nexus nexus.default.10.103.12.20.nip.io
80 16h
root@kuber-master:~#

Thanks,
best regards,

On Tue, Dec 20, 2016 at 12:16 PM, Michael Pleshakov <
[email protected]> wrote:

@abdoo https://github.com/abdoo
Could you also list Ingress resources?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-268301584,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhz76AqZwQbpO_pjhzAWrHFAlzx4qIks5rKA1rgaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[pleshakov]

@abdoo The problematic one is http://fabric8.default.10.103.12.20.nip.io ?

Could you do the following:

$ kubectl get -o yaml ing fabric8
$ kubectl describe svc fabric8

Could you also looks through NGINX logs and find the generated config related to the fabric8 ingress?

[fattahmahran]

Hi Michael,

actually all of them giving "502 bad gateway".

root@kuber-master:~# kubectl get -oyaml ing fabric8
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
creationTimestamp: 2016-12-20T02:55:38Z
generation: 1
labels:
provider: fabric8
name: fabric8
namespace: default
resourceVersion: "549512"
selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/fabric8
uid: c9b294b7-c65f-11e6-b3d9-0050560116bd
spec:
rules:

• host: fabric8.default.10.103.12.20.nip.io
  http:
  paths:
  
  
  
  • backend:
    
    serviceName: fabric8
    
    servicePort: 80
    
    status:
    
    loadBalancer: {}
    
    root@kuber-master:~#
  
  

root@kuber-master:~# kubectl describe svc fabric8
Name: fabric8
Namespace: default
Labels: expose=true
group=io.fabric8.apps
project=fabric8-console
provider=fabric8
version=2.2.199
Selector:
expose=true,group=io.fabric8.platform.console,project=fabric8-console,provider=fabric8
Type: ClusterIP
IP: 192.168.3.182
Port: 80/TCP
Endpoints: 172.16.87.3:9090
Session Affinity: None
No events.root@kuber-master:~#

Attached logs for ingress pod while starting.

Thanks,
best regards,

On Tue, Dec 20, 2016 at 3:12 PM, Michael Pleshakov <[email protected]

wrote:

@abdoo https://github.com/abdoo The problematic one is
http://fabric8.default.10.103.12.20.nip.io ?

Could you do the following:

$ kubectl get -o yaml ing fabric8
$ kubectl describe svc fabric8

Could you also looks through NGINX logs and find the generated config
related to the fabric8 ingress?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-268346204,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhz-8He6BnmSDsBeReNG1QeEdrwQzBks5rKDbLgaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

root@kuber-master:~# kubectl -n fabric8-system get podslogs -f ingress-nginx-2015555637-tayf7
I1220 20:19:43.107886 1 nginx.go:240] Writing NGINX conf to /etc/nginx/nginx.conf

user nginx;
worker_processes 1;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
worker_connections 1024;
}

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

access_log  /var/log/nginx/access.log  main;

sendfile        on;
#tcp_nopush     on;

keepalive_timeout  65;

#gzip  on;

server_names_hash_max_size 512;


map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

include /etc/nginx/conf.d/*.conf;

}
I1220 20:19:43.112387 1 nginx.go:258] The main NGINX configuration file had been updated
I1220 20:19:43.112593 1 nginx.go:213] executing nginx
I1220 20:19:43.257849 1 controller.go:143] Adding endpoints: kube-scheduler
I1220 20:19:43.257883 1 controller.go:143] Adding endpoints: jenkins
I1220 20:19:43.257894 1 controller.go:143] Adding endpoints: kube-controller-manager
I1220 20:19:43.257904 1 controller.go:143] Adding endpoints: kube-dns
I1220 20:19:43.257914 1 controller.go:143] Adding endpoints: kubernetes-dashboard
I1220 20:19:43.257927 1 controller.go:143] Adding endpoints: gogs-ssh
I1220 20:19:43.257936 1 controller.go:143] Adding endpoints: jenkins-jnlp
I1220 20:19:43.257948 1 controller.go:143] Adding endpoints: nexus
I1220 20:19:43.257972 1 controller.go:143] Adding endpoints: mongo
I1220 20:19:43.257984 1 controller.go:143] Adding endpoints: kubernetes
I1220 20:19:43.257993 1 controller.go:143] Adding endpoints: fabric8
I1220 20:19:43.258002 1 controller.go:143] Adding endpoints: fabric8-docker-registry
I1220 20:19:43.258011 1 controller.go:143] Adding endpoints: fabric8-forge
I1220 20:19:43.258036 1 controller.go:143] Adding endpoints: gogs
I1220 20:19:43.258049 1 controller.go:143] Adding endpoints: jenkinshift
I1220 20:19:43.258064 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:19:43.258077 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:19:43.258090 1 utils.go:70] Syncing default/jenkins
I1220 20:19:43.258099 1 controller.go:275] Syncing endpoints default/jenkins
I1220 20:19:43.258107 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:19:43.258113 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:19:43.258129 1 utils.go:70] Syncing kube-system/kube-dns
I1220 20:19:43.258138 1 controller.go:275] Syncing endpoints kube-system/kube-dns
I1220 20:19:43.258146 1 utils.go:70] Syncing kube-system/kubernetes-dashboard
I1220 20:19:43.258155 1 controller.go:275] Syncing endpoints kube-system/kubernetes-dashboard
I1220 20:19:43.258162 1 utils.go:70] Syncing default/gogs-ssh
I1220 20:19:43.258171 1 controller.go:275] Syncing endpoints default/gogs-ssh
I1220 20:19:43.258178 1 utils.go:70] Syncing default/jenkins-jnlp
I1220 20:19:43.258183 1 controller.go:275] Syncing endpoints default/jenkins-jnlp
I1220 20:19:43.258190 1 utils.go:70] Syncing default/nexus
I1220 20:19:43.258195 1 controller.go:275] Syncing endpoints default/nexus
I1220 20:19:43.258203 1 utils.go:70] Syncing default/mongo
I1220 20:19:43.258208 1 controller.go:275] Syncing endpoints default/mongo
I1220 20:19:43.258215 1 utils.go:70] Syncing default/kubernetes
I1220 20:19:43.258220 1 controller.go:275] Syncing endpoints default/kubernetes
I1220 20:19:43.258227 1 utils.go:70] Syncing default/fabric8
I1220 20:19:43.258232 1 controller.go:275] Syncing endpoints default/fabric8
I1220 20:19:43.258239 1 utils.go:70] Syncing default/fabric8-docker-registry
I1220 20:19:43.258244 1 controller.go:275] Syncing endpoints default/fabric8-docker-registry
I1220 20:19:43.258251 1 utils.go:70] Syncing default/fabric8-forge
I1220 20:19:43.258256 1 controller.go:275] Syncing endpoints default/fabric8-forge
I1220 20:19:43.258263 1 utils.go:70] Syncing default/gogs
I1220 20:19:43.258268 1 controller.go:275] Syncing endpoints default/gogs
I1220 20:19:43.258275 1 utils.go:70] Syncing default/jenkinshift
I1220 20:19:43.258280 1 controller.go:275] Syncing endpoints default/jenkinshift
I1220 20:19:43.263305 1 controller.go:117] Adding service: kubernetes
I1220 20:19:43.263369 1 controller.go:368] ignoring service kubernetes: No ingress for service kubernetes
I1220 20:19:43.263402 1 controller.go:117] Adding service: kube-dns
I1220 20:19:43.263410 1 controller.go:368] ignoring service kube-dns: No ingress for service kube-dns
I1220 20:19:43.263420 1 controller.go:117] Adding service: fabric8
I1220 20:19:43.263426 1 controller.go:368] ignoring service fabric8: No ingress for service fabric8
I1220 20:19:43.263440 1 controller.go:117] Adding service: gogs
I1220 20:19:43.263446 1 controller.go:368] ignoring service gogs: No ingress for service gogs
I1220 20:19:43.263458 1 controller.go:117] Adding service: jenkins-jnlp
I1220 20:19:43.263465 1 controller.go:368] ignoring service jenkins-jnlp: No ingress for service jenkins-jnlp
I1220 20:19:43.263474 1 controller.go:117] Adding service: kubernetes-dashboard
I1220 20:19:43.263480 1 controller.go:368] ignoring service kubernetes-dashboard: No ingress for service kubernetes-dashboard
I1220 20:19:43.263497 1 controller.go:117] Adding service: fabric8-docker-registry
I1220 20:19:43.263508 1 controller.go:368] ignoring service fabric8-docker-registry: No ingress for service fabric8-docker-registry
I1220 20:19:43.263517 1 controller.go:117] Adding service: fabric8-forge
I1220 20:19:43.263523 1 controller.go:368] ignoring service fabric8-forge: No ingress for service fabric8-forge
I1220 20:19:43.263545 1 controller.go:117] Adding service: gogs-ssh
I1220 20:19:43.263555 1 controller.go:368] ignoring service gogs-ssh: No ingress for service gogs-ssh
I1220 20:19:43.263570 1 controller.go:117] Adding service: jenkins
I1220 20:19:43.263586 1 controller.go:368] ignoring service jenkins: No ingress for service jenkins
I1220 20:19:43.263596 1 controller.go:117] Adding service: jenkinshift
I1220 20:19:43.263602 1 controller.go:368] ignoring service jenkinshift: No ingress for service jenkinshift
I1220 20:19:43.263611 1 controller.go:117] Adding service: nexus
I1220 20:19:43.263617 1 controller.go:368] ignoring service nexus: No ingress for service nexus
I1220 20:19:43.263626 1 controller.go:117] Adding service: mongo
I1220 20:19:43.263632 1 controller.go:368] ignoring service mongo: No ingress for service mongo
I1220 20:19:43.264525 1 controller.go:178] Adding ConfigMap: nginx-config
I1220 20:19:43.264555 1 utils.go:70] Syncing fabric8-system/nginx-config
I1220 20:19:43.264562 1 controller.go:297] Syncing configmap fabric8-system/nginx-config
I1220 20:19:43.264891 1 nginx.go:240] Writing NGINX conf to /etc/nginx/nginx.conf

user nginx;
worker_processes 1;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
worker_connections 1024;
}

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

access_log  /var/log/nginx/access.log  main;

sendfile        on;
#tcp_nopush     on;

keepalive_timeout  65;

#gzip  on;

server_names_hash_max_size 1024;
server_names_hash_bucket_size 256;

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

include /etc/nginx/conf.d/*.conf;

}
I1220 20:19:43.265559 1 nginx.go:258] The main NGINX configuration file had been updated
I1220 20:19:43.268010 1 controller.go:85] Adding Ingress: jenkins
I1220 20:19:43.268038 1 controller.go:85] Adding Ingress: fabric8-docker-registry
I1220 20:19:43.268049 1 controller.go:85] Adding Ingress: jenkins-jnlp
I1220 20:19:43.268060 1 controller.go:85] Adding Ingress: gogs
I1220 20:19:43.268072 1 controller.go:85] Adding Ingress: fabric8
I1220 20:19:43.268083 1 controller.go:85] Adding Ingress: fabric8-forge
I1220 20:19:43.268092 1 controller.go:85] Adding Ingress: gogs-ssh
I1220 20:19:43.268101 1 controller.go:85] Adding Ingress: jenkinshift
I1220 20:19:43.268115 1 controller.go:85] Adding Ingress: nexus
I1220 20:19:43.268128 1 utils.go:70] Syncing default/jenkins
I1220 20:19:43.268138 1 controller.go:334] Syncing default/jenkins
I1220 20:19:43.268146 1 controller.go:349] Adding or Updating Ingress: default/jenkins
I1220 20:19:43.268286 1 nginx.go:108] Updating NGINX configuration
I1220 20:19:43.268860 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-jenkins.conf

upstream default-jenkins-jenkins.default.10.103.12.20.nip.io-jenkins {

server 172.16.97.5:8080;
}

server {
listen 80;

server_name jenkins.default.10.103.12.20.nip.io;

location / {
proxy_http_version 1.1;

proxy_connect_timeout 500s;
proxy_read_timeout 500s;
client_max_body_size 2000m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://default-jenkins-jenkins.default.10.103.12.20.nip.io-jenkins;
}
}
I1220 20:19:43.281185 1 nginx.go:174] NGINX configuration file had been updated
I1220 20:19:43.281221 1 nginx.go:213] executing nginx -t
I1220 20:19:43.301186 1 nginx.go:213] executing nginx -s reload
2016/12/20 20:19:43 [notice] 18#18: signal process started
I1220 20:19:43.317017 1 utils.go:70] Syncing default/fabric8-docker-registry
I1220 20:19:43.317071 1 controller.go:334] Syncing default/fabric8-docker-registry
I1220 20:19:43.317093 1 controller.go:349] Adding or Updating Ingress: default/fabric8-docker-registry
I1220 20:19:43.317260 1 nginx.go:108] Updating NGINX configuration
I1220 20:19:43.317999 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-fabric8-docker-registry.conf

upstream default-fabric8-docker-registry-fabric8-docker-registry.default.10.103.12.20.nip.io-fabric8-docker-registry {

server 172.16.49.4:5000;
}

server {
listen 80;

server_name fabric8-docker-registry.default.10.103.12.20.nip.io;

location / {
proxy_http_version 1.1;

proxy_connect_timeout 500s;
proxy_read_timeout 500s;
client_max_body_size 2000m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://default-fabric8-docker-registry-fabric8-docker-registry.default.10.103.12.20.nip.io-fabric8-docker-registry;
}
}
I1220 20:19:43.320623 1 nginx.go:174] NGINX configuration file had been updated
I1220 20:19:43.320651 1 nginx.go:213] executing nginx -t
I1220 20:19:43.343128 1 nginx.go:213] executing nginx -s reload
2016/12/20 20:19:43 [notice] 23#23: signal process started
I1220 20:19:43.354240 1 utils.go:70] Syncing default/jenkins-jnlp
I1220 20:19:43.354379 1 controller.go:334] Syncing default/jenkins-jnlp
I1220 20:19:43.354481 1 controller.go:349] Adding or Updating Ingress: default/jenkins-jnlp
I1220 20:19:43.354781 1 nginx.go:108] Updating NGINX configuration
I1220 20:19:43.355683 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-jenkins-jnlp.conf

upstream default-jenkins-jnlp-jenkins-jnlp.default.10.103.12.20.nip.io-jenkins-jnlp {

server 172.16.97.5:50000;
}

server {
listen 80;

server_name jenkins-jnlp.default.10.103.12.20.nip.io;

location / {
proxy_http_version 1.1;

proxy_connect_timeout 500s;
proxy_read_timeout 500s;
client_max_body_size 2000m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://default-jenkins-jnlp-jenkins-jnlp.default.10.103.12.20.nip.io-jenkins-jnlp;
}
}
I1220 20:19:43.358216 1 nginx.go:174] NGINX configuration file had been updated
I1220 20:19:43.358343 1 nginx.go:213] executing nginx -t
I1220 20:19:43.374987 1 nginx.go:213] executing nginx -s reload
2016/12/20 20:19:43 [notice] 27#27: signal process started

upstream default-gogs-gogs.default.10.103.12.20.nip.io-gogs {

server 172.16.49.6:3000;
}

server {
listen 80;

server_name gogs.default.10.103.12.20.nip.io;

location / {
proxy_http_version 1.1;

proxy_connect_timeout 500s;
proxy_read_timeout 500s;
client_max_body_size 2000m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://default-gogs-gogs.default.10.103.12.20.nip.io-gogs;
}
}
I1220 20:19:43.388641 1 utils.go:70] Syncing default/gogs
I1220 20:19:43.388686 1 controller.go:334] Syncing default/gogs
I1220 20:19:43.388703 1 controller.go:349] Adding or Updating Ingress: default/gogs
I1220 20:19:43.388835 1 nginx.go:108] Updating NGINX configuration
I1220 20:19:43.389547 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-gogs.conf
I1220 20:19:43.390482 1 nginx.go:174] NGINX configuration file had been updated
I1220 20:19:43.390515 1 nginx.go:213] executing nginx -t
I1220 20:19:43.412396 1 nginx.go:213] executing nginx -s reload
I1220 20:19:43.414080 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:19:43.414227 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:19:43.414244 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:19:43.432724 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:19:43.432778 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:19:43.434289 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
2016/12/20 20:19:43 [notice] 31#31: signal process started
I1220 20:19:43.452114 1 utils.go:70] Syncing default/fabric8
I1220 20:19:43.452241 1 controller.go:334] Syncing default/fabric8
I1220 20:19:43.452310 1 controller.go:349] Adding or Updating Ingress: default/fabric8
I1220 20:19:43.452487 1 nginx.go:108] Updating NGINX configuration
I1220 20:19:43.453202 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-fabric8.conf

upstream default-fabric8-fabric8.default.10.103.12.20.nip.io-fabric8 {

server 172.16.87.3:9090;
}

server {
listen 80;

server_name fabric8.default.10.103.12.20.nip.io;

location / {
proxy_http_version 1.1;

proxy_connect_timeout 500s;
proxy_read_timeout 500s;
client_max_body_size 2000m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://default-fabric8-fabric8.default.10.103.12.20.nip.io-fabric8;
}
}
I1220 20:19:43.454848 1 nginx.go:174] NGINX configuration file had been updated
I1220 20:19:43.454940 1 nginx.go:213] executing nginx -t
I1220 20:19:43.476880 1 nginx.go:213] executing nginx -s reload
2016/12/20 20:19:43 [notice] 36#36: signal process started
I1220 20:19:43.502286 1 utils.go:70] Syncing default/fabric8-forge
I1220 20:19:43.502448 1 controller.go:334] Syncing default/fabric8-forge
I1220 20:19:43.502532 1 controller.go:349] Adding or Updating Ingress: default/fabric8-forge
I1220 20:19:43.502791 1 nginx.go:108] Updating NGINX configuration
I1220 20:19:43.503600 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-fabric8-forge.conf

upstream default-fabric8-forge-fabric8-forge.default.10.103.12.20.nip.io-fabric8-forge {

server 172.16.97.4:8080;
}

server {
listen 80;

server_name fabric8-forge.default.10.103.12.20.nip.io;

location / {
proxy_http_version 1.1;

proxy_connect_timeout 500s;
proxy_read_timeout 500s;
client_max_body_size 2000m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://default-fabric8-forge-fabric8-forge.default.10.103.12.20.nip.io-fabric8-forge;
}
}
I1220 20:19:43.505642 1 nginx.go:174] NGINX configuration file had been updated
I1220 20:19:43.505764 1 nginx.go:213] executing nginx -t
I1220 20:19:43.524679 1 nginx.go:213] executing nginx -s reload
2016/12/20 20:19:43 [notice] 40#40: signal process started
I1220 20:19:43.544801 1 utils.go:70] Syncing default/gogs-ssh
I1220 20:19:43.544848 1 controller.go:334] Syncing default/gogs-ssh
I1220 20:19:43.544899 1 controller.go:349] Adding or Updating Ingress: default/gogs-ssh
I1220 20:19:43.545085 1 nginx.go:108] Updating NGINX configuration
I1220 20:19:43.546166 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-gogs-ssh.conf

upstream default-gogs-ssh-gogs-ssh.default.10.103.12.20.nip.io-gogs-ssh {

server 172.16.49.6:2222;
}

server {
listen 80;

server_name gogs-ssh.default.10.103.12.20.nip.io;

location / {
proxy_http_version 1.1;

proxy_connect_timeout 500s;
proxy_read_timeout 500s;
client_max_body_size 2000m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://default-gogs-ssh-gogs-ssh.default.10.103.12.20.nip.io-gogs-ssh;
}
}
I1220 20:19:43.548603 1 nginx.go:174] NGINX configuration file had been updated
I1220 20:19:43.548632 1 nginx.go:213] executing nginx -t
I1220 20:19:43.568165 1 nginx.go:213] executing nginx -s reload
2016/12/20 20:19:43 [notice] 45#45: signal process started
I1220 20:19:43.584852 1 utils.go:70] Syncing default/jenkinshift
I1220 20:19:43.584939 1 controller.go:334] Syncing default/jenkinshift
I1220 20:19:43.585024 1 controller.go:349] Adding or Updating Ingress: default/jenkinshift
I1220 20:19:43.585142 1 nginx.go:108] Updating NGINX configuration
I1220 20:19:43.585842 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-jenkinshift.conf

upstream default-jenkinshift-jenkinshift.default.10.103.12.20.nip.io-jenkinshift {

server 172.16.87.3:9191;
}

server {
listen 80;

server_name jenkinshift.default.10.103.12.20.nip.io;

location / {
proxy_http_version 1.1;

proxy_connect_timeout 500s;
proxy_read_timeout 500s;
client_max_body_size 2000m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://default-jenkinshift-jenkinshift.default.10.103.12.20.nip.io-jenkinshift;
}
}
I1220 20:19:43.590415 1 nginx.go:174] NGINX configuration file had been updated
I1220 20:19:43.590463 1 nginx.go:213] executing nginx -t
I1220 20:19:43.607452 1 nginx.go:213] executing nginx -s reload
2016/12/20 20:19:43 [notice] 49#49: signal process started
I1220 20:19:43.620424 1 utils.go:70] Syncing default/nexus
I1220 20:19:43.620455 1 controller.go:334] Syncing default/nexus
I1220 20:19:43.620465 1 controller.go:349] Adding or Updating Ingress: default/nexus
I1220 20:19:43.620514 1 nginx.go:108] Updating NGINX configuration
I1220 20:19:43.620824 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-nexus.conf

upstream default-nexus-nexus.default.10.103.12.20.nip.io-nexus {

server 172.16.49.2:8081;
}

server {
listen 80;

server_name nexus.default.10.103.12.20.nip.io;

location / {
proxy_http_version 1.1;

proxy_connect_timeout 500s;
proxy_read_timeout 500s;
client_max_body_size 2000m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://default-nexus-nexus.default.10.103.12.20.nip.io-nexus;
}
}
I1220 20:19:43.621904 1 nginx.go:174] NGINX configuration file had been updated
I1220 20:19:43.621933 1 nginx.go:213] executing nginx -t
I1220 20:19:43.631064 1 nginx.go:213] executing nginx -s reload
2016/12/20 20:19:43 [notice] 53#53: signal process started
I1220 20:19:45.416818 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:19:45.416876 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:19:45.416888 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:19:45.440454 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:19:45.440494 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:19:45.440501 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:19:47.424786 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:19:47.424831 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:19:47.424841 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:19:47.446089 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:19:47.446132 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:19:47.446142 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:19:49.433960 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:19:49.434015 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:19:49.434027 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:19:49.457482 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:19:49.457526 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:19:49.457537 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:19:51.454488 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:19:51.454604 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:19:51.454617 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:19:51.471905 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:19:51.471951 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:19:51.471966 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:19:53.471170 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:19:53.471224 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:19:53.471235 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:19:53.485512 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:19:53.485548 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:19:53.485558 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:19:55.477402 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:19:55.477442 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:19:55.477472 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:19:55.499894 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:19:55.499964 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:19:55.499976 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:19:57.487559 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:19:57.487624 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:19:57.487635 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:19:57.508453 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:19:57.508537 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:19:57.508549 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:19:59.492359 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:19:59.492400 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:19:59.492410 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:19:59.514079 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:19:59.514119 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:19:59.514128 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:20:01.504817 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:20:01.504878 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:20:01.504891 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:20:01.521787 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:20:01.521831 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:20:01.521844 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:20:03.511644 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:20:03.511694 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:20:03.511704 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:20:03.529068 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:20:03.529142 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:20:03.529153 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:20:05.519338 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:20:05.519401 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:20:05.519414 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:20:05.536118 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:20:05.536154 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:20:05.536199 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:20:07.527061 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:20:07.527111 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:20:07.527157 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:20:07.545302 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:20:07.545371 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:20:07.545381 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:20:09.537279 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:20:09.537772 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:20:09.537857 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:20:09.555983 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:20:09.556044 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:20:09.556055 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:20:12.283176 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:20:12.283603 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:20:12.284866 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:20:12.285009 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:20:12.285296 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:20:12.286194 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:20:13.555725 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:20:13.555807 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:20:13.555825 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:20:13.568339 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:20:13.568412 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:20:13.568427 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:20:15.562319 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:20:15.562356 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:20:15.562365 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:20:15.578309 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:20:15.578370 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:20:15.578380 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager
I1220 20:20:17.573304 1 controller.go:154] Endpoints kube-scheduler changed, syncing
I1220 20:20:17.573341 1 utils.go:70] Syncing kube-system/kube-scheduler
I1220 20:20:17.573349 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler
I1220 20:20:17.588338 1 controller.go:154] Endpoints kube-controller-manager changed, syncing
I1220 20:20:17.588404 1 utils.go:70] Syncing kube-system/kube-controller-manager
I1220 20:20:17.588414 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager

[pleshakov]

@abdoo The generated configuration looks good.

When you get the 502 error, could you past the related log entry from the NGINX logs?
Could you connect to the pod, install curl and try to connect to any of the endpoints?

$ kubectl exec <nginx-pod> -it -- bash
# apt-get update -y
# apt-get install -y curl
# curl http://172.16.87.3:9090

[fattahmahran]

@Michael:

I couldn't connect with bash

root@kuber-master:~# kubectl -n fabric8-system get pods
NAME READY STATUS RESTARTS AGE
ingress-nginx-2015555637-tayf7 1/1 Running 0 16m
root@kuber-master:~# kubectl -n fabric8-system exec
ingress-nginx-2015555637-tayf7 -it -- bash
error: unable to upgrade connection: HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">

ERROR

The requested URL could not be retrieved

---

The following error was encountered while trying to retrieve the URL: /api/v1/namespaces/fabric8-system/pods/ingress-nginx-2015555637-tayf7/exec?command=bash&container=nginx-ingress&container=nginx-ingress&stdin=true&stdout=true&tty=true

Invalid URL

Some aspect of the requested URL is incorrect.

Some possible problems are:

• Missing or incorrect access protocol (should be http:// or similar)

• Missing hostname

• Illegal double-escape in the URL-Path

• Illegal character in hostname; underscores are not allowed.

Your cache administrator is root.

---

Generated Tue, 20 Dec 2016 20:39:41 GMT by www-proxy.naladmz.ericsson.ca (squid/3.1.16)

root@kuber-master:~#

Thanks,
best regards,

On Tue, Dec 20, 2016 at 3:33 PM, Michael Pleshakov <[email protected]

wrote:

@abdoo https://github.com/abdoo The generated configuration looks good.

When you get the 502 error, could you past the related log entry from the
NGINX logs?
Could you connect to the pod, install curl and try to connect to any of
the endpoints?

$ kubectl exec -it -- bash

apt-get update -y

apt-get install -y curl

curl http://172.16.87.3:9090

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-268351091,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhz867_k1uzFZtKKvDFTAgzxfUHK5vks5rKDuVgaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[pleshakov]

@abdoo Could you setup kubectl to bypass the squid proxy when connecting to the API server?

[fattahmahran]

I don't understand what do you mean by squid proxy? you mean company's
proxy? If this is the case so I can't do that because we will not be able
to use Internets for downloading images..etc

Thanks,
best regards,

On Tue, Dec 20, 2016 at 3:57 PM, Michael Pleshakov <[email protected]

wrote:

@abdoo https://github.com/abdoo Could you setup kubectl to bypass the
squid proxy when connecting to the API server?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-268357718,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhz-lexGoJDubfIsmhtJsiQ55Lo1lDks5rKEFFgaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[pleshakov]

@abdoo
In your case, kubectl using a proxy to connect to the API server, which resides on the same host from my understanding, so using a proxy for that isn't necessary. Could make it bypass the proxy temporary?

Also, could you try to run the commands like below fist and see if they will work:

$ kubectl exec <nginx-pod> -- apt-get update -y
...

[fattahmahran]

Hi Michael,

this command didnt work as well. we need the proxy during installation of
kubernetes to be able to connect to Internet. If there is a way to stop
using it now it would be great. If you can share how to make kubernetes
skip it.

Thanks,
best regards,

On Tue, Dec 20, 2016 at 4:31 PM, Michael Pleshakov <[email protected]

wrote:

@abdoo https://github.com/abdoo
In your case, kubectl using a proxy to connect to the API server, which
resides on the same host from my understanding, so using a proxy for that
isn't necessary. Could make it bypass the proxy temporary?

Also, could you try to run the commands like below fist and see if they
will work:

$ kubectl exec -- apt-get update -y
...

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nginxinc/kubernetes-ingress/issues/101#issuecomment-268365106,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMhzzLgYfY4Bue649b95RmeKOh_qIrpks5rKEkggaJpZM4LPvNH
.

--
Thanks
Best Regards,
Abd El-Fattah Mahran
http://www.linkedin.com/in/abdoomahran

[pleshakov]

@abdoo
Could you try to connect to any of the servers like http://fabric8.default.10.103.12.20.nip.io and then find the error in the NGINX logs? It will tell why NGINX couldn't connect to the endpoint.

This might work to bypass the proxy: $ HTTPS_PROXY='' kubectl exec ...