Weave: DENY all traffic from other namespaces

Created on 17 Jan 2018 · 11Comments · Source: weaveworks/weave

What you expected to happen?

deny all the traffic from other namespaces while allowing all the traffic coming from the same namespace

What happened?

Traffic allowed from all namespaces

How to reproduce it?

Create a new namespace called ns-1 and start a web service:

kubectl create namespace secondary

kubectl run web --namespace ns-1--image=nginx \
    --labels=app=web --expose --port 80

Save the following manifest to web-deny-other-namespaces.yaml and apply to the cluster:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  namespace: ns-1
  name: deny-from-other-namespaces
spec:
  podSelector:
    matchLabels:
  ingress:
  - from:
    - podSelector: {}

$ kubectl apply -f web-deny-from-other-namespaces.yaml
networkpolicy "deny-from-other-namespaces" created"

Query the web service from the default namespace:

$ kubectl run test-$RANDOM --namespace=default --rm -i -t --image=alpine -- sh
/ # wget -qO- --timeout=2 http://web.ns-1
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

So, It doesn't block the traffic from default namespace!.... but it should blocks it

[componenpolicy] bug

Source

raniatriki

Most helpful comment

@raniatriki Yep, thanks. Investigating.

brb on 17 Jan 2018

👍2

All 11 comments

@raniatriki Can you please provide the following info:

Kubernetes version
Weave version
iptables-save output
kubectl logs $WEAVE-POD -n kube-system weave-npc output, where WEAVE-POD is any pod name from kubectl get pods -n kube-system -l name=weave-net

brb on 17 Jan 2018

- Kubernetes version

kubectl version
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.2", GitCommit:"922a86cfcd65915a9b2f69f3f193b8907d741d9c", GitTreeState:"clean", BuildDate:"2017-07-21T08:23:22Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.1", GitCommit:"3a1c9449a956b6026f075fa3134ff92f7d55f812", GitTreeState:"clean", BuildDate:"2018-01-04T11:40:06Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}

- Weave version

weave status

        Version: 2.1.3 (up to date; next check at 2018/01/17 19:16:05)

        Service: router
       Protocol: weave 1..2
           Name: 8e:8e:7b:f5:4d:74(k8s-1)
     Encryption: disabled
  PeerDiscovery: enabled
        Targets: 1
    Connections: 3 (2 established, 1 failed)
          Peers: 3 (with 6 established connections)
 TrustedSubnets: none

        Service: ipam
         Status: ready
          Range: 10.32.0.0/12
  DefaultSubnet: 10.32.0.0/12

- iptables-save output

 iptables-save
# Generated by iptables-save v1.6.0 on Wed Jan 17 12:13:11 2018
*nat
:PREROUTING ACCEPT [25:2878]
:INPUT ACCEPT [10:1464]
:OUTPUT ACCEPT [17:2742]
:POSTROUTING ACCEPT [27:3706]
:DOCKER - [0:0]
:KUBE-MARK-DROP - [0:0]
:KUBE-MARK-MASQ - [0:0]
:KUBE-NODEPORTS - [0:0]
:KUBE-POSTROUTING - [0:0]
:KUBE-SEP-3DU66DE6VORVEQVD - [0:0]
:KUBE-SEP-4SZD3IGGQH7OS76W - [0:0]
:KUBE-SEP-7ZCF6XYU2MJUS3G3 - [0:0]
:KUBE-SEP-BHB36F4FNBM77ZNX - [0:0]
:KUBE-SEP-OPBRHBWOV23JTSFZ - [0:0]
:KUBE-SEP-QFC4IGGGYTWWGIXI - [0:0]
:KUBE-SEP-RWLNQJOMCEVSLMGE - [0:0]
:KUBE-SEP-S4MK5EVI7CLHCCS6 - [0:0]
:KUBE-SEP-UJ4S36ZD6CZMUCVN - [0:0]
:KUBE-SEP-UKQ4V7Y7PU5CFTC4 - [0:0]
:KUBE-SERVICES - [0:0]
:KUBE-SVC-4N57TFCL4MD7ZTDA - [0:0]
:KUBE-SVC-4UOFSNIU3RMN7P7P - [0:0]
:KUBE-SVC-BJM46V3U5RZHCFRZ - [0:0]
:KUBE-SVC-ERIFXISQEP7F7OF4 - [0:0]
:KUBE-SVC-NPX46M4PTMTKRN6Y - [0:0]
:KUBE-SVC-TCOU7JCQXEZGVUNU - [0:0]
:KUBE-SVC-XGLOHA7QRQ3V22RZ - [0:0]
:WEAVE - [0:0]
-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -j WEAVE
-A DOCKER -i docker0 -j RETURN
-A KUBE-MARK-DROP -j MARK --set-xmark 0x8000/0x8000
-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -m mark --mark 0x4000/0x4000 -j MASQUERADE
-A KUBE-SEP-3DU66DE6VORVEQVD -s 10.32.0.3/32 -m comment --comment "kube-system/kube-dns:dns" -j KUBE-MARK-MASQ
-A KUBE-SEP-3DU66DE6VORVEQVD -p udp -m comment --comment "kube-system/kube-dns:dns" -m udp -j DNAT --to-destination 10.32.0.3:53
-A KUBE-SEP-4SZD3IGGQH7OS76W -s 172.16.10.19/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ
-A KUBE-SEP-4SZD3IGGQH7OS76W -p tcp -m comment --comment "default/kubernetes:https" -m recent --set --name KUBE-SEP-4SZD3IGGQH7OS76W --mask 255.255.255.255 --rsource -m tcp -j DNAT --to-destination 172.16.10.19:6443
-A KUBE-SEP-7ZCF6XYU2MJUS3G3 -s 10.40.0.5/32 -m comment --comment "kube-system/kubernetes-dashboard:" -j KUBE-MARK-MASQ
-A KUBE-SEP-7ZCF6XYU2MJUS3G3 -p tcp -m comment --comment "kube-system/kubernetes-dashboard:" -m tcp -j DNAT --to-destination 10.40.0.5:8443
-A KUBE-SEP-BHB36F4FNBM77ZNX -s 10.46.0.1/32 -m comment --comment "default/nginx:" -j KUBE-MARK-MASQ
-A KUBE-SEP-BHB36F4FNBM77ZNX -p tcp -m comment --comment "default/nginx:" -m tcp -j DNAT --to-destination 10.46.0.1:80
-A KUBE-SEP-OPBRHBWOV23JTSFZ -s 10.40.0.2/32 -m comment --comment "default/who-is-there:" -j KUBE-MARK-MASQ
-A KUBE-SEP-OPBRHBWOV23JTSFZ -p tcp -m comment --comment "default/who-is-there:" -m tcp -j DNAT --to-destination 10.40.0.2:8080
-A KUBE-SEP-QFC4IGGGYTWWGIXI -s 10.40.0.0/32 -m comment --comment "default/who-is-there:" -j KUBE-MARK-MASQ
-A KUBE-SEP-QFC4IGGGYTWWGIXI -p tcp -m comment --comment "default/who-is-there:" -m tcp -j DNAT --to-destination 10.40.0.0:8080
-A KUBE-SEP-RWLNQJOMCEVSLMGE -s 10.46.0.2/32 -m comment --comment "kube-system/heapster:" -j KUBE-MARK-MASQ
-A KUBE-SEP-RWLNQJOMCEVSLMGE -p tcp -m comment --comment "kube-system/heapster:" -m tcp -j DNAT --to-destination 10.46.0.2:8082
-A KUBE-SEP-S4MK5EVI7CLHCCS6 -s 10.32.0.3/32 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-MARK-MASQ
-A KUBE-SEP-S4MK5EVI7CLHCCS6 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp" -m tcp -j DNAT --to-destination 10.32.0.3:53
-A KUBE-SEP-UJ4S36ZD6CZMUCVN -s 10.40.0.1/32 -m comment --comment "default/nginx:" -j KUBE-MARK-MASQ
-A KUBE-SEP-UJ4S36ZD6CZMUCVN -p tcp -m comment --comment "default/nginx:" -m tcp -j DNAT --to-destination 10.40.0.1:80
-A KUBE-SEP-UKQ4V7Y7PU5CFTC4 -s 10.46.0.0/32 -m comment --comment "default/who-is-there:" -j KUBE-MARK-MASQ
-A KUBE-SEP-UKQ4V7Y7PU5CFTC4 -p tcp -m comment --comment "default/who-is-there:" -m tcp -j DNAT --to-destination 10.46.0.0:8080
-A KUBE-SERVICES -d 10.96.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
-A KUBE-SERVICES -d 10.96.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-SVC-ERIFXISQEP7F7OF4
-A KUBE-SERVICES -d 10.107.182.147/32 -p tcp -m comment --comment "kube-system/kubernetes-dashboard: cluster IP" -m tcp --dport 443 -j KUBE-SVC-XGLOHA7QRQ3V22RZ
-A KUBE-SERVICES -d 10.96.219.228/32 -p tcp -m comment --comment "kube-system/heapster: cluster IP" -m tcp --dport 80 -j KUBE-SVC-BJM46V3U5RZHCFRZ
-A KUBE-SERVICES -d 10.98.132.43/32 -p tcp -m comment --comment "default/nginx: cluster IP" -m tcp --dport 80 -j KUBE-SVC-4N57TFCL4MD7ZTDA
-A KUBE-SERVICES -d 10.96.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-SVC-TCOU7JCQXEZGVUNU
-A KUBE-SERVICES -d 10.101.203.15/32 -p tcp -m comment --comment "default/who-is-there: cluster IP" -m tcp --dport 8080 -j KUBE-SVC-4UOFSNIU3RMN7P7P
-A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS
-A KUBE-SVC-4N57TFCL4MD7ZTDA -m comment --comment "default/nginx:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-UJ4S36ZD6CZMUCVN
-A KUBE-SVC-4N57TFCL4MD7ZTDA -m comment --comment "default/nginx:" -j KUBE-SEP-BHB36F4FNBM77ZNX
-A KUBE-SVC-4UOFSNIU3RMN7P7P -m comment --comment "default/who-is-there:" -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-QFC4IGGGYTWWGIXI
-A KUBE-SVC-4UOFSNIU3RMN7P7P -m comment --comment "default/who-is-there:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-OPBRHBWOV23JTSFZ
-A KUBE-SVC-4UOFSNIU3RMN7P7P -m comment --comment "default/who-is-there:" -j KUBE-SEP-UKQ4V7Y7PU5CFTC4
-A KUBE-SVC-BJM46V3U5RZHCFRZ -m comment --comment "kube-system/heapster:" -j KUBE-SEP-RWLNQJOMCEVSLMGE
-A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-SEP-S4MK5EVI7CLHCCS6
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -m recent --rcheck --seconds 10800 --reap --name KUBE-SEP-4SZD3IGGQH7OS76W --mask 255.255.255.255 --rsource -j KUBE-SEP-4SZD3IGGQH7OS76W
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -j KUBE-SEP-4SZD3IGGQH7OS76W
-A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment "kube-system/kube-dns:dns" -j KUBE-SEP-3DU66DE6VORVEQVD
-A KUBE-SVC-XGLOHA7QRQ3V22RZ -m comment --comment "kube-system/kubernetes-dashboard:" -j KUBE-SEP-7ZCF6XYU2MJUS3G3
-A WEAVE -s 10.32.0.0/12 -d 224.0.0.0/4 -j RETURN
-A WEAVE ! -s 10.32.0.0/12 -d 10.32.0.0/12 -j MASQUERADE
-A WEAVE -s 10.32.0.0/12 ! -d 10.32.0.0/12 -j MASQUERADE
COMMIT
# Completed on Wed Jan 17 12:13:11 2018
# Generated by iptables-save v1.6.0 on Wed Jan 17 12:13:11 2018
*filter
:INPUT ACCEPT [719:152634]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [730:527537]
:DOCKER - [0:0]
:DOCKER-ISOLATION - [0:0]
:DOCKER-USER - [0:0]
:KUBE-FIREWALL - [0:0]
:KUBE-FORWARD - [0:0]
:KUBE-SERVICES - [0:0]
:WEAVE-NPC - [0:0]
:WEAVE-NPC-DEFAULT - [0:0]
:WEAVE-NPC-INGRESS - [0:0]
-A INPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A INPUT -j KUBE-FIREWALL
-A FORWARD -m comment --comment "kubernetes forward rules" -j KUBE-FORWARD
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o weave -j WEAVE-NPC
-A FORWARD -o weave -m state --state NEW -j NFLOG --nflog-group 86
-A FORWARD -o weave -j DROP
-A FORWARD -i weave ! -o weave -j ACCEPT
-A FORWARD -o weave -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT -j KUBE-FIREWALL
-A DOCKER-ISOLATION -j RETURN
-A DOCKER-USER -j RETURN
-A KUBE-FIREWALL -m comment --comment "kubernetes firewall for dropping marked packets" -m mark --mark 0x8000/0x8000 -j DROP
-A KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -m mark --mark 0x4000/0x4000 -j ACCEPT
-A WEAVE-NPC -m state --state RELATED,ESTABLISHED -j ACCEPT
-A WEAVE-NPC -d 224.0.0.0/4 -j ACCEPT
-A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-DEFAULT
-A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-INGRESS
-A WEAVE-NPC -m set ! --match-set weave-local-pods dst -j ACCEPT
-A WEAVE-NPC-DEFAULT -m set --match-set weave-?b%zl9GIe0AET1(QI^7NWe*fO dst -m comment --comment "DefaultAllow isolation for namespace: kube-system" -j ACCEPT
-A WEAVE-NPC-DEFAULT -m set --match-set weave-0EHD/vdN#O4]V?o4Tx7kS;APH dst -m comment --comment "DefaultAllow isolation for namespace: kube-public" -j ACCEPT
-A WEAVE-NPC-DEFAULT -m set --match-set weave-E.1.0W^NGSp]0_t5WwH/]gX@L dst -m comment --comment "DefaultAllow isolation for namespace: default" -j ACCEPT
COMMIT
# Completed on Wed Jan 17 12:13:11 2018

raniatriki on 17 Jan 2018

Thanks. But I'm still missing the weave-npc logs.

brb on 17 Jan 2018

kubectl logs weave-net-4s7vw -n kube-system weave-npc
INFO: 2018/01/17 12:21:03.314572 Starting Weaveworks NPC 2.1.3; node name "k8s-1"
INFO: 2018/01/17 12:21:03.364846 Serving /metrics on :6781
Wed Jan 17 12:21:03 2018 <5> ulogd.c:843 building new pluginstance stack: 'log1:NFLOG,base1:BASE,pcap1:PCAP'
DEBU: 2018/01/17 12:21:03.386215 Got list of ipsets: [weave-local-pods weave-?b%zl9GIe0AET1(QI^7NWe*fO weave-iuZcey(5DeXbzgRFs8Szo]+@p weave-0EHD/vdN#O4]V?o4Tx7kS;APH weave-4vtqMI+kx/2]jD%_c0S%thO%V weave-z4cN+%s?50k!Z)QvX;[=C%d$M weave-H?Wgfw.#Ut^=wxajFiTHd@9AF weave-E.1.0W^NGSp]0_t5WwH/]gX@L weave-k?Z;25^M}|1s7P3|H9i;*;MhG]
DEBU: 2018/01/17 12:21:03.386432 Flushing ipset 'weave-local-pods'
DEBU: 2018/01/17 12:21:03.389824 Flushing ipset 'weave-?b%zl9GIe0AET1(QI^7NWe*fO'
DEBU: 2018/01/17 12:21:03.392174 Flushing ipset 'weave-iuZcey(5DeXbzgRFs8Szo]+@p'
DEBU: 2018/01/17 12:21:03.394549 Flushing ipset 'weave-0EHD/vdN#O4]V?o4Tx7kS;APH'
DEBU: 2018/01/17 12:21:03.396856 Flushing ipset 'weave-4vtqMI+kx/2]jD%_c0S%thO%V'
DEBU: 2018/01/17 12:21:03.398967 Flushing ipset 'weave-z4cN+%s?50k!Z)QvX;[=C%d$M'
DEBU: 2018/01/17 12:21:03.400974 Flushing ipset 'weave-H?Wgfw.#Ut^=wxajFiTHd@9AF'
DEBU: 2018/01/17 12:21:03.403149 Flushing ipset 'weave-E.1.0W^NGSp]0_t5WwH/]gX@L'
DEBU: 2018/01/17 12:21:03.410181 Flushing ipset 'weave-k?Z;25^M}|1s7P3|H9i;*;MhG'
DEBU: 2018/01/17 12:21:03.412027 Destroying ipset 'weave-local-pods'
DEBU: 2018/01/17 12:21:03.413899 Destroying ipset 'weave-?b%zl9GIe0AET1(QI^7NWe*fO'
DEBU: 2018/01/17 12:21:03.415859 Destroying ipset 'weave-iuZcey(5DeXbzgRFs8Szo]+@p'
DEBU: 2018/01/17 12:21:03.417691 Destroying ipset 'weave-0EHD/vdN#O4]V?o4Tx7kS;APH'
DEBU: 2018/01/17 12:21:03.419493 Destroying ipset 'weave-4vtqMI+kx/2]jD%_c0S%thO%V'
DEBU: 2018/01/17 12:21:03.421438 Destroying ipset 'weave-z4cN+%s?50k!Z)QvX;[=C%d$M'
DEBU: 2018/01/17 12:21:03.423495 Destroying ipset 'weave-H?Wgfw.#Ut^=wxajFiTHd@9AF'
DEBU: 2018/01/17 12:21:03.425274 Destroying ipset 'weave-E.1.0W^NGSp]0_t5WwH/]gX@L'
DEBU: 2018/01/17 12:21:03.427051 Destroying ipset 'weave-k?Z;25^M}|1s7P3|H9i;*;MhG'
INFO: 2018/01/17 12:21:03.462788 EVENT AddNamespace {"metadata":{"name":"kube-system","selfLink":"/api/v1/namespaces/kube-system","uid":"44652820-f6d7-11e7-882d-fa163eea23ff","resourceVersion":"12","creationTimestamp":"2018-01-11T13:56:48Z"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}}
INFO: 2018/01/17 12:21:03.465583 creating ipset: &npc.selectorSpec{key:"", selector:labels.internalSelector{}, dst:false, ipsetType:"hash:ip", ipsetName:"weave-iuZcey(5DeXbzgRFs8Szo]+@p", nsName:"kube-system"}
DEBU: 2018/01/17 12:21:03.466960 ensuring rule for DefaultAllow in namespace: kube-system, set weave-?b%zl9GIe0AET1(QI^7NWe*fO
INFO: 2018/01/17 12:21:03.469007 EVENT AddNamespace {"metadata":{"name":"kube-public","selfLink":"/api/v1/namespaces/kube-public","uid":"466423f2-f6d7-11e7-882d-fa163eea23ff","resourceVersion":"41","creationTimestamp":"2018-01-11T13:56:51Z"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}}
INFO: 2018/01/17 12:21:03.470356 creating ipset: &npc.selectorSpec{key:"", selector:labels.internalSelector{}, dst:false, ipsetType:"hash:ip", ipsetName:"weave-4vtqMI+kx/2]jD%_c0S%thO%V", nsName:"kube-public"}
DEBU: 2018/01/17 12:21:03.471610 ensuring rule for DefaultAllow in namespace: kube-public, set weave-0EHD/vdN#O4]V?o4Tx7kS;APH
INFO: 2018/01/17 12:21:03.473148 EVENT AddNamespace {"metadata":{"name":"ns-1","selfLink":"/api/v1/namespaces/ns-1","uid":"e13c8db8-fb80-11e7-882d-fa163eea23ff","resourceVersion":"673021","creationTimestamp":"2018-01-17T12:21:01Z"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}}
INFO: 2018/01/17 12:21:03.474545 creating ipset: &npc.selectorSpec{key:"", selector:labels.internalSelector{}, dst:false, ipsetType:"hash:ip", ipsetName:"weave-H]AmKZI8BX85N/tS?%rB:R%;v", nsName:"ns-1"}
DEBU: 2018/01/17 12:21:03.475733 ensuring rule for DefaultAllow in namespace: ns-1, set weave-z4cN+%s?50k!Z)QvX;[=C%d$M
INFO: 2018/01/17 12:21:03.487272 EVENT AddNamespace {"metadata":{"name":"default","selfLink":"/api/v1/namespaces/default","uid":"43fa5835-f6d7-11e7-882d-fa163eea23ff","resourceVersion":"6","creationTimestamp":"2018-01-11T13:56:47Z"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}}
INFO: 2018/01/17 12:21:03.488329 creating ipset: &npc.selectorSpec{key:"", selector:labels.internalSelector{}, dst:false, ipsetType:"hash:ip", ipsetName:"weave-k?Z;25^M}|1s7P3|H9i;*;MhG", nsName:"default"}
DEBU: 2018/01/17 12:21:03.489449 ensuring rule for DefaultAllow in namespace: default, set weave-E.1.0W^NGSp]0_t5WwH/]gX@L
DEBU: 2018/01/17 12:21:03.604265 EVENT AddPod {"metadata":{"name":"kube-proxy-tzldg","generateName":"kube-proxy-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/kube-proxy-tzldg","uid":"f95c094e-f6d7-11e7-882d-fa163eea23ff","resourceVersion":"3937","creationTimestamp":"2018-01-11T14:01:52Z","labels":{"controller-revision-hash":"2240558267","k8s-app":"kube-proxy","pod-template-generation":"1"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"DaemonSet","name":"kube-proxy","uid":"48871426-f6d7-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"kube-proxy","configMap":{"name":"kube-proxy","defaultMode":420}},{"name":"xtables-lock","hostPath":{"path":"/run/xtables.lock","type":"FileOrCreate"}},{"name":"lib-modules","hostPath":{"path":"/lib/modules","type":""}},{"name":"kube-proxy-token-2tbjf","secret":{"secretName":"kube-proxy-token-2tbjf","defaultMode":420}}],"containers":[{"name":"kube-proxy","image":"gcr.io/google_containers/kube-proxy-amd64:v1.9.1","command":["/usr/local/bin/kube-proxy","--config=/var/lib/kube-proxy/config.conf"],"resources":{},"volumeMounts":[{"name":"kube-proxy","mountPath":"/var/lib/kube-proxy"},{"name":"xtables-lock","mountPath":"/run/xtables.lock"},{"name":"lib-modules","readOnly":true,"mountPath":"/lib/modules"},{"name":"kube-proxy-token-2tbjf","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"kube-proxy","serviceAccount":"kube-proxy","nodeName":"k8s-1","hostNetwork":true,"securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node-role.kubernetes.io/master","effect":"NoSchedule"},{"key":"node.cloudprovider.kubernetes.io/uninitialized","value":"true","effect":"NoSchedule"},{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/disk-pressure","operator":"Exists","effect":"NoSchedule"},{"key":"node.kubernetes.io/memory-pressure","operator":"Exists","effect":"NoSchedule"}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:01:52Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:02:26Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:02:26Z"}],"hostIP":"172.16.10.19","podIP":"172.16.10.19","startTime":"2018-01-11T14:01:52Z","containerStatuses":[{"name":"kube-proxy","state":{"running":{"startedAt":"2018-01-11T14:02:25Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"gcr.io/google_containers/kube-proxy-amd64:v1.9.1","imageID":"docker-pullable://gcr.io/google_containers/kube-proxy-amd64@sha256:224a8c62797066507b4d0098c1cce45b79899ad4f4c79e8ab0360f19793e33ed","containerID":"docker://9a6e6cad6c8864feb47152e767a36388798655af259a43322d2af4bb3c625ce6"}],"qosClass":"BestEffort"}}
DEBU: 2018/01/17 12:21:03.604837 EVENT AddPod {"metadata":{"name":"nginx-8586cf59-qw26n","generateName":"nginx-8586cf59-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/nginx-8586cf59-qw26n","uid":"f42e6751-f6e4-11e7-882d-fa163eea23ff","resourceVersion":"13114","creationTimestamp":"2018-01-11T15:34:46Z","labels":{"pod-template-hash":"41427915","run":"nginx"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"nginx-8586cf59","uid":"f42afad4-f6e4-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"nginx","image":"nginx","resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-2","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T15:34:46Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T15:34:49Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T15:34:46Z"}],"hostIP":"172.16.10.15","podIP":"10.46.0.1","startTime":"2018-01-11T15:34:46Z","containerStatuses":[{"name":"nginx","state":{"running":{"startedAt":"2018-01-11T15:34:49Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"nginx:latest","imageID":"docker-pullable://nginx@sha256:285b49d42c703fdf257d1e2422765c4ba9d3e37768d6ea83d7fe2043dad6e63d","containerID":"docker://89864fbea69dda7b75d1bac59c4d0f0c356927c10318ad51b0fb6e09928c1261"}],"qosClass":"BestEffort"}}
INFO: 2018/01/17 12:21:03.605020 adding entry 10.46.0.1 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of f42e6751-f6e4-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.605160 added entry 10.46.0.1 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of f42e6751-f6e4-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.617100 adding entry 10.46.0.1 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of f42e6751-f6e4-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.617301 added entry 10.46.0.1 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of f42e6751-f6e4-11e7-882d-fa163eea23ff
DEBU: 2018/01/17 12:21:03.618837 EVENT AddPod {"metadata":{"name":"heapster-5c448886d-98j8l","generateName":"heapster-5c448886d-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/heapster-5c448886d-98j8l","uid":"e93554c7-f6e6-11e7-882d-fa163eea23ff","resourceVersion":"14714","creationTimestamp":"2018-01-11T15:48:47Z","labels":{"k8s-app":"heapster","pod-template-hash":"170044428","task":"monitoring"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"heapster-5c448886d","uid":"e92c5094-f6e6-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"heapster-token-4t46w","secret":{"secretName":"heapster-token-4t46w","defaultMode":420}}],"containers":[{"name":"heapster","image":"k8s.gcr.io/heapster-amd64:v1.4.2","command":["/heapster","--source=kubernetes:https://kubernetes.default","--sink=influxdb:http://monitoring-influxdb.kube-system.svc:8086"],"resources":{},"volumeMounts":[{"name":"heapster-token-4t46w","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"heapster","serviceAccount":"heapster","nodeName":"k8s-2","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T15:48:47Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T15:49:15Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T15:48:47Z"}],"hostIP":"172.16.10.15","podIP":"10.46.0.2","startTime":"2018-01-11T15:48:47Z","containerStatuses":[{"name":"heapster","state":{"running":{"startedAt":"2018-01-11T15:49:14Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"k8s.gcr.io/heapster-amd64:v1.4.2","imageID":"docker-pullable://k8s.gcr.io/heapster-amd64@sha256:f58ded16b56884eeb73b1ba256bcc489714570bacdeca43d4ba3b91ef9897b20","containerID":"docker://53723117112e8b7061f8833df4031a4f999a4a5266944a5013572e048e8a7736"}],"qosClass":"BestEffort"}}
INFO: 2018/01/17 12:21:03.619051 adding entry 10.46.0.2 to weave-iuZcey(5DeXbzgRFs8Szo]+@p of e93554c7-f6e6-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.619193 added entry 10.46.0.2 to weave-iuZcey(5DeXbzgRFs8Szo]+@p of e93554c7-f6e6-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.620253 adding entry 10.46.0.2 to weave-?b%zl9GIe0AET1(QI^7NWe*fO of e93554c7-f6e6-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.620343 added entry 10.46.0.2 to weave-?b%zl9GIe0AET1(QI^7NWe*fO of e93554c7-f6e6-11e7-882d-fa163eea23ff
DEBU: 2018/01/17 12:21:03.621542 EVENT AddPod {"metadata":{"name":"who-is-there-8c9594776-7ttzk","generateName":"who-is-there-8c9594776-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/who-is-there-8c9594776-7ttzk","uid":"5c0b04e2-f6d8-11e7-882d-fa163eea23ff","resourceVersion":"4340","creationTimestamp":"2018-01-11T14:04:37Z","labels":{"pod-template-hash":"475150332","run":"who-is-there"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"who-is-there-8c9594776","uid":"527f72aa-f6d8-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"who-is-there","image":"xian310/who-is-there","resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-2","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:04:37Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:04:40Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:04:37Z"}],"hostIP":"172.16.10.15","podIP":"10.46.0.0","startTime":"2018-01-11T14:04:37Z","containerStatuses":[{"name":"who-is-there","state":{"running":{"startedAt":"2018-01-11T14:04:40Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"xian310/who-is-there:1","imageID":"docker-pullable://xian310/who-is-there@sha256:72181e5fab0debed045fb2e9d91faa2b6b27e6116ddb5c33cb090e09558eeab8","containerID":"docker://b300f2ca09e2c03f35f8dc510f70224159c408457a1da81fdd119fc20d452b5c"}],"qosClass":"BestEffort"}}
INFO: 2018/01/17 12:21:03.621722 adding entry 10.46.0.0 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of 5c0b04e2-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.621937 added entry 10.46.0.0 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of 5c0b04e2-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.623043 adding entry 10.46.0.0 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of 5c0b04e2-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.623185 added entry 10.46.0.0 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of 5c0b04e2-f6d8-11e7-882d-fa163eea23ff
DEBU: 2018/01/17 12:21:03.624548 EVENT AddPod {"metadata":{"name":"kube-apiserver-k8s-1","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/kube-apiserver-k8s-1","uid":"6d44a2dc-f6d7-11e7-882d-fa163eea23ff","resourceVersion":"1142","creationTimestamp":"2018-01-11T13:57:56Z","labels":{"component":"kube-apiserver","tier":"control-plane"},"annotations":{"kubernetes.io/config.hash":"ac71b479af856d44b327e64388ac3973","kubernetes.io/config.mirror":"ac71b479af856d44b327e64388ac3973","kubernetes.io/config.seen":"2018-01-11T13:54:56.600153582Z","kubernetes.io/config.source":"file","scheduler.alpha.kubernetes.io/critical-pod":""}},"spec":{"volumes":[{"name":"k8s-certs","hostPath":{"path":"/etc/kubernetes/pki","type":"DirectoryOrCreate"}},{"name":"ca-certs","hostPath":{"path":"/etc/ssl/certs","type":"DirectoryOrCreate"}}],"containers":[{"name":"kube-apiserver","image":"gcr.io/google_containers/kube-apiserver-amd64:v1.9.1","command":["kube-apiserver","--requestheader-username-headers=X-Remote-User","--requestheader-allowed-names=front-proxy-client","--service-account-key-file=/etc/kubernetes/pki/sa.pub","--kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key","--secure-port=6443","--proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt","--enable-bootstrap-token-auth=true","--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname","--tls-cert-file=/etc/kubernetes/pki/apiserver.crt","--kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt","--allow-privileged=true","--requestheader-group-headers=X-Remote-Group","--requestheader-extra-headers-prefix=X-Remote-Extra-","--advertise-address=172.16.10.19","--service-cluster-ip-range=10.96.0.0/12","--client-ca-file=/etc/kubernetes/pki/ca.crt","--tls-private-key-file=/etc/kubernetes/pki/apiserver.key","--requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt","--insecure-port=0","--admission-control=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota","--proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key","--authorization-mode=Node,RBAC","--etcd-servers=http://127.0.0.1:2379"],"resources":{"requests":{"cpu":"250m"}},"volumeMounts":[{"name":"k8s-certs","readOnly":true,"mountPath":"/etc/kubernetes/pki"},{"name":"ca-certs","readOnly":true,"mountPath":"/etc/ssl/certs"}],"livenessProbe":{"httpGet":{"path":"/healthz","port":6443,"host":"172.16.10.19","scheme":"HTTPS"},"initialDelaySeconds":15,"timeoutSeconds":15,"periodSeconds":10,"successThreshold":1,"failureThreshold":8},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","nodeName":"k8s-1","hostNetwork":true,"securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists","effect":"NoExecute"}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:55:01Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:56:38Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:55:01Z"}],"hostIP":"172.16.10.19","podIP":"172.16.10.19","startTime":"2018-01-11T13:55:01Z","containerStatuses":[{"name":"kube-apiserver","state":{"running":{"startedAt":"2018-01-11T13:56:38Z"}},"lastState":{"terminated":{"exitCode":255,"reason":"Error","startedAt":"2018-01-11T13:55:24Z","finishedAt":"2018-01-11T13:56:36Z","containerID":"docker://f685c943349c278a1b20006e4423015ff0f5c57492f46aeedc7e8acb996669e4"}},"ready":true,"restartCount":1,"image":"gcr.io/google_containers/kube-apiserver-amd64:v1.9.1","imageID":"docker-pullable://gcr.io/google_containers/kube-apiserver-amd64@sha256:f70af327840f10eb0b47b95a9848593958ead9d875f7a01bd7287fc70478e069","containerID":"docker://4cd80db11c16635b2a060ac3e864987e398655a0d2106edcd48cee0076805aca"}],"qosClass":"Burstable"}}
DEBU: 2018/01/17 12:21:03.624975 EVENT AddPod {"metadata":{"name":"weave-net-wrx29","generateName":"weave-net-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/weave-net-wrx29","uid":"1d66838a-f6d8-11e7-882d-fa163eea23ff","resourceVersion":"673043","creationTimestamp":"2018-01-11T14:02:52Z","labels":{"controller-revision-hash":"2209123374","name":"weave-net","pod-template-generation":"1"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"DaemonSet","name":"weave-net","uid":"e3e05d89-f6d7-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"weavedb","hostPath":{"path":"/var/lib/weave","type":""}},{"name":"cni-bin","hostPath":{"path":"/opt","type":""}},{"name":"cni-bin2","hostPath":{"path":"/home","type":""}},{"name":"cni-conf","hostPath":{"path":"/etc","type":""}},{"name":"dbus","hostPath":{"path":"/var/lib/dbus","type":""}},{"name":"lib-modules","hostPath":{"path":"/lib/modules","type":""}},{"name":"xtables-lock","hostPath":{"path":"/run/xtables.lock","type":""}},{"name":"weave-net-token-98zhk","secret":{"secretName":"weave-net-token-98zhk","defaultMode":420}}],"containers":[{"name":"weave","image":"weaveworks/weave-kube:2.1.3","command":["/home/weave/launch.sh"],"env":[{"name":"HOSTNAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}],"resources":{"requests":{"cpu":"10m"}},"volumeMounts":[{"name":"weavedb","mountPath":"/weavedb"},{"name":"cni-bin","mountPath":"/host/opt"},{"name":"cni-bin2","mountPath":"/host/home"},{"name":"cni-conf","mountPath":"/host/etc"},{"name":"dbus","mountPath":"/host/var/lib/dbus"},{"name":"lib-modules","mountPath":"/lib/modules"},{"name":"xtables-lock","mountPath":"/run/xtables.lock"},{"name":"weave-net-token-98zhk","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"livenessProbe":{"httpGet":{"path":"/status","port":6784,"host":"127.0.0.1","scheme":"HTTP"},"initialDelaySeconds":30,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}},{"name":"weave-npc","image":"weaveworks/weave-npc:2.1.3","env":[{"name":"HOSTNAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}],"resources":{"requests":{"cpu":"10m"}},"volumeMounts":[{"name":"xtables-lock","mountPath":"/run/xtables.lock"},{"name":"weave-net-token-98zhk","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"weave-net","serviceAccount":"weave-net","nodeName":"k8s-2","hostNetwork":true,"hostPID":true,"securityContext":{"seLinuxOptions":{}},"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists","effect":"NoSchedule"},{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/disk-pressure","operator":"Exists","effect":"NoSchedule"},{"key":"node.kubernetes.io/memory-pressure","operator":"Exists","effect":"NoSchedule"}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:02:57Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:21:02Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:02:59Z"}],"hostIP":"172.16.10.15","podIP":"172.16.10.15","startTime":"2018-01-11T14:02:57Z","containerStatuses":[{"name":"weave","state":{"running":{"startedAt":"2018-01-11T14:02:58Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"weaveworks/weave-kube:2.1.3","imageID":"docker-pullable://weaveworks/weave-kube@sha256:07a3d56b8592ea3e00ace6f2c3eb7e65f3cc4945188a9e2a884b8172e6a0007e","containerID":"docker://e4ca1018f6c2353ec4390f88114248442db4e80e45b0f11743ed994f0ca09525"},{"name":"weave-npc","state":{"running":{"startedAt":"2018-01-17T12:21:02Z"}},"lastState":{"terminated":{"exitCode":1,"reason":"Error","startedAt":"2018-01-17T09:19:57Z","finishedAt":"2018-01-17T12:21:01Z","containerID":"docker://91ebf61b6c9eaf39d04ad29449ae990b9ac76a0d6075a5364cf7d88037b7536f"}},"ready":true,"restartCount":7,"image":"weaveworks/weave-npc:2.1.3","imageID":"docker-pullable://weaveworks/weave-npc@sha256:f35eb8166d7dae3fa7bb4d9892ab6dc8ea5c969f73791be590a0a213767c0f07","containerID":"docker://f797f871ad1df317cfc08f9103e83c04092e4e808c9aa0521d2e69cf5ef0ee22"}],"qosClass":"Burstable"}}
DEBU: 2018/01/17 12:21:03.625349 EVENT AddPod {"metadata":{"name":"who-is-there-8c9594776-kcbnv","generateName":"who-is-there-8c9594776-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/who-is-there-8c9594776-kcbnv","uid":"52862b69-f6d8-11e7-882d-fa163eea23ff","resourceVersion":"4241","creationTimestamp":"2018-01-11T14:04:21Z","labels":{"pod-template-hash":"475150332","run":"who-is-there"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"who-is-there-8c9594776","uid":"527f72aa-f6d8-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"who-is-there","image":"xian310/who-is-there","resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-3","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:04:21Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:04:24Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:04:21Z"}],"hostIP":"172.16.10.16","podIP":"10.40.0.0","startTime":"2018-01-11T14:04:21Z","containerStatuses":[{"name":"who-is-there","state":{"running":{"startedAt":"2018-01-11T14:04:24Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"xian310/who-is-there:1","imageID":"docker-pullable://xian310/who-is-there@sha256:72181e5fab0debed045fb2e9d91faa2b6b27e6116ddb5c33cb090e09558eeab8","containerID":"docker://d3f73322a326b35b72a421c28b5401304af37dc50824917868415fa91bce9009"}],"qosClass":"BestEffort"}}
INFO: 2018/01/17 12:21:03.625492 adding entry 10.40.0.0 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of 52862b69-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.625587 added entry 10.40.0.0 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of 52862b69-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.626728 adding entry 10.40.0.0 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of 52862b69-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.626865 added entry 10.40.0.0 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of 52862b69-f6d8-11e7-882d-fa163eea23ff
DEBU: 2018/01/17 12:21:03.628218 EVENT AddPod {"metadata":{"name":"kube-proxy-h8rm2","generateName":"kube-proxy-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/kube-proxy-h8rm2","uid":"1d66259f-f6d8-11e7-882d-fa163eea23ff","resourceVersion":"4100","creationTimestamp":"2018-01-11T14:02:52Z","labels":{"controller-revision-hash":"2240558267","k8s-app":"kube-proxy","pod-template-generation":"1"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"DaemonSet","name":"kube-proxy","uid":"48871426-f6d7-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"kube-proxy","configMap":{"name":"kube-proxy","defaultMode":420}},{"name":"xtables-lock","hostPath":{"path":"/run/xtables.lock","type":"FileOrCreate"}},{"name":"lib-modules","hostPath":{"path":"/lib/modules","type":""}},{"name":"kube-proxy-token-2tbjf","secret":{"secretName":"kube-proxy-token-2tbjf","defaultMode":420}}],"containers":[{"name":"kube-proxy","image":"gcr.io/google_containers/kube-proxy-amd64:v1.9.1","command":["/usr/local/bin/kube-proxy","--config=/var/lib/kube-proxy/config.conf"],"resources":{},"volumeMounts":[{"name":"kube-proxy","mountPath":"/var/lib/kube-proxy"},{"name":"xtables-lock","mountPath":"/run/xtables.lock"},{"name":"lib-modules","readOnly":true,"mountPath":"/lib/modules"},{"name":"kube-proxy-token-2tbjf","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"kube-proxy","serviceAccount":"kube-proxy","nodeName":"k8s-2","hostNetwork":true,"securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node-role.kubernetes.io/master","effect":"NoSchedule"},{"key":"node.cloudprovider.kubernetes.io/uninitialized","value":"true","effect":"NoSchedule"},{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/disk-pressure","operator":"Exists","effect":"NoSchedule"},{"key":"node.kubernetes.io/memory-pressure","operator":"Exists","effect":"NoSchedule"}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:02:57Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:03:25Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:03:25Z"}],"hostIP":"172.16.10.15","podIP":"172.16.10.15","startTime":"2018-01-11T14:02:57Z","containerStatuses":[{"name":"kube-proxy","state":{"running":{"startedAt":"2018-01-11T14:03:25Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"gcr.io/google_containers/kube-proxy-amd64:v1.9.1","imageID":"docker-pullable://gcr.io/google_containers/kube-proxy-amd64@sha256:224a8c62797066507b4d0098c1cce45b79899ad4f4c79e8ab0360f19793e33ed","containerID":"docker://8f30e124096b3a28d15ae135759f0960694c1545cec35b3d49d83113973c3355"}],"qosClass":"BestEffort"}}
DEBU: 2018/01/17 12:21:03.628500 EVENT AddPod {"metadata":{"name":"nginx-8586cf59-tjrqz","generateName":"nginx-8586cf59-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/nginx-8586cf59-tjrqz","uid":"f433c8c6-f6e4-11e7-882d-fa163eea23ff","resourceVersion":"13117","creationTimestamp":"2018-01-11T15:34:46Z","labels":{"pod-template-hash":"41427915","run":"nginx"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"nginx-8586cf59","uid":"f42afad4-f6e4-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"nginx","image":"nginx","resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-3","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T15:34:46Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T15:34:49Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T15:34:46Z"}],"hostIP":"172.16.10.16","podIP":"10.40.0.1","startTime":"2018-01-11T15:34:46Z","containerStatuses":[{"name":"nginx","state":{"running":{"startedAt":"2018-01-11T15:34:49Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"nginx:latest","imageID":"docker-pullable://nginx@sha256:285b49d42c703fdf257d1e2422765c4ba9d3e37768d6ea83d7fe2043dad6e63d","containerID":"docker://5d274b4903a00169803d3d0d837c3f13082f2542508b4d33e0cafed8b5664e75"}],"qosClass":"BestEffort"}}
INFO: 2018/01/17 12:21:03.628637 adding entry 10.40.0.1 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of f433c8c6-f6e4-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.628718 added entry 10.40.0.1 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of f433c8c6-f6e4-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.629851 adding entry 10.40.0.1 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of f433c8c6-f6e4-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.629992 added entry 10.40.0.1 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of f433c8c6-f6e4-11e7-882d-fa163eea23ff
DEBU: 2018/01/17 12:21:03.631335 EVENT AddPod {"metadata":{"name":"busybox","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/busybox","uid":"844de276-fb68-11e7-882d-fa163eea23ff","resourceVersion":"658823","creationTimestamp":"2018-01-17T09:26:37Z","labels":{"run":"busybox"}},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"busybox","image":"busybox","args":["sh"],"resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","stdin":true,"stdinOnce":true,"tty":true}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-3","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Failed","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T09:26:37Z"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2018-01-17T09:27:36Z","reason":"ContainersNotReady","message":"containers with unready status: [busybox]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T09:26:37Z"}],"hostIP":"172.16.10.16","podIP":"10.40.0.3","startTime":"2018-01-17T09:26:37Z","containerStatuses":[{"name":"busybox","state":{"terminated":{"exitCode":1,"reason":"Error","startedAt":"2018-01-17T09:26:38Z","finishedAt":"2018-01-17T09:27:35Z","containerID":"docker://b11804bcf5cc2d41d93714ae6db9067ff3087183d00c8a1e1af19d39209a0e2b"}},"lastState":{},"ready":false,"restartCount":0,"image":"busybox:latest","imageID":"docker-pullable://busybox@sha256:2b30a608a73cb0706bee4fbf16cbb0cc25ccc5874d5a183cb93a7fd45b032c25","containerID":"docker://b11804bcf5cc2d41d93714ae6db9067ff3087183d00c8a1e1af19d39209a0e2b"}],"qosClass":"BestEffort"}}
DEBU: 2018/01/17 12:21:03.631699 EVENT AddPod {"metadata":{"name":"kube-dns-6f4fd4bdf-p7f9t","generateName":"kube-dns-6f4fd4bdf-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/kube-dns-6f4fd4bdf-p7f9t","uid":"4ba6d7ae-f6d7-11e7-882d-fa163eea23ff","resourceVersion":"4165","creationTimestamp":"2018-01-11T13:57:00Z","labels":{"k8s-app":"kube-dns","pod-template-hash":"290980689"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"kube-dns-6f4fd4bdf","uid":"4ba2c555-f6d7-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"kube-dns-config","configMap":{"name":"kube-dns","defaultMode":420,"optional":true}},{"name":"kube-dns-token-knbt9","secret":{"secretName":"kube-dns-token-knbt9","defaultMode":420}}],"containers":[{"name":"kubedns","image":"gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.7","args":["--domain=cluster.local.","--dns-port=10053","--config-dir=/kube-dns-config","--v=2"],"ports":[{"name":"dns-local","containerPort":10053,"protocol":"UDP"},{"name":"dns-tcp-local","containerPort":10053,"protocol":"TCP"},{"name":"metrics","containerPort":10055,"protocol":"TCP"}],"env":[{"name":"PROMETHEUS_PORT","value":"10055"}],"resources":{"limits":{"memory":"170Mi"},"requests":{"cpu":"100m","memory":"70Mi"}},"volumeMounts":[{"name":"kube-dns-config","mountPath":"/kube-dns-config"},{"name":"kube-dns-token-knbt9","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"livenessProbe":{"httpGet":{"path":"/healthcheck/kubedns","port":10054,"scheme":"HTTP"},"initialDelaySeconds":60,"timeoutSeconds":5,"periodSeconds":10,"successThreshold":1,"failureThreshold":5},"readinessProbe":{"httpGet":{"path":"/readiness","port":8081,"scheme":"HTTP"},"initialDelaySeconds":3,"timeoutSeconds":5,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"},{"name":"dnsmasq","image":"gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.7","args":["-v=2","-logtostderr","-configDir=/etc/k8s/dns/dnsmasq-nanny","-restartDnsmasq=true","--","-k","--cache-size=1000","--no-negcache","--log-facility=-","--server=/cluster.local/127.0.0.1#10053","--server=/in-addr.arpa/127.0.0.1#10053","--server=/ip6.arpa/127.0.0.1#10053"],"ports":[{"name":"dns","containerPort":53,"protocol":"UDP"},{"name":"dns-tcp","containerPort":53,"protocol":"TCP"}],"resources":{"requests":{"cpu":"150m","memory":"20Mi"}},"volumeMounts":[{"name":"kube-dns-config","mountPath":"/etc/k8s/dns/dnsmasq-nanny"},{"name":"kube-dns-token-knbt9","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"livenessProbe":{"httpGet":{"path":"/healthcheck/dnsmasq","port":10054,"scheme":"HTTP"},"initialDelaySeconds":60,"timeoutSeconds":5,"periodSeconds":10,"successThreshold":1,"failureThreshold":5},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"},{"name":"sidecar","image":"gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.7","args":["--v=2","--logtostderr","--probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,SRV","--probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,SRV"],"ports":[{"name":"metrics","containerPort":10054,"protocol":"TCP"}],"resources":{"requests":{"cpu":"10m","memory":"20Mi"}},"volumeMounts":[{"name":"kube-dns-token-knbt9","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"livenessProbe":{"httpGet":{"path":"/metrics","port":10054,"scheme":"HTTP"},"initialDelaySeconds":60,"timeoutSeconds":5,"periodSeconds":10,"successThreshold":1,"failureThreshold":5},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"Default","serviceAccountName":"kube-dns","serviceAccount":"kube-dns","nodeName":"k8s-1","securityContext":{},"affinity":{"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"beta.kubernetes.io/arch","operator":"In","values":["amd64"]}]}]}}},"schedulerName":"default-scheduler","tolerations":[{"key":"CriticalAddonsOnly","operator":"Exists"},{"key":"node-role.kubernetes.io/master","effect":"NoSchedule"},{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:03:09Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:03:48Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:03:09Z"}],"hostIP":"172.16.10.19","podIP":"10.32.0.3","startTime":"2018-01-11T14:03:09Z","containerStatuses":[{"name":"dnsmasq","state":{"running":{"startedAt":"2018-01-11T14:03:27Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.7","imageID":"docker-pullable://gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64@sha256:6cfb9f9c2756979013dbd3074e852c2d8ac99652570c5d17d152e0c0eb3321d6","containerID":"docker://9edfecb9c0d2d6392fb198b8ef9c79b4b69d3fca756e7be40341e69df259cca8"},{"name":"kubedns","state":{"running":{"startedAt":"2018-01-11T14:03:20Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.7","imageID":"docker-pullable://gcr.io/google_containers/k8s-dns-kube-dns-amd64@sha256:f5bddc71efe905f4e4b96f3ca346414be6d733610c1525b98fff808f93966680","containerID":"docker://4832cbf8e70dcf9efe1722cbcdc12874085f92ddbd4c6bd2bcd6b06dfe5c118f"},{"name":"sidecar","state":{"running":{"startedAt":"2018-01-11T14:03:39Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.7","imageID":"docker-pullable://gcr.io/google_containers/k8s-dns-sidecar-amd64@sha256:f80f5f9328107dc516d67f7b70054354b9367d31d4946a3bffd3383d83d7efe8","containerID":"docker://e1896dae59382ea3e20fe1a22a03f4e19706601f57e65c7c1225604c00d854b1"}],"qosClass":"Burstable"}}
INFO: 2018/01/17 12:21:03.631918 adding entry 10.32.0.3 to weave-local-pods of 4ba6d7ae-f6d7-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.632031 added entry 10.32.0.3 to weave-local-pods of 4ba6d7ae-f6d7-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.633150 adding entry 10.32.0.3 to weave-iuZcey(5DeXbzgRFs8Szo]+@p of 4ba6d7ae-f6d7-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.633314 added entry 10.32.0.3 to weave-iuZcey(5DeXbzgRFs8Szo]+@p of 4ba6d7ae-f6d7-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.634462 adding entry 10.32.0.3 to weave-?b%zl9GIe0AET1(QI^7NWe*fO of 4ba6d7ae-f6d7-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.634612 added entry 10.32.0.3 to weave-?b%zl9GIe0AET1(QI^7NWe*fO of 4ba6d7ae-f6d7-11e7-882d-fa163eea23ff
DEBU: 2018/01/17 12:21:03.635765 EVENT AddPod {"metadata":{"name":"kube-proxy-k2hr7","generateName":"kube-proxy-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/kube-proxy-k2hr7","uid":"2f918a76-f6d8-11e7-882d-fa163eea23ff","resourceVersion":"4183","creationTimestamp":"2018-01-11T14:03:22Z","labels":{"controller-revision-hash":"2240558267","k8s-app":"kube-proxy","pod-template-generation":"1"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"DaemonSet","name":"kube-proxy","uid":"48871426-f6d7-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"kube-proxy","configMap":{"name":"kube-proxy","defaultMode":420}},{"name":"xtables-lock","hostPath":{"path":"/run/xtables.lock","type":"FileOrCreate"}},{"name":"lib-modules","hostPath":{"path":"/lib/modules","type":""}},{"name":"kube-proxy-token-2tbjf","secret":{"secretName":"kube-proxy-token-2tbjf","defaultMode":420}}],"containers":[{"name":"kube-proxy","image":"gcr.io/google_containers/kube-proxy-amd64:v1.9.1","command":["/usr/local/bin/kube-proxy","--config=/var/lib/kube-proxy/config.conf"],"resources":{},"volumeMounts":[{"name":"kube-proxy","mountPath":"/var/lib/kube-proxy"},{"name":"xtables-lock","mountPath":"/run/xtables.lock"},{"name":"lib-modules","readOnly":true,"mountPath":"/lib/modules"},{"name":"kube-proxy-token-2tbjf","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"kube-proxy","serviceAccount":"kube-proxy","nodeName":"k8s-3","hostNetwork":true,"securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node-role.kubernetes.io/master","effect":"NoSchedule"},{"key":"node.cloudprovider.kubernetes.io/uninitialized","value":"true","effect":"NoSchedule"},{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/disk-pressure","operator":"Exists","effect":"NoSchedule"},{"key":"node.kubernetes.io/memory-pressure","operator":"Exists","effect":"NoSchedule"}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:03:27Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:03:52Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:03:52Z"}],"hostIP":"172.16.10.16","podIP":"172.16.10.16","startTime":"2018-01-11T14:03:27Z","containerStatuses":[{"name":"kube-proxy","state":{"running":{"startedAt":"2018-01-11T14:03:52Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"gcr.io/google_containers/kube-proxy-amd64:v1.9.1","imageID":"docker-pullable://gcr.io/google_containers/kube-proxy-amd64@sha256:224a8c62797066507b4d0098c1cce45b79899ad4f4c79e8ab0360f19793e33ed","containerID":"docker://c010bfe1fb17662967973b6dccc3ff08317e24d72843b21295cc3aa60cc8c43c"}],"qosClass":"BestEffort"}}
DEBU: 2018/01/17 12:21:03.635987 EVENT AddPod {"metadata":{"name":"who-is-there-8c9594776-pcnw6","generateName":"who-is-there-8c9594776-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/who-is-there-8c9594776-pcnw6","uid":"5c2281fa-f6d8-11e7-882d-fa163eea23ff","resourceVersion":"4347","creationTimestamp":"2018-01-11T14:04:37Z","labels":{"pod-template-hash":"475150332","run":"who-is-there"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"who-is-there-8c9594776","uid":"527f72aa-f6d8-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"who-is-there","image":"xian310/who-is-there","resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-3","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:04:37Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:04:41Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:04:37Z"}],"hostIP":"172.16.10.16","podIP":"10.40.0.2","startTime":"2018-01-11T14:04:37Z","containerStatuses":[{"name":"who-is-there","state":{"running":{"startedAt":"2018-01-11T14:04:41Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"xian310/who-is-there:1","imageID":"docker-pullable://xian310/who-is-there@sha256:72181e5fab0debed045fb2e9d91faa2b6b27e6116ddb5c33cb090e09558eeab8","containerID":"docker://e287e2203236dc796fff1ea4774abc2a658fb8675f68cc368042ee920e9b5b1e"}],"qosClass":"BestEffort"}}
INFO: 2018/01/17 12:21:03.636156 adding entry 10.40.0.2 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of 5c2281fa-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.636250 added entry 10.40.0.2 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of 5c2281fa-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.637340 adding entry 10.40.0.2 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of 5c2281fa-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.637561 added entry 10.40.0.2 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of 5c2281fa-f6d8-11e7-882d-fa163eea23ff
DEBU: 2018/01/17 12:21:03.638881 EVENT AddPod {"metadata":{"name":"kubernetes-dashboard-6ddcb6df4c-j46jk","generateName":"kubernetes-dashboard-6ddcb6df4c-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/kubernetes-dashboard-6ddcb6df4c-j46jk","uid":"d1d62225-f6d8-11e7-882d-fa163eea23ff","resourceVersion":"4678","creationTimestamp":"2018-01-11T14:07:55Z","labels":{"k8s-app":"kubernetes-dashboard","pod-template-hash":"2887628907"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"kubernetes-dashboard-6ddcb6df4c","uid":"d1cb3d30-f6d8-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"kubernetes-dashboard-certs","secret":{"secretName":"kubernetes-dashboard-certs","defaultMode":420}},{"name":"tmp-volume","emptyDir":{}},{"name":"kubernetes-dashboard-token-8zmcx","secret":{"secretName":"kubernetes-dashboard-token-8zmcx","defaultMode":420}}],"containers":[{"name":"kubernetes-dashboard","image":"k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.1","args":["--auto-generate-certificates"],"ports":[{"containerPort":8443,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"kubernetes-dashboard-certs","mountPath":"/certs"},{"name":"tmp-volume","mountPath":"/tmp"},{"name":"kubernetes-dashboard-token-8zmcx","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"livenessProbe":{"httpGet":{"path":"/","port":8443,"scheme":"HTTPS"},"initialDelaySeconds":30,"timeoutSeconds":30,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"kubernetes-dashboard","serviceAccount":"kubernetes-dashboard","nodeName":"k8s-3","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node-role.kubernetes.io/master","effect":"NoSchedule"},{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:07:55Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:07:57Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:07:55Z"}],"hostIP":"172.16.10.16","podIP":"10.40.0.5","startTime":"2018-01-11T14:07:55Z","containerStatuses":[{"name":"kubernetes-dashboard","state":{"running":{"startedAt":"2018-01-11T14:07:56Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.1","imageID":"docker-pullable://k8s.gcr.io/kubernetes-dashboard-amd64@sha256:3861695e962972965a4c611bcabc2032f885d8cbdb0bccc9bf513ef16335fe33","containerID":"docker://3394fcdf9f431f5a3907b73e47b66c0c189c0018ed23124badb604abb91a8b0a"}],"qosClass":"BestEffort"}}
INFO: 2018/01/17 12:21:03.639017 adding entry 10.40.0.5 to weave-iuZcey(5DeXbzgRFs8Szo]+@p of d1d62225-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.639095 added entry 10.40.0.5 to weave-iuZcey(5DeXbzgRFs8Szo]+@p of d1d62225-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.640254 adding entry 10.40.0.5 to weave-?b%zl9GIe0AET1(QI^7NWe*fO of d1d62225-f6d8-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:21:03.640343 added entry 10.40.0.5 to weave-?b%zl9GIe0AET1(QI^7NWe*fO of d1d62225-f6d8-11e7-882d-fa163eea23ff
DEBU: 2018/01/17 12:21:03.641625 EVENT AddPod {"metadata":{"name":"kube-scheduler-k8s-1","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/kube-scheduler-k8s-1","uid":"495d4363-f6d7-11e7-882d-fa163eea23ff","resourceVersion":"436","creationTimestamp":"2018-01-11T13:56:56Z","labels":{"component":"kube-scheduler","tier":"control-plane"},"annotations":{"kubernetes.io/config.hash":"149413a6901331c7f8577d115ee2cb68","kubernetes.io/config.mirror":"149413a6901331c7f8577d115ee2cb68","kubernetes.io/config.seen":"2018-01-11T13:54:56.600157924Z","kubernetes.io/config.source":"file","scheduler.alpha.kubernetes.io/critical-pod":""}},"spec":{"volumes":[{"name":"kubeconfig","hostPath":{"path":"/etc/kubernetes/scheduler.conf","type":"FileOrCreate"}}],"containers":[{"name":"kube-scheduler","image":"gcr.io/google_containers/kube-scheduler-amd64:v1.9.1","command":["kube-scheduler","--address=127.0.0.1","--leader-elect=true","--kubeconfig=/etc/kubernetes/scheduler.conf"],"resources":{"requests":{"cpu":"100m"}},"volumeMounts":[{"name":"kubeconfig","readOnly":true,"mountPath":"/etc/kubernetes/scheduler.conf"}],"livenessProbe":{"httpGet":{"path":"/healthz","port":10251,"host":"127.0.0.1","scheme":"HTTP"},"initialDelaySeconds":15,"timeoutSeconds":15,"periodSeconds":10,"successThreshold":1,"failureThreshold":8},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","nodeName":"k8s-1","hostNetwork":true,"securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists","effect":"NoExecute"}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:55:01Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:55:52Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:55:01Z"}],"hostIP":"172.16.10.19","podIP":"172.16.10.19","startTime":"2018-01-11T13:55:01Z","containerStatuses":[{"name":"kube-scheduler","state":{"running":{"startedAt":"2018-01-11T13:55:52Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"gcr.io/google_containers/kube-scheduler-amd64:v1.9.1","imageID":"docker-pullable://gcr.io/google_containers/kube-scheduler-amd64@sha256:63b8bf87539e5aaf2c73527e5426cf47d01e8b23aa76848bcd92823174494b59","containerID":"docker://548a8c1966c3f01c68e649807fe3aa8fb49688e9de6a48330d9cd9b0826ed2eb"}],"qosClass":"Burstable"}}
DEBU: 2018/01/17 12:21:03.642030 EVENT AddPod {"metadata":{"name":"weave-net-xh4v5","generateName":"weave-net-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/weave-net-xh4v5","uid":"2f925867-f6d8-11e7-882d-fa163eea23ff","resourceVersion":"673039","creationTimestamp":"2018-01-11T14:03:22Z","labels":{"controller-revision-hash":"2209123374","name":"weave-net","pod-template-generation":"1"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"DaemonSet","name":"weave-net","uid":"e3e05d89-f6d7-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"weavedb","hostPath":{"path":"/var/lib/weave","type":""}},{"name":"cni-bin","hostPath":{"path":"/opt","type":""}},{"name":"cni-bin2","hostPath":{"path":"/home","type":""}},{"name":"cni-conf","hostPath":{"path":"/etc","type":""}},{"name":"dbus","hostPath":{"path":"/var/lib/dbus","type":""}},{"name":"lib-modules","hostPath":{"path":"/lib/modules","type":""}},{"name":"xtables-lock","hostPath":{"path":"/run/xtables.lock","type":""}},{"name":"weave-net-token-98zhk","secret":{"secretName":"weave-net-token-98zhk","defaultMode":420}}],"containers":[{"name":"weave","image":"weaveworks/weave-kube:2.1.3","command":["/home/weave/launch.sh"],"env":[{"name":"HOSTNAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}],"resources":{"requests":{"cpu":"10m"}},"volumeMounts":[{"name":"weavedb","mountPath":"/weavedb"},{"name":"cni-bin","mountPath":"/host/opt"},{"name":"cni-bin2","mountPath":"/host/home"},{"name":"cni-conf","mountPath":"/host/etc"},{"name":"dbus","mountPath":"/host/var/lib/dbus"},{"name":"lib-modules","mountPath":"/lib/modules"},{"name":"xtables-lock","mountPath":"/run/xtables.lock"},{"name":"weave-net-token-98zhk","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"livenessProbe":{"httpGet":{"path":"/status","port":6784,"host":"127.0.0.1","scheme":"HTTP"},"initialDelaySeconds":30,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}},{"name":"weave-npc","image":"weaveworks/weave-npc:2.1.3","env":[{"name":"HOSTNAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}],"resources":{"requests":{"cpu":"10m"}},"volumeMounts":[{"name":"xtables-lock","mountPath":"/run/xtables.lock"},{"name":"weave-net-token-98zhk","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"weave-net","serviceAccount":"weave-net","nodeName":"k8s-3","hostNetwork":true,"hostPID":true,"securityContext":{"seLinuxOptions":{}},"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists","effect":"NoSchedule"},{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/disk-pressure","operator":"Exists","effect":"NoSchedule"},{"key":"node.kubernetes.io/memory-pressure","operator":"Exists","effect":"NoSchedule"}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:03:27Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:21:02Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:03:29Z"}],"hostIP":"172.16.10.16","podIP":"172.16.10.16","startTime":"2018-01-11T14:03:27Z","containerStatuses":[{"name":"weave","state":{"running":{"startedAt":"2018-01-11T14:03:28Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"weaveworks/weave-kube:2.1.3","imageID":"docker-pullable://weaveworks/weave-kube@sha256:07a3d56b8592ea3e00ace6f2c3eb7e65f3cc4945188a9e2a884b8172e6a0007e","containerID":"docker://3c0b98bc1d15700dbc5fb37e008ee8569194bb5c2a259b094227ccefcddc750c"},{"name":"weave-npc","state":{"running":{"startedAt":"2018-01-17T12:21:01Z"}},"lastState":{"terminated":{"exitCode":1,"reason":"Error","startedAt":"2018-01-17T09:19:57Z","finishedAt":"2018-01-17T12:21:01Z","containerID":"docker://9930644bafe0ed211d03854e039a4302d9bf3113e1e13e6df67b5570ecaad544"}},"ready":true,"restartCount":7,"image":"weaveworks/weave-npc:2.1.3","imageID":"docker-pullable://weaveworks/weave-npc@sha256:f35eb8166d7dae3fa7bb4d9892ab6dc8ea5c969f73791be590a0a213767c0f07","containerID":"docker://5587e0e2891a09419b076212673da62ea31a8c43e4516b19dc587679803cb878"}],"qosClass":"Burstable"}}
DEBU: 2018/01/17 12:21:03.642256 EVENT AddPod {"metadata":{"name":"kube-controller-manager-k8s-1","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/kube-controller-manager-k8s-1","uid":"49f4172c-f6d7-11e7-882d-fa163eea23ff","resourceVersion":"433","creationTimestamp":"2018-01-11T13:56:57Z","labels":{"component":"kube-controller-manager","tier":"control-plane"},"annotations":{"kubernetes.io/config.hash":"a1c3907300cbc1a19ca29521ad6c5937","kubernetes.io/config.mirror":"a1c3907300cbc1a19ca29521ad6c5937","kubernetes.io/config.seen":"2018-01-11T13:54:56.6001559Z","kubernetes.io/config.source":"file","scheduler.alpha.kubernetes.io/critical-pod":""}},"spec":{"volumes":[{"name":"k8s-certs","hostPath":{"path":"/etc/kubernetes/pki","type":"DirectoryOrCreate"}},{"name":"ca-certs","hostPath":{"path":"/etc/ssl/certs","type":"DirectoryOrCreate"}},{"name":"kubeconfig","hostPath":{"path":"/etc/kubernetes/controller-manager.conf","type":"FileOrCreate"}},{"name":"flexvolume-dir","hostPath":{"path":"/usr/libexec/kubernetes/kubelet-plugins/volume/exec","type":"DirectoryOrCreate"}}],"containers":[{"name":"kube-controller-manager","image":"gcr.io/google_containers/kube-controller-manager-amd64:v1.9.1","command":["kube-controller-manager","--controllers=*,bootstrapsigner,tokencleaner","--cluster-signing-key-file=/etc/kubernetes/pki/ca.key","--cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt","--address=127.0.0.1","--leader-elect=true","--use-service-account-credentials=true","--kubeconfig=/etc/kubernetes/controller-manager.conf","--root-ca-file=/etc/kubernetes/pki/ca.crt","--service-account-private-key-file=/etc/kubernetes/pki/sa.key"],"resources":{"requests":{"cpu":"200m"}},"volumeMounts":[{"name":"kubeconfig","readOnly":true,"mountPath":"/etc/kubernetes/controller-manager.conf"},{"name":"flexvolume-dir","mountPath":"/usr/libexec/kubernetes/kubelet-plugins/volume/exec"},{"name":"k8s-certs","readOnly":true,"mountPath":"/etc/kubernetes/pki"},{"name":"ca-certs","readOnly":true,"mountPath":"/etc/ssl/certs"}],"livenessProbe":{"httpGet":{"path":"/healthz","port":10252,"host":"127.0.0.1","scheme":"HTTP"},"initialDelaySeconds":15,"timeoutSeconds":15,"periodSeconds":10,"successThreshold":1,"failureThreshold":8},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","nodeName":"k8s-1","hostNetwork":true,"securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists","effect":"NoExecute"}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:55:01Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:55:42Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:55:01Z"}],"hostIP":"172.16.10.19","podIP":"172.16.10.19","startTime":"2018-01-11T13:55:01Z","containerStatuses":[{"name":"kube-controller-manager","state":{"running":{"startedAt":"2018-01-11T13:55:42Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"gcr.io/google_containers/kube-controller-manager-amd64:v1.9.1","imageID":"docker-pullable://gcr.io/google_containers/kube-controller-manager-amd64@sha256:d4ae3c042bfd04f4be9b7fed8efc52b8293503f2cf30399de4ceb99b6182001b","containerID":"docker://c928e85b4a085b1f8ebc6b58ecea329dff9bda4757f0cb0cfe73bf298ca7bcfa"}],"qosClass":"Burstable"}}
DEBU: 2018/01/17 12:21:03.642436 EVENT AddPod {"metadata":{"name":"etcd-k8s-1","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/etcd-k8s-1","uid":"69b12513-f6d7-11e7-882d-fa163eea23ff","resourceVersion":"1025","creationTimestamp":"2018-01-11T13:57:50Z","labels":{"component":"etcd","tier":"control-plane"},"annotations":{"kubernetes.io/config.hash":"408851a572c13f8177557fdb9151111c","kubernetes.io/config.mirror":"408851a572c13f8177557fdb9151111c","kubernetes.io/config.seen":"2018-01-11T13:54:56.600146154Z","kubernetes.io/config.source":"file","scheduler.alpha.kubernetes.io/critical-pod":""}},"spec":{"volumes":[{"name":"etcd","hostPath":{"path":"/var/lib/etcd","type":"DirectoryOrCreate"}}],"containers":[{"name":"etcd","image":"gcr.io/google_containers/etcd-amd64:3.1.10","command":["etcd","--listen-client-urls=http://127.0.0.1:2379","--advertise-client-urls=http://127.0.0.1:2379","--data-dir=/var/lib/etcd"],"resources":{},"volumeMounts":[{"name":"etcd","mountPath":"/var/lib/etcd"}],"livenessProbe":{"httpGet":{"path":"/health","port":2379,"host":"127.0.0.1","scheme":"HTTP"},"initialDelaySeconds":15,"timeoutSeconds":15,"periodSeconds":10,"successThreshold":1,"failureThreshold":8},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","nodeName":"k8s-1","hostNetwork":true,"securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists","effect":"NoExecute"}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:55:01Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:56:37Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T13:55:01Z"}],"hostIP":"172.16.10.19","podIP":"172.16.10.19","startTime":"2018-01-11T13:55:01Z","containerStatuses":[{"name":"etcd","state":{"running":{"startedAt":"2018-01-11T13:56:37Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"gcr.io/google_containers/etcd-amd64:3.1.10","imageID":"docker-pullable://gcr.io/google_containers/etcd-amd64@sha256:28cf78933de29fd26d7a879e51ebd39784cd98109568fd3da61b141257fb85a6","containerID":"docker://b5f8b5144d3a70164acbd1a7f7c463ccd46863c7d7c72811de3a51bbea849645"}],"qosClass":"BestEffort"}}
DEBU: 2018/01/17 12:21:03.642684 EVENT AddPod {"metadata":{"name":"weave-net-4s7vw","generateName":"weave-net-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/weave-net-4s7vw","uid":"f9f59e56-f6d7-11e7-882d-fa163eea23ff","resourceVersion":"673045","creationTimestamp":"2018-01-11T14:01:53Z","labels":{"controller-revision-hash":"2209123374","name":"weave-net","pod-template-generation":"1"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"DaemonSet","name":"weave-net","uid":"e3e05d89-f6d7-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"weavedb","hostPath":{"path":"/var/lib/weave","type":""}},{"name":"cni-bin","hostPath":{"path":"/opt","type":""}},{"name":"cni-bin2","hostPath":{"path":"/home","type":""}},{"name":"cni-conf","hostPath":{"path":"/etc","type":""}},{"name":"dbus","hostPath":{"path":"/var/lib/dbus","type":""}},{"name":"lib-modules","hostPath":{"path":"/lib/modules","type":""}},{"name":"xtables-lock","hostPath":{"path":"/run/xtables.lock","type":""}},{"name":"weave-net-token-98zhk","secret":{"secretName":"weave-net-token-98zhk","defaultMode":420}}],"containers":[{"name":"weave","image":"weaveworks/weave-kube:2.1.3","command":["/home/weave/launch.sh"],"env":[{"name":"HOSTNAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}],"resources":{"requests":{"cpu":"10m"}},"volumeMounts":[{"name":"weavedb","mountPath":"/weavedb"},{"name":"cni-bin","mountPath":"/host/opt"},{"name":"cni-bin2","mountPath":"/host/home"},{"name":"cni-conf","mountPath":"/host/etc"},{"name":"dbus","mountPath":"/host/var/lib/dbus"},{"name":"lib-modules","mountPath":"/lib/modules"},{"name":"xtables-lock","mountPath":"/run/xtables.lock"},{"name":"weave-net-token-98zhk","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"livenessProbe":{"httpGet":{"path":"/status","port":6784,"host":"127.0.0.1","scheme":"HTTP"},"initialDelaySeconds":30,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}},{"name":"weave-npc","image":"weaveworks/weave-npc:2.1.3","env":[{"name":"HOSTNAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}],"resources":{"requests":{"cpu":"10m"}},"volumeMounts":[{"name":"xtables-lock","mountPath":"/run/xtables.lock"},{"name":"weave-net-token-98zhk","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"weave-net","serviceAccount":"weave-net","nodeName":"k8s-1","hostNetwork":true,"hostPID":true,"securityContext":{"seLinuxOptions":{}},"schedulerName":"default-scheduler","tolerations":[{"operator":"Exists","effect":"NoSchedule"},{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute"},{"key":"node.kubernetes.io/disk-pressure","operator":"Exists","effect":"NoSchedule"},{"key":"node.kubernetes.io/memory-pressure","operator":"Exists","effect":"NoSchedule"}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:01:53Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:21:03Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-11T14:02:51Z"}],"hostIP":"172.16.10.19","podIP":"172.16.10.19","startTime":"2018-01-11T14:01:53Z","containerStatuses":[{"name":"weave","state":{"running":{"startedAt":"2018-01-11T14:02:42Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"weaveworks/weave-kube:2.1.3","imageID":"docker-pullable://weaveworks/weave-kube@sha256:07a3d56b8592ea3e00ace6f2c3eb7e65f3cc4945188a9e2a884b8172e6a0007e","containerID":"docker://d8b0f65d58693dfa614bd27c50c9cef27826d8f05396cb5cdf96f4b9db2014fe"},{"name":"weave-npc","state":{"running":{"startedAt":"2018-01-17T12:21:02Z"}},"lastState":{"terminated":{"exitCode":1,"reason":"Error","startedAt":"2018-01-17T09:19:58Z","finishedAt":"2018-01-17T12:21:01Z","containerID":"docker://09a322adbaec417368868f04b87d0fee5df3d1a13c00fafcf0fffce3f8d0709e"}},"ready":true,"restartCount":7,"image":"weaveworks/weave-npc:2.1.3","imageID":"docker-pullable://weaveworks/weave-npc@sha256:f35eb8166d7dae3fa7bb4d9892ab6dc8ea5c969f73791be590a0a213767c0f07","containerID":"docker://8bc8c8131a654fe6aa43442f40b2a3ad1553bb786c70afb78c690ab1ff249469"}],"qosClass":"Burstable"}}
DEBU: 2018/01/17 12:22:10.974801 EVENT AddPod {"metadata":{"name":"web-667bdcb4d8-qbq2f","generateName":"web-667bdcb4d8-","namespace":"ns-1","selfLink":"/api/v1/namespaces/ns-1/pods/web-667bdcb4d8-qbq2f","uid":"0acf7090-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673146","creationTimestamp":"2018-01-17T12:22:10Z","labels":{"app":"web","pod-template-hash":"2236876084"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"web-667bdcb4d8","uid":"0ac85530-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-9mt9k","secret":{"secretName":"default-token-9mt9k","defaultMode":420}}],"containers":[{"name":"web","image":"nginx","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-9mt9k","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","qosClass":"BestEffort"}}
DEBU: 2018/01/17 12:22:10.980233 EVENT UpdatePod {"metadata":{"name":"web-667bdcb4d8-qbq2f","generateName":"web-667bdcb4d8-","namespace":"ns-1","selfLink":"/api/v1/namespaces/ns-1/pods/web-667bdcb4d8-qbq2f","uid":"0acf7090-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673146","creationTimestamp":"2018-01-17T12:22:10Z","labels":{"app":"web","pod-template-hash":"2236876084"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"web-667bdcb4d8","uid":"0ac85530-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-9mt9k","secret":{"secretName":"default-token-9mt9k","defaultMode":420}}],"containers":[{"name":"web","image":"nginx","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-9mt9k","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","qosClass":"BestEffort"}} {"metadata":{"name":"web-667bdcb4d8-qbq2f","generateName":"web-667bdcb4d8-","namespace":"ns-1","selfLink":"/api/v1/namespaces/ns-1/pods/web-667bdcb4d8-qbq2f","uid":"0acf7090-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673151","creationTimestamp":"2018-01-17T12:22:10Z","labels":{"app":"web","pod-template-hash":"2236876084"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"web-667bdcb4d8","uid":"0ac85530-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-9mt9k","secret":{"secretName":"default-token-9mt9k","defaultMode":420}}],"containers":[{"name":"web","image":"nginx","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-9mt9k","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-2","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","conditions":[{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:22:10Z"}],"qosClass":"BestEffort"}}
DEBU: 2018/01/17 12:22:11.011394 EVENT UpdatePod {"metadata":{"name":"web-667bdcb4d8-qbq2f","generateName":"web-667bdcb4d8-","namespace":"ns-1","selfLink":"/api/v1/namespaces/ns-1/pods/web-667bdcb4d8-qbq2f","uid":"0acf7090-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673151","creationTimestamp":"2018-01-17T12:22:10Z","labels":{"app":"web","pod-template-hash":"2236876084"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"web-667bdcb4d8","uid":"0ac85530-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-9mt9k","secret":{"secretName":"default-token-9mt9k","defaultMode":420}}],"containers":[{"name":"web","image":"nginx","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-9mt9k","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-2","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","conditions":[{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:22:10Z"}],"qosClass":"BestEffort"}} {"metadata":{"name":"web-667bdcb4d8-qbq2f","generateName":"web-667bdcb4d8-","namespace":"ns-1","selfLink":"/api/v1/namespaces/ns-1/pods/web-667bdcb4d8-qbq2f","uid":"0acf7090-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673155","creationTimestamp":"2018-01-17T12:22:10Z","labels":{"app":"web","pod-template-hash":"2236876084"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"web-667bdcb4d8","uid":"0ac85530-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-9mt9k","secret":{"secretName":"default-token-9mt9k","defaultMode":420}}],"containers":[{"name":"web","image":"nginx","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-9mt9k","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-2","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:22:10Z"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:22:10Z","reason":"ContainersNotReady","message":"containers with unready status: [web]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:22:10Z"}],"hostIP":"172.16.10.15","startTime":"2018-01-17T12:22:10Z","containerStatuses":[{"name":"web","state":{"waiting":{"reason":"ContainerCreating"}},"lastState":{},"ready":false,"restartCount":0,"image":"nginx","imageID":""}],"qosClass":"BestEffort"}}
DEBU: 2018/01/17 12:22:14.184058 EVENT UpdatePod {"metadata":{"name":"web-667bdcb4d8-qbq2f","generateName":"web-667bdcb4d8-","namespace":"ns-1","selfLink":"/api/v1/namespaces/ns-1/pods/web-667bdcb4d8-qbq2f","uid":"0acf7090-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673155","creationTimestamp":"2018-01-17T12:22:10Z","labels":{"app":"web","pod-template-hash":"2236876084"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"web-667bdcb4d8","uid":"0ac85530-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-9mt9k","secret":{"secretName":"default-token-9mt9k","defaultMode":420}}],"containers":[{"name":"web","image":"nginx","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-9mt9k","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-2","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:22:10Z"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:22:10Z","reason":"ContainersNotReady","message":"containers with unready status: [web]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:22:10Z"}],"hostIP":"172.16.10.15","startTime":"2018-01-17T12:22:10Z","containerStatuses":[{"name":"web","state":{"waiting":{"reason":"ContainerCreating"}},"lastState":{},"ready":false,"restartCount":0,"image":"nginx","imageID":""}],"qosClass":"BestEffort"}} {"metadata":{"name":"web-667bdcb4d8-qbq2f","generateName":"web-667bdcb4d8-","namespace":"ns-1","selfLink":"/api/v1/namespaces/ns-1/pods/web-667bdcb4d8-qbq2f","uid":"0acf7090-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673164","creationTimestamp":"2018-01-17T12:22:10Z","labels":{"app":"web","pod-template-hash":"2236876084"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"web-667bdcb4d8","uid":"0ac85530-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-9mt9k","secret":{"secretName":"default-token-9mt9k","defaultMode":420}}],"containers":[{"name":"web","image":"nginx","ports":[{"containerPort":80,"protocol":"TCP"}],"resources":{},"volumeMounts":[{"name":"default-token-9mt9k","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-2","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:22:10Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:22:14Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:22:10Z"}],"hostIP":"172.16.10.15","podIP":"10.46.0.3","startTime":"2018-01-17T12:22:10Z","containerStatuses":[{"name":"web","state":{"running":{"startedAt":"2018-01-17T12:22:13Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"nginx:latest","imageID":"docker-pullable://nginx@sha256:285b49d42c703fdf257d1e2422765c4ba9d3e37768d6ea83d7fe2043dad6e63d","containerID":"docker://519eee92f9ee4403279e9cd4fe975822f403b6de6cbf8765e201451cd1e0882d"}],"qosClass":"BestEffort"}}
INFO: 2018/01/17 12:22:14.185373 adding entry 10.46.0.3 to weave-H]AmKZI8BX85N/tS?%rB:R%;v of 0acf7090-fb81-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:22:14.185756 added entry 10.46.0.3 to weave-H]AmKZI8BX85N/tS?%rB:R%;v of 0acf7090-fb81-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:22:14.206612 adding entry 10.46.0.3 to weave-z4cN+%s?50k!Z)QvX;[=C%d$M of 0acf7090-fb81-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:22:14.207104 added entry 10.46.0.3 to weave-z4cN+%s?50k!Z)QvX;[=C%d$M of 0acf7090-fb81-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:23:42.581614 EVENT AddNetworkPolicy {"metadata":{"name":"deny-from-other-namespaces","namespace":"ns-1","selfLink":"/apis/networking.k8s.io/v1/namespaces/ns-1/networkpolicies/deny-from-other-namespaces","uid":"416f897e-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673283","generation":1,"creationTimestamp":"2018-01-17T12:23:42Z","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"networking.k8s.io/v1\",\"kind\":\"NetworkPolicy\",\"metadata\":{\"annotations\":{},\"name\":\"deny-from-other-namespaces\",\"namespace\":\"ns-1\"},\"spec\":{\"ingress\":[{\"from\":[{\"podSelector\":{}}]}],\"podSelector\":{\"matchLabels\":null}}}\n"}},"spec":{"podSelector":{},"ingress":[{"from":[{"podSelector":{}}]}],"policyTypes":["Ingress"]}}
INFO: 2018/01/17 12:23:42.584548 adding rule: [-m set --match-set weave-H]AmKZI8BX85N/tS?%rB:R%;v src -m set --match-set weave-H]AmKZI8BX85N/tS?%rB:R%;v dst -j ACCEPT -m comment --comment pods: namespace: ns-1, selector:  -> pods: namespace: ns-1, selector: ]
DEBU: 2018/01/17 12:23:56.870779 EVENT AddPod {"metadata":{"name":"test-12363-7cdd7f8cd4-47w5j","generateName":"test-12363-7cdd7f8cd4-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/test-12363-7cdd7f8cd4-47w5j","uid":"49f272c0-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673308","creationTimestamp":"2018-01-17T12:23:56Z","labels":{"pod-template-hash":"3788394780","run":"test-12363"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"test-12363-7cdd7f8cd4","uid":"49e97a1f-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"test-12363","image":"alpine","args":["sh"],"resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always","stdin":true,"tty":true}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","qosClass":"BestEffort"}}
DEBU: 2018/01/17 12:23:56.884419 EVENT UpdatePod {"metadata":{"name":"test-12363-7cdd7f8cd4-47w5j","generateName":"test-12363-7cdd7f8cd4-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/test-12363-7cdd7f8cd4-47w5j","uid":"49f272c0-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673308","creationTimestamp":"2018-01-17T12:23:56Z","labels":{"pod-template-hash":"3788394780","run":"test-12363"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"test-12363-7cdd7f8cd4","uid":"49e97a1f-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"test-12363","image":"alpine","args":["sh"],"resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always","stdin":true,"tty":true}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","qosClass":"BestEffort"}} {"metadata":{"name":"test-12363-7cdd7f8cd4-47w5j","generateName":"test-12363-7cdd7f8cd4-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/test-12363-7cdd7f8cd4-47w5j","uid":"49f272c0-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673312","creationTimestamp":"2018-01-17T12:23:56Z","labels":{"pod-template-hash":"3788394780","run":"test-12363"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"test-12363-7cdd7f8cd4","uid":"49e97a1f-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"test-12363","image":"alpine","args":["sh"],"resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always","stdin":true,"tty":true}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-3","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","conditions":[{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:23:56Z"}],"qosClass":"BestEffort"}}
DEBU: 2018/01/17 12:23:56.911330 EVENT UpdatePod {"metadata":{"name":"test-12363-7cdd7f8cd4-47w5j","generateName":"test-12363-7cdd7f8cd4-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/test-12363-7cdd7f8cd4-47w5j","uid":"49f272c0-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673312","creationTimestamp":"2018-01-17T12:23:56Z","labels":{"pod-template-hash":"3788394780","run":"test-12363"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"test-12363-7cdd7f8cd4","uid":"49e97a1f-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"test-12363","image":"alpine","args":["sh"],"resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always","stdin":true,"tty":true}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-3","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","conditions":[{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:23:56Z"}],"qosClass":"BestEffort"}} {"metadata":{"name":"test-12363-7cdd7f8cd4-47w5j","generateName":"test-12363-7cdd7f8cd4-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/test-12363-7cdd7f8cd4-47w5j","uid":"49f272c0-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673315","creationTimestamp":"2018-01-17T12:23:56Z","labels":{"pod-template-hash":"3788394780","run":"test-12363"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"test-12363-7cdd7f8cd4","uid":"49e97a1f-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"test-12363","image":"alpine","args":["sh"],"resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always","stdin":true,"tty":true}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-3","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:23:56Z"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:23:56Z","reason":"ContainersNotReady","message":"containers with unready status: [test-12363]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:23:56Z"}],"hostIP":"172.16.10.16","startTime":"2018-01-17T12:23:56Z","containerStatuses":[{"name":"test-12363","state":{"waiting":{"reason":"ContainerCreating"}},"lastState":{},"ready":false,"restartCount":0,"image":"alpine","imageID":""}],"qosClass":"BestEffort"}}
DEBU: 2018/01/17 12:23:59.918701 EVENT UpdatePod {"metadata":{"name":"test-12363-7cdd7f8cd4-47w5j","generateName":"test-12363-7cdd7f8cd4-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/test-12363-7cdd7f8cd4-47w5j","uid":"49f272c0-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673315","creationTimestamp":"2018-01-17T12:23:56Z","labels":{"pod-template-hash":"3788394780","run":"test-12363"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"test-12363-7cdd7f8cd4","uid":"49e97a1f-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"test-12363","image":"alpine","args":["sh"],"resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always","stdin":true,"tty":true}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-3","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Pending","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:23:56Z"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:23:56Z","reason":"ContainersNotReady","message":"containers with unready status: [test-12363]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:23:56Z"}],"hostIP":"172.16.10.16","startTime":"2018-01-17T12:23:56Z","containerStatuses":[{"name":"test-12363","state":{"waiting":{"reason":"ContainerCreating"}},"lastState":{},"ready":false,"restartCount":0,"image":"alpine","imageID":""}],"qosClass":"BestEffort"}} {"metadata":{"name":"test-12363-7cdd7f8cd4-47w5j","generateName":"test-12363-7cdd7f8cd4-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/test-12363-7cdd7f8cd4-47w5j","uid":"49f272c0-fb81-11e7-882d-fa163eea23ff","resourceVersion":"673326","creationTimestamp":"2018-01-17T12:23:56Z","labels":{"pod-template-hash":"3788394780","run":"test-12363"},"ownerReferences":[{"apiVersion":"extensions/v1beta1","kind":"ReplicaSet","name":"test-12363-7cdd7f8cd4","uid":"49e97a1f-fb81-11e7-882d-fa163eea23ff","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"default-token-jp7ws","secret":{"secretName":"default-token-jp7ws","defaultMode":420}}],"containers":[{"name":"test-12363","image":"alpine","args":["sh"],"resources":{},"volumeMounts":[{"name":"default-token-jp7ws","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always","stdin":true,"tty":true}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-3","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}]},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:23:56Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:23:59Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2018-01-17T12:23:56Z"}],"hostIP":"172.16.10.16","podIP":"10.40.0.3","startTime":"2018-01-17T12:23:56Z","containerStatuses":[{"name":"test-12363","state":{"running":{"startedAt":"2018-01-17T12:23:59Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"alpine:latest","imageID":"docker-pullable://alpine@sha256:7df6db5aa61ae9480f52f0b3a06a140ab98d427f86d8d5de0bedab9b8df6b1c0","containerID":"docker://74611eaaf9ce1f60ee296d1e4c38a817e997eb76b66f8c3c03be760794bf7e36"}],"qosClass":"BestEffort"}}
INFO: 2018/01/17 12:23:59.919725 adding entry 10.40.0.3 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of 49f272c0-fb81-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:23:59.919753 added entry 10.40.0.3 to weave-k?Z;25^M}|1s7P3|H9i;*;MhG of 49f272c0-fb81-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:23:59.935127 adding entry 10.40.0.3 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of 49f272c0-fb81-11e7-882d-fa163eea23ff
INFO: 2018/01/17 12:23:59.935591 added entry 10.40.0.3 to weave-E.1.0W^NGSp]0_t5WwH/]gX@L of 49f272c0-fb81-11e7-882d-fa163eea23ff

raniatriki on 17 Jan 2018

That's strange. The iptables-save output does not correspond to the weave-npc logs (missing rules). Have you run iptables-save from the same host?

brb on 17 Jan 2018

iptables-save
# Generated by iptables-save v1.6.0 on Wed Jan 17 12:38:48 2018
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [3:180]
:POSTROUTING ACCEPT [3:180]
:DOCKER - [0:0]
:KUBE-MARK-DROP - [0:0]
:KUBE-MARK-MASQ - [0:0]
:KUBE-NODEPORTS - [0:0]
:KUBE-POSTROUTING - [0:0]
:KUBE-SEP-3DU66DE6VORVEQVD - [0:0]
:KUBE-SEP-4SZD3IGGQH7OS76W - [0:0]
:KUBE-SEP-7ZCF6XYU2MJUS3G3 - [0:0]
:KUBE-SEP-BHB36F4FNBM77ZNX - [0:0]
:KUBE-SEP-OPBRHBWOV23JTSFZ - [0:0]
:KUBE-SEP-QFC4IGGGYTWWGIXI - [0:0]
:KUBE-SEP-QN2NHVJ7ENDXQZLI - [0:0]
:KUBE-SEP-RWLNQJOMCEVSLMGE - [0:0]
:KUBE-SEP-S4MK5EVI7CLHCCS6 - [0:0]
:KUBE-SEP-UJ4S36ZD6CZMUCVN - [0:0]
:KUBE-SEP-UKQ4V7Y7PU5CFTC4 - [0:0]
:KUBE-SERVICES - [0:0]
:KUBE-SVC-4N57TFCL4MD7ZTDA - [0:0]
:KUBE-SVC-4UOFSNIU3RMN7P7P - [0:0]
:KUBE-SVC-BJM46V3U5RZHCFRZ - [0:0]
:KUBE-SVC-ERIFXISQEP7F7OF4 - [0:0]
:KUBE-SVC-JO5K3VVLGMR3IRSZ - [0:0]
:KUBE-SVC-NPX46M4PTMTKRN6Y - [0:0]
:KUBE-SVC-TCOU7JCQXEZGVUNU - [0:0]
:KUBE-SVC-XGLOHA7QRQ3V22RZ - [0:0]
:WEAVE - [0:0]
-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -j WEAVE
-A DOCKER -i docker0 -j RETURN
-A KUBE-MARK-DROP -j MARK --set-xmark 0x8000/0x8000
-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -m mark --mark 0x4000/0x4000 -j MASQUERADE
-A KUBE-SEP-3DU66DE6VORVEQVD -s 10.32.0.3/32 -m comment --comment "kube-system/kube-dns:dns" -j KUBE-MARK-MASQ
-A KUBE-SEP-3DU66DE6VORVEQVD -p udp -m comment --comment "kube-system/kube-dns:dns" -m udp -j DNAT --to-destination 10.32.0.3:53
-A KUBE-SEP-4SZD3IGGQH7OS76W -s 172.16.10.19/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ
-A KUBE-SEP-4SZD3IGGQH7OS76W -p tcp -m comment --comment "default/kubernetes:https" -m recent --set --name KUBE-SEP-4SZD3IGGQH7OS76W --mask 255.255.255.255 --rsource -m tcp -j DNAT --to-destination 172.16.10.19:6443
-A KUBE-SEP-7ZCF6XYU2MJUS3G3 -s 10.40.0.5/32 -m comment --comment "kube-system/kubernetes-dashboard:" -j KUBE-MARK-MASQ
-A KUBE-SEP-7ZCF6XYU2MJUS3G3 -p tcp -m comment --comment "kube-system/kubernetes-dashboard:" -m tcp -j DNAT --to-destination 10.40.0.5:8443
-A KUBE-SEP-BHB36F4FNBM77ZNX -s 10.46.0.1/32 -m comment --comment "default/nginx:" -j KUBE-MARK-MASQ
-A KUBE-SEP-BHB36F4FNBM77ZNX -p tcp -m comment --comment "default/nginx:" -m tcp -j DNAT --to-destination 10.46.0.1:80
-A KUBE-SEP-OPBRHBWOV23JTSFZ -s 10.40.0.2/32 -m comment --comment "default/who-is-there:" -j KUBE-MARK-MASQ
-A KUBE-SEP-OPBRHBWOV23JTSFZ -p tcp -m comment --comment "default/who-is-there:" -m tcp -j DNAT --to-destination 10.40.0.2:8080
-A KUBE-SEP-QFC4IGGGYTWWGIXI -s 10.40.0.0/32 -m comment --comment "default/who-is-there:" -j KUBE-MARK-MASQ
-A KUBE-SEP-QFC4IGGGYTWWGIXI -p tcp -m comment --comment "default/who-is-there:" -m tcp -j DNAT --to-destination 10.40.0.0:8080
-A KUBE-SEP-QN2NHVJ7ENDXQZLI -s 10.46.0.3/32 -m comment --comment "ns-1/web:" -j KUBE-MARK-MASQ
-A KUBE-SEP-QN2NHVJ7ENDXQZLI -p tcp -m comment --comment "ns-1/web:" -m tcp -j DNAT --to-destination 10.46.0.3:80
-A KUBE-SEP-RWLNQJOMCEVSLMGE -s 10.46.0.2/32 -m comment --comment "kube-system/heapster:" -j KUBE-MARK-MASQ
-A KUBE-SEP-RWLNQJOMCEVSLMGE -p tcp -m comment --comment "kube-system/heapster:" -m tcp -j DNAT --to-destination 10.46.0.2:8082
-A KUBE-SEP-S4MK5EVI7CLHCCS6 -s 10.32.0.3/32 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-MARK-MASQ
-A KUBE-SEP-S4MK5EVI7CLHCCS6 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp" -m tcp -j DNAT --to-destination 10.32.0.3:53
-A KUBE-SEP-UJ4S36ZD6CZMUCVN -s 10.40.0.1/32 -m comment --comment "default/nginx:" -j KUBE-MARK-MASQ
-A KUBE-SEP-UJ4S36ZD6CZMUCVN -p tcp -m comment --comment "default/nginx:" -m tcp -j DNAT --to-destination 10.40.0.1:80
-A KUBE-SEP-UKQ4V7Y7PU5CFTC4 -s 10.46.0.0/32 -m comment --comment "default/who-is-there:" -j KUBE-MARK-MASQ
-A KUBE-SEP-UKQ4V7Y7PU5CFTC4 -p tcp -m comment --comment "default/who-is-there:" -m tcp -j DNAT --to-destination 10.46.0.0:8080
-A KUBE-SERVICES -d 10.96.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-SVC-TCOU7JCQXEZGVUNU
-A KUBE-SERVICES -d 10.101.203.15/32 -p tcp -m comment --comment "default/who-is-there: cluster IP" -m tcp --dport 8080 -j KUBE-SVC-4UOFSNIU3RMN7P7P
-A KUBE-SERVICES -d 10.110.50.72/32 -p tcp -m comment --comment "ns-1/web: cluster IP" -m tcp --dport 80 -j KUBE-SVC-JO5K3VVLGMR3IRSZ
-A KUBE-SERVICES -d 10.98.132.43/32 -p tcp -m comment --comment "default/nginx: cluster IP" -m tcp --dport 80 -j KUBE-SVC-4N57TFCL4MD7ZTDA
-A KUBE-SERVICES -d 10.96.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
-A KUBE-SERVICES -d 10.96.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-SVC-ERIFXISQEP7F7OF4
-A KUBE-SERVICES -d 10.107.182.147/32 -p tcp -m comment --comment "kube-system/kubernetes-dashboard: cluster IP" -m tcp --dport 443 -j KUBE-SVC-XGLOHA7QRQ3V22RZ
-A KUBE-SERVICES -d 10.96.219.228/32 -p tcp -m comment --comment "kube-system/heapster: cluster IP" -m tcp --dport 80 -j KUBE-SVC-BJM46V3U5RZHCFRZ
-A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS
-A KUBE-SVC-4N57TFCL4MD7ZTDA -m comment --comment "default/nginx:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-UJ4S36ZD6CZMUCVN
-A KUBE-SVC-4N57TFCL4MD7ZTDA -m comment --comment "default/nginx:" -j KUBE-SEP-BHB36F4FNBM77ZNX
-A KUBE-SVC-4UOFSNIU3RMN7P7P -m comment --comment "default/who-is-there:" -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-QFC4IGGGYTWWGIXI
-A KUBE-SVC-4UOFSNIU3RMN7P7P -m comment --comment "default/who-is-there:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-OPBRHBWOV23JTSFZ
-A KUBE-SVC-4UOFSNIU3RMN7P7P -m comment --comment "default/who-is-there:" -j KUBE-SEP-UKQ4V7Y7PU5CFTC4
-A KUBE-SVC-BJM46V3U5RZHCFRZ -m comment --comment "kube-system/heapster:" -j KUBE-SEP-RWLNQJOMCEVSLMGE
-A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-SEP-S4MK5EVI7CLHCCS6
-A KUBE-SVC-JO5K3VVLGMR3IRSZ -m comment --comment "ns-1/web:" -j KUBE-SEP-QN2NHVJ7ENDXQZLI
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -m recent --rcheck --seconds 10800 --reap --name KUBE-SEP-4SZD3IGGQH7OS76W --mask 255.255.255.255 --rsource -j KUBE-SEP-4SZD3IGGQH7OS76W
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -j KUBE-SEP-4SZD3IGGQH7OS76W
-A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment "kube-system/kube-dns:dns" -j KUBE-SEP-3DU66DE6VORVEQVD
-A KUBE-SVC-XGLOHA7QRQ3V22RZ -m comment --comment "kube-system/kubernetes-dashboard:" -j KUBE-SEP-7ZCF6XYU2MJUS3G3
-A WEAVE -s 10.32.0.0/12 -d 224.0.0.0/4 -j RETURN
-A WEAVE ! -s 10.32.0.0/12 -d 10.32.0.0/12 -j MASQUERADE
-A WEAVE -s 10.32.0.0/12 ! -d 10.32.0.0/12 -j MASQUERADE
COMMIT
# Completed on Wed Jan 17 12:38:48 2018
# Generated by iptables-save v1.6.0 on Wed Jan 17 12:38:48 2018
*filter
:INPUT ACCEPT [333:89802]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [347:142174]
:DOCKER - [0:0]
:DOCKER-ISOLATION - [0:0]
:DOCKER-USER - [0:0]
:KUBE-FIREWALL - [0:0]
:KUBE-FORWARD - [0:0]
:KUBE-SERVICES - [0:0]
:WEAVE-NPC - [0:0]
:WEAVE-NPC-DEFAULT - [0:0]
:WEAVE-NPC-INGRESS - [0:0]
-A INPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A INPUT -j KUBE-FIREWALL
-A FORWARD -m comment --comment "kubernetes forward rules" -j KUBE-FORWARD
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o weave -j WEAVE-NPC
-A FORWARD -o weave -m state --state NEW -j NFLOG --nflog-group 86
-A FORWARD -o weave -j DROP
-A FORWARD -i weave ! -o weave -j ACCEPT
-A FORWARD -o weave -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT -j KUBE-FIREWALL
-A DOCKER-ISOLATION -j RETURN
-A DOCKER-USER -j RETURN
-A KUBE-FIREWALL -m comment --comment "kubernetes firewall for dropping marked packets" -m mark --mark 0x8000/0x8000 -j DROP
-A KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -m mark --mark 0x4000/0x4000 -j ACCEPT
-A WEAVE-NPC -m state --state RELATED,ESTABLISHED -j ACCEPT
-A WEAVE-NPC -d 224.0.0.0/4 -j ACCEPT
-A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-DEFAULT
-A WEAVE-NPC -m state --state NEW -j WEAVE-NPC-INGRESS
-A WEAVE-NPC -m set ! --match-set weave-local-pods dst -j ACCEPT
-A WEAVE-NPC-DEFAULT -m set --match-set weave-?b%zl9GIe0AET1(QI^7NWe*fO dst -m comment --comment "DefaultAllow isolation for namespace: kube-system" -j ACCEPT
-A WEAVE-NPC-DEFAULT -m set --match-set weave-0EHD/vdN#O4]V?o4Tx7kS;APH dst -m comment --comment "DefaultAllow isolation for namespace: kube-public" -j ACCEPT
-A WEAVE-NPC-DEFAULT -m set --match-set weave-z4cN+%s?50k!Z)QvX;[=C%d$M dst -m comment --comment "DefaultAllow isolation for namespace: ns-1" -j ACCEPT
-A WEAVE-NPC-DEFAULT -m set --match-set weave-E.1.0W^NGSp]0_t5WwH/]gX@L dst -m comment --comment "DefaultAllow isolation for namespace: default" -j ACCEPT
-A WEAVE-NPC-INGRESS -m set --match-set weave-H]AmKZI8BX85N/tS?%rB:R%;v src -m set --match-set weave-H]AmKZI8BX85N/tS?%rB:R%;v dst -m comment --comment "pods: namespace: ns-1, selector:  -> pods: namespace: ns-1, selector: " -j ACCEPT
COMMIT
# Completed on Wed Jan 17 12:38:48 2018

raniatriki on 17 Jan 2018

i have runned other policies betwenn the two commands maybe that's why

I updated the command iptables-save

raniatriki on 17 Jan 2018

it's ok for logs ? @brb

raniatriki on 17 Jan 2018

@raniatriki Yep, thanks. Investigating.

brb on 17 Jan 2018

👍2

I was able to reproduce the issue. For some reasons, after the netpol has been created, the 10.46.0.3 PodIP of ns-1/web is not removed from ns-1/Default-Allow ipset(weave-z4cN+%s?50k!Z)QvX;[=C%d$M).

brb on 1 Feb 2018

Fixed by #3237