Python: No access to API server from pod in python

Created on 14 Jun 2017 · 26Comments · Source: kubernetes-client/python

Trying to access api service from the pod in python:

from kubernetes import client, config
core_v1 = client.CoreV1Api()
config.load_incluster_config()
core_v1.list_node()

results in following error:

# python
Python 2.7.5 (default, Nov  6 2016, 00:28:07) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> from kubernetes import client, config
>>> core_v1 = client.CoreV1Api()
>>> config.load_incluster_config()
>>> core_v1.list_node()         
2017-06-14 13:52:41,486 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x36c0e50>: Failed to establish a new connection: [Errno 111] Connection refused',)': /api/v1/nodes
2017-06-14 13:52:41,487 WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x36b5b50>: Failed to establish a new connection: [Errno 111] Connection refused',)': /api/v1/nodes
2017-06-14 13:52:41,487 WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x36b5c90>: Failed to establish a new connection: [Errno 111] Connection refused',)': /api/v1/nodes
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/kubernetes/client/apis/core_v1_api.py", line 13284, in list_node
  File "build/bdist.linux-x86_64/egg/kubernetes/client/apis/core_v1_api.py", line 13377, in list_node_with_http_info
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 329, in call_api
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 153, in __call_api
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 361, in request
  File "build/bdist.linux-x86_64/egg/kubernetes/client/rest.py", line 240, in GET
  File "build/bdist.linux-x86_64/egg/kubernetes/client/rest.py", line 214, in request
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/request.py", line 66, in request
    **urlopen_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/request.py", line 87, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/poolmanager.py", line 321, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/connectionpool.py", line 678, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/connectionpool.py", line 678, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/connectionpool.py", line 678, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/connectionpool.py", line 649, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /api/v1/nodes (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x36b5d90>: Failed to establish a new connection: [Errno 111] Connection refused',))

For some reason HTTPSConnectionPool is trying to access localhost instead of API server, though API url in kubernetes configuration seems to be correct:

>>> from kubernetes.client import configuration
>>> configuration.host
'https://10.0.0.1:443'

At the same time curl is working perfectly well:
curl -v --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" https://kubernetes/

Source

sutasu

Most helpful comment

I was getting this same error this afternoon from this sequence:

from kubernetes import client, config
config.load_incluster_config()
core_v1 = client.CoreV1Api()
ret = v1.list_namespaced_pod("my-namespace")

(I had some other stuff and several print statements)

No amount of messing about with python libs helped. However, there were the "WARNING Retrying" logs starting right at the beginning.
So, importing the client AFTER the config is done worked:

from kubernetes import config
config.load_incluster_config()
from kubernetes import client
core_v1 = client.CoreV1Api()
ret = v1.list_namespaced_pod("my-namespace")

awootton on 18 Aug 2017

👍7 😄3

All 26 comments

Probably the sequence of the command has to be reversed:

from kubernetes import client, config
config.load_incluster_config()
core_v1 = client.CoreV1Api()
core_v1.list_node()

Still getting following:

2017-06-14 20:35:46,197 ERROR Certificate did not match expected hostname: 10.0.0.1. Certificate: {'subjectAltName': (('DNS', 'kubernetes.default.svc.cluster.local'), ('DNS', 'kubernetes.default.svc'), ('DNS', 'kubernetes.default'), ('DNS', 'kubernetes'), ('IP Address', '192.168.99.101'), ('IP Address', '10.0.0.1')), 'notBefore': u'Jun 14 13:28:30 2017 GMT', 'serialNumber': u'02', 'notAfter': 'Jun 14 13:28:30 2018 GMT', 'version': 3L, 'subject': ((('organizationName', u'system:masters'),), (('commonName', u'minikube'),)), 'issuer': ((('commonName', u'minikubeCA'),),)}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/kubernetes/client/apis/core_v1_api.py", line 13284, in list_node
  File "build/bdist.linux-x86_64/egg/kubernetes/client/apis/core_v1_api.py", line 13377, in list_node_with_http_info
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 329, in call_api
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 153, in __call_api
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 361, in request
  File "build/bdist.linux-x86_64/egg/kubernetes/client/rest.py", line 240, in GET
  File "build/bdist.linux-x86_64/egg/kubernetes/client/rest.py", line 217, in request
kubernetes.client.rest.ApiException: (0)
Reason: SSLError
hostname '10.0.0.1' doesn't match either of 'kubernetes.default.svc.cluster.local', 'kubernetes.default.svc', 'kubernetes.default', 'kubernetes'

sutasu on 14 Jun 2017

👍1

Check https://github.com/kubernetes-incubator/client-python#hostname-doesnt-match

(btw, this was a bug in 2.0 that requirements are not all getting installed with the version we expected. should be fixed in 3.0)

mbohlool on 15 Jun 2017

Still the same behavior with following packages which seem to satisfy version requirements:

certifi==2017.4.17
six==1.10.0
python-dateutil==2.6.0
setuptools-21.0.0
urllib3==1.20
PyYAML==3.12
oauth2client==4.1.1
ipaddress==1.0.18
websocket-client==0.40.0

sutasu on 16 Jun 2017

deleted previous comment. I saw you have platform information above. Is this standard distribution or something like anaconda? Can you give it a try on python 2.7.12?

mbohlool on 16 Jun 2017

@mbohlool Same issue. Some version info as below:

Python 2.7.12 :: Anaconda 4.2.0 (64-bit)
urllib3 version: 1.19.1
ipaddress version: 1.0.18
Kubernetes version: 1.6.5

I can't even load the incluster config.

>>> config.load_incluster_config()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mind/anaconda2/lib/python2.7/site-packages/kubernetes/config/incluster_config.py", line 91, in load_incluster_config
    cert_filename=SERVICE_CERT_FILENAME).load_and_set()
  File "/home/mind/anaconda2/lib/python2.7/site-packages/kubernetes/config/incluster_config.py", line 45, in load_and_set
    self._load_config()
  File "/home/mind/anaconda2/lib/python2.7/site-packages/kubernetes/config/incluster_config.py", line 51, in _load_config
    raise ConfigException("Service host/port is not set.")
kubernetes.config.config_exception.ConfigException: Service host/port is not set.

If I load kube config, it doesn't work either.

>>> config.load_kube_config('/home/mind/admin.conf')
>>> core_v1 = client.CoreV1Api()
>>> core_v1.list_node()
2017-06-21 15:41:25,573 ERROR Certificate did not match expected hostname: 9.96.101.174. Certificate: {'subjectAltName': (('DNS', '9-96-101-174'), ('DNS', 'kubernetes'), ('DNS', 'kubernetes.default'), ('DNS', 'kubernetes.default.svc'), ('DNS', 'kubernetes.default.svc.cluster.local'), ('IP Address', '10.96.0.1'), ('IP Address', '9.96.101.174')), 'notBefore': u'Jun 20 01:59:12 2017 GMT', 'serialNumber': u'7D78506EA7A03E76', 'notAfter': 'Jun 20 01:59:12 2018 GMT', 'version': 3L, 'subject': ((('commonName', u'kube-apiserver'),),), 'issuer': ((('commonName', u'kubernetes'),),)}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mind/anaconda2/lib/python2.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 13284, in list_node
    (data) = self.list_node_with_http_info(**kwargs)
  File "/home/mind/anaconda2/lib/python2.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 13377, in list_node_with_http_info
    collection_formats=collection_formats)
  File "/home/mind/anaconda2/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 329, in call_api
    _return_http_data_only, collection_formats, _preload_content, _request_timeout)
  File "/home/mind/anaconda2/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 153, in __call_api
    _request_timeout=_request_timeout)
  File "/home/mind/anaconda2/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 361, in request
    headers=headers)
  File "/home/mind/anaconda2/lib/python2.7/site-packages/kubernetes/client/rest.py", line 240, in GET
    query_params=query_params)
  File "/home/mind/anaconda2/lib/python2.7/site-packages/kubernetes/client/rest.py", line 217, in request
    raise ApiException(status=0, reason=msg)
kubernetes.client.rest.ApiException: (0)
Reason: SSLError
hostname '9.96.101.174' doesn't match either of '9-96-101-174', 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local'

DjangoPeng on 21 Jun 2017

Anaconda is not supported yet. Have you tried it on standard distribution?

mbohlool on 21 Jun 2017

I tried to add --insecure-skip-tls-verify=true flag for kubernetes cluster, and it can avoid the error. I don't know why the client is not supported anaconda. Is there a gap between anaconda and standard distribution?

DjangoPeng on 22 Jun 2017

165

mbohlool on 22 Jun 2017

@mbohlool Got, Thanks!

DjangoPeng on 23 Jun 2017

👍1

I was getting this same error this afternoon from this sequence:

from kubernetes import client, config
config.load_incluster_config()
core_v1 = client.CoreV1Api()
ret = v1.list_namespaced_pod("my-namespace")

(I had some other stuff and several print statements)

No amount of messing about with python libs helped. However, there were the "WARNING Retrying" logs starting right at the beginning.
So, importing the client AFTER the config is done worked:

from kubernetes import config
config.load_incluster_config()
from kubernetes import client
core_v1 = client.CoreV1Api()
ret = v1.list_namespaced_pod("my-namespace")

awootton on 18 Aug 2017

👍7 😄3

That is strange. It should be a bug. Importing client should not have any effect on config unless you create a client. Would you mind to file a separate issue for this, so somebody debug it? Include as much logs as possible in both cases.

mbohlool on 18 Aug 2017

@mbohlool I prepared a simplified test this morning and it failed to reproduce the problem.
Frack.

awootton on 19 Aug 2017

That is ok. Please file an issue if it happened again and you could reproduce it.

mbohlool on 29 Aug 2017

Same problem

faultuser on 9 Sep 2017

@fr7 Can you send a reproducible example? Curious.

awootton on 9 Sep 2017

I checked the requirements，but it still does not work.

[root@cronjob-4165617455-cjrc4 ~]# pip show kubernetes                              
Name: kubernetes
Version: 3.0.0
Summary: Kubernetes python client
Home-page: https://github.com/kubernetes-incubator/client-python
Author: Kubernetes
Author-email: UNKNOWN
License: Apache License Version 2.0
Location: /usr/lib/python2.7/site-packages
Requires: google-auth, urllib3, requests, setuptools, ipaddress, six, websocket-client, python-dateutil, 
pyyaml, certifi


[root@cronjob-4165617455-cjrc4 ~]# pip install -r py.txt 
Requirement already satisfied: certifi>=14.05.14 in /usr/lib/python2.7/site-packages (from -r py.txt 
(line 1))
Requirement already satisfied: six>=1.9.0 in /usr/lib/python2.7/site-packages (from -r py.txt (line 2))
Requirement already satisfied: python-dateutil>=2.5.3 in /usr/lib/python2.7/site-packages (from -r py.txt (line 3))
Requirement already satisfied: setuptools>=21.0.0 in /usr/lib/python2.7/site-packages (from -r py.txt (line 4))
Requirement already satisfied: urllib3!=1.21,>=1.19.1 in /usr/lib/python2.7/site-packages (from -r py.txt (line 5))
Requirement already satisfied: pyyaml>=3.12 in /usr/lib64/python2.7/site-packages (from -r py.txt (line 6))
Requirement already satisfied: google-auth>=1.0.1 in /usr/lib/python2.7/site-packages (from -r py.txt (line 7))
Requirement already satisfied: ipaddress>=1.0.17 in /usr/lib/python2.7/site-packages (from -r py.txt (line 8))
Requirement already satisfied: websocket-client<=0.40.0,>=0.32.0 in /usr/lib/python2.7/site-packages (from -r py.txt (line 9))
Requirement already satisfied: requests in /usr/lib/python2.7/site-packages (from -r py.txt (line 10))
Requirement already satisfied: requests-oauthlib in /usr/lib/python2.7/site-packages (from -r py.txt (line 11))
Requirement already satisfied: pyasn1>=0.1.7 in /usr/lib/python2.7/site-packages (from google-auth>=1.0.1->-r py.txt (line 7))
Requirement already satisfied: pyasn1-modules>=0.0.5 in /usr/lib/python2.7/site-packages (from google-auth>=1.0.1->-r py.txt (line 7))
Requirement already satisfied: cachetools>=2.0.0 in /usr/lib/python2.7/site-packages (from google-auth>=1.0.1->-r py.txt (line 7))
Requirement already satisfied: rsa>=3.1.4 in /usr/lib/python2.7/site-packages (from google-auth>=1.0.1->-r py.txt (line 7))
Requirement already satisfied: backports.ssl_match_hostname in /usr/lib/python2.7/site-packages (from websocket-client<=0.40.0,>=0.32.0->-r py.txt (line 9))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python2.7/site-packages (from requests->-r py.txt (line 10))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/lib/python2.7/site-packages (from requests->-r py.txt (line 10))
Requirement already satisfied: oauthlib>=0.6.2 in /usr/lib/python2.7/site-packages (from requests-oauthlib->-r py.txt (line 11))


[root@cronjob-4165617455-cjrc4 ~]# python
Python 2.7.5 (default, Nov  6 2016, 00:28:07) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from kubernetes import config, client
>>> config.load_incluster_config()
>>> v1 = client.CoreV1Api()
>>> v1.list_node()
2017-09-09 10:48:33,146 ERROR Certificate did not match expected hostname: 10.254.0.1. Certificate: {'subjectAltName': (('IP Address', '172.17.0.1'), ('IP Address', '172.22.117.111'), ('IP Address', '172.22.117.101'), ('IP Address', '10.254.0.1'), ('DNS', 'kubernetes'), ('DNS', 'kubernetes.default'), ('DNS', 'kubernetes.default.svc'), ('DNS', 'kubernetes.default.svc.cluster.local'), ('DNS', '172.22.117.101')), 'notBefore': u'Aug  5 08:03:47 2017 GMT', 'serialNumber': u'01', 'notAfter': 'Aug  3 08:03:47 2027 GMT', 'version': 3L, 'subject': ((('commonName', u'172.22.117.101'),),), 'issuer': ((('commonName', u'172.17.0.1,IP:172.22.117.111,IP:172.22.117.101@1501920227'),),)}
2017-09-09 10:48:33,147 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(CertificateError("hostname '10.254.0.1' doesn't match either of 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local', '172.22.117.101'",),)': /api/v1/nodes
2017-09-09 10:48:33,162 ERROR Certificate did not match expected hostname: 10.254.0.1. Certificate: {'subjectAltName': (('IP Address', '172.17.0.1'), ('IP Address', '172.22.117.111'), ('IP Address', '172.22.117.101'), ('IP Address', '10.254.0.1'), ('DNS', 'kubernetes'), ('DNS', 'kubernetes.default'), ('DNS', 'kubernetes.default.svc'), ('DNS', 'kubernetes.default.svc.cluster.local'), ('DNS', '172.22.117.101')), 'notBefore': u'Aug  5 08:03:47 2017 GMT', 'serialNumber': u'01', 'notAfter': 'Aug  3 08:03:47 2027 GMT', 'version': 3L, 'subject': ((('commonName', u'172.22.117.101'),),), 'issuer': ((('commonName', u'172.17.0.1,IP:172.22.117.111,IP:172.22.117.101@1501920227'),),)}
2017-09-09 10:48:33,163 WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(CertificateError("hostname '10.254.0.1' doesn't match either of 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local', '172.22.117.101'",),)': /api/v1/nodes
2017-09-09 10:48:33,177 ERROR Certificate did not match expected hostname: 10.254.0.1. Certificate: {'subjectAltName': (('IP Address', '172.17.0.1'), ('IP Address', '172.22.117.111'), ('IP Address', '172.22.117.101'), ('IP Address', '10.254.0.1'), ('DNS', 'kubernetes'), ('DNS', 'kubernetes.default'), ('DNS', 'kubernetes.default.svc'), ('DNS', 'kubernetes.default.svc.cluster.local'), ('DNS', '172.22.117.101')), 'notBefore': u'Aug  5 08:03:47 2017 GMT', 'serialNumber': u'01', 'notAfter': 'Aug  3 08:03:47 2027 GMT', 'version': 3L, 'subject': ((('commonName', u'172.22.117.101'),),), 'issuer': ((('commonName', u'172.17.0.1,IP:172.22.117.111,IP:172.22.117.101@1501920227'),),)}
2017-09-09 10:48:33,178 WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(CertificateError("hostname '10.254.0.1' doesn't match either of 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local', '172.22.117.101'",),)': /api/v1/nodes
2017-09-09 10:48:33,193 ERROR Certificate did not match expected hostname: 10.254.0.1. Certificate: {'subjectAltName': (('IP Address', '172.17.0.1'), ('IP Address', '172.22.117.111'), ('IP Address', '172.22.117.101'), ('IP Address', '10.254.0.1'), ('DNS', 'kubernetes'), ('DNS', 'kubernetes.default'), ('DNS', 'kubernetes.default.svc'), ('DNS', 'kubernetes.default.svc.cluster.local'), ('DNS', '172.22.117.101')), 'notBefore': u'Aug  5 08:03:47 2017 GMT', 'serialNumber': u'01', 'notAfter': 'Aug  3 08:03:47 2027 GMT', 'version': 3L, 'subject': ((('commonName', u'172.22.117.101'),),), 'issuer': ((('commonName', u'172.17.0.1,IP:172.22.117.111,IP:172.22.117.101@1501920227'),),)}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 13990, in list_node
(data) = self.list_node_with_http_info(**kwargs)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 14086, in 
list_node_with_http_info
collection_formats=collection_formats)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 335, in call_api
_preload_content, _request_timeout)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 148, in __call_api
_request_timeout=_request_timeout)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 371, in request
headers=headers)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/rest.py", line 250, in GET
query_params=query_params)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/rest.py", line 223, in request
headers=headers)
  File "/usr/lib/python2.7/site-packages/urllib3/request.py", line 66, in request
**urlopen_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/request.py", line 87, in request_encode_url
return self.urlopen(method, url, **extra_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/poolmanager.py", line 321, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
**response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
**response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
**response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
  File "/usr/lib/python2.7/site-packages/urllib3/util/retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='10.254.0.1', port=443): Max retries exceeded with url: /api/v1/nodes (Caused by SSLError(CertificateError("hostname '10.254.0.1' doesn't match either of 'kubernetes', 'kubernetes.default', 'kubernetes.default.svc', 'kubernetes.default.svc.cluster.local', '172.22.117.101'",),))

faultuser on 9 Sep 2017

getting the same issue on google container engine...
tried to use os.environ['KUBERNETES_SERVICE']= 'kubernetes' but did nothing

karmab on 17 Oct 2017

@karmab can you provide the code plus the error you are getting and the version of python pand python packages?

mbohlool on 18 Oct 2017

the code is here https://github.com/karmab/samplecontroller/blob/master/controller.py
i m also experiencing repeated crashes ( in the watch loop i d say)
i m running within a centos container with python2.7 and commit 2c0bed9c4f653472289324914a8f0ad4cbb3a1cb

karmab on 18 Oct 2017

I don't think this issue is in watch. Could be related or not but if you give more details about version of python packages installed and some of the error messages you are getting, I maybe able to help.

mbohlool on 18 Oct 2017

so technically i m using

python 2.7.10
client-python.git@2c0bed9c4f653472289324914a8f0ad4cbb3a1cb

i dont get any traceback, the pod simply dies.
i have tried to workaround it adding _request_timeout=31536000 as parameter to the stream method but no luck

karmab on 18 Oct 2017

i ve also tried with latest git version of the client python
and same result
the pod goes into state completed without any additional information...

karmab on 18 Oct 2017

If you don't get an error message with traceback, this is more like a kubernetes question/issue than this client. To debug this, you need to do normal kubernetes debugging probably with kubectl (or even this client) by getting the logs of the dying pod and investigate it. If you enable Configuration.debug, you will also get more information on what client calls are and what api-server responses are. You can do the same with kubectl by -v9 flag.

mbohlool on 18 Oct 2017

@mbohlool am getting problem while creating a service in kubernetes. The problem is of certification error and due to this i got connection refused error.

`Traceback (most recent call last):
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
conn.connect()
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/connection.py", line 284, in connect
conn = self._new_conn()
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./tests/kubernetes_service_test.py", line 6, in
kbs.create_service()
File "/home/hushenbeg/devzone/aeroflow-dags/aerolib/core/kubernetes_service.py", line 64, in create_service
api_response = api_instance.create_namespaced_service(namespace, body, pretty=pretty)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 6820, in create_namespaced_service
(data) = self.create_namespaced_service_with_http_info(namespace, body, *kwargs)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 6905, in create_namespaced_service_with_http_info
collection_formats=collection_formats)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 321, in call_api
_return_http_data_only, collection_formats, _preload_content, _request_timeout)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 155, in __call_api
_request_timeout=_request_timeout)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 364, in request
body=body)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 266, in POST
body=body)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 166, in request
headers=headers)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/request.py", line 70, in request
*urlopen_kw)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/request.py", line 148, in request_encode_body
return self.urlopen(method, url, *extra_kw)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/poolmanager.py", line 321, in urlopen
response = conn.urlopen(method, u.request_uri, *kw)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 668, in urlopen
*response_kw)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 668, in urlopen
*response_kw)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 668, in urlopen
**response_kw)
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/home/hushenbeg/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /api/v1/namespaces/namespace_example/services?pretty=Welcome+to+stackaero+kubernetes (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))`

hushenbeg on 1 Jun 2018

👍2

I have to access the Kubernetes proxy API server from other machine using python library functions. Is this possible ?

vj319 on 18 Jan 2019

This worked for me:

config.load_incluster_config() #get the config from within the cluster and set it as the default config for all new clients
c=client.Configuration() #go and get a copy of the default config
c.verify_ssl=False #set verify_ssl to false in that config
client.Configuration.set_default(c) #make that config the default for all new clients
v1 = client.CoreV1Api()