Core: 0.103 error during setup huawei_lte

[tomasz8w]

Home Assistant release with the issue:
0.103

Last working Home Assistant release (if known):
0.102.3

Operating environment (Hass.io/Docker/Windows/etc.):
Ubuntu 19.10 / hassio / Docker

Integration:
https://www.home-assistant.io/integrations/huawei_lte/

Description of problem:
After upgrade to 0.103 integration fails during setup. It happens also in authenticated mode.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

huawei_lte:
  - url: http://192.168.x.x

Traceback (if applicable):

Error setting up entry B525s-23a for huawei_lte
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.7/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/local/lib/python3.7/http/client.py", line 306, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.7/http/client.py", line 275, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 720, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.7/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/local/lib/python3.7/http/client.py", line 306, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.7/http/client.py", line 275, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 192, in async_setup
    hass, self
  File "/usr/src/homeassistant/homeassistant/components/huawei_lte/__init__.py", line 288, in async_setup_entry
    connection = await hass.async_add_executor_job(get_connection)
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/huawei_lte/__init__.py", line 277, in get_connection
    url, username=username, password=password, timeout=CONNECTION_TIMEOUT
  File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/AuthorizedConnection.py", line 24, in __init__
    if self.user.login(True):
  File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/api/User.py", line 76, in login
    state_login = self.state_login()
  File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/api/User.py", line 27, in state_login
    return self._connection.get('user/state-login')
  File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/Connection.py", line 22, in wrapped
    return fn(*args, **kw)
  File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/Connection.py", line 166, in get
    timeout=self.timeout,
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Additional information:

[tomasz8w]

Ok, I didn't notice it earlier but with credentials provided, last part of traceback is little different and it shows up few minutes after boot.
It looks that despite the error occurs every few minutes, device_tracker works so far.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/huawei_lte/__init__.py", line 351, in _update_router
    router.update()
  File "/usr/src/homeassistant/homeassistant/components/huawei_lte/__init__.py", line 175, in update
    get_data(KEY_DEVICE_INFORMATION, self.client.device.information)
  File "/usr/src/homeassistant/homeassistant/components/huawei_lte/__init__.py", line 161, in get_data
    self.data[key] = func()
  File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/api/Device.py", line 7, in information
    return self._connection.get('device/information')
  File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/Connection.py", line 22, in wrapped
    return fn(*args, **kw)
  File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/Connection.py", line 166, in get
    timeout=self.timeout,
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Connection lost. Reconnecting…

[probot-home-assistant[bot]]

Hey there @scop, mind taking a look at this issue as its been labeled with a integration (huawei_lte) you are listed as a codeowner for? Thanks!

[adriangarbacea]

i have the same issue which appeared after 103 upgrade
Sun Dec 22 2019 14:06:11 GMT+0200 (Eastern European Standard Time)
Unknown error connecting to device
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
httplib_response = conn.getresponse()
File "/usr/local/lib/python3.7/http/client.py", line 1344, in getresponse
response.begin()
File "/usr/local/lib/python3.7/http/client.py", line 306, in begin
version, status, reason = self._read_status()
File "/usr/local/lib/python3.7/http/client.py", line 275, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 400, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 734, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
httplib_response = conn.getresponse()
File "/usr/local/lib/python3.7/http/client.py", line 1344, in getresponse
response.begin()
File "/usr/local/lib/python3.7/http/client.py", line 306, in begin
version, status, reason = self._read_status()
File "/usr/local/lib/python3.7/http/client.py", line 275, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/huawei_lte/config_flow.py", line 178, in async_step_user
try_connect, username, password
File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(self.args, *self.kwargs)
File "/usr/src/homeassistant/homeassistant/components/huawei_lte/config_flow.py", line 133, in try_connect
timeout=CONNECTION_TIMEOUT,
File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/AuthorizedConnection.py", line 24, in __init__
if self.user.login(True):
File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/api/User.py", line 76, in login
state_login = self.state_login()
File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/api/User.py", line 27, in state_login
return self._connection.get('user/state-login')
File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/Connection.py", line 22, in wrapped
return fn(args, *kw)
File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/Connection.py", line 166, in get
timeout=self.timeout,
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 546, in get
return self.request('GET', url, *kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, *send_kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Connection lost. Reconnecting…

[scop]

29531 and #29597 (will appear in 0.104) contain some fixes that may have some effect on this. If you have a chance, testing with the current dev branch would be useful.

Also, while testing, make sure that you are not logged in to the router web interface from a browser. Being logged in from there will interfere with the HA integration (and vice versa).

[tomasz8w]

I moved to dev with #29597 fix just after opening this issue but there is still something wrong. My observation - this error happens randomly while restarting home assistant. Sometimes I have to restart few times in a row to get it working but when it is loaded properly on the beginning, it works. Even if errors occurs afterwards, it doesn't break the functionality.

[adriangarbacea]

Also, while testing, make sure that you are not logged in to the router web interface from a browser. Being logged in from there will interfere with the HA integration (and vice versa).

now that you mentioned this, i realized that i have huawei-lte integration in 2 homeassistant instances, probably interfering each other. after i stopped the old one, i have this error only sporadically, after several server restarts, but mainly now is working ok.

[oneteam2win]

I fear I have a similar or the same problem.
Since a few days (I think after the update from 102.x to 103.x) it is no longer possible to connect my Huawei LTE modem.
This is an HA installation on a Synology. The Huawei LTE modem is a B529s-23a.
And everything has worked fine.
Can anyone help me solve the problem or do you need more information about my system?
I always get the following error message. No matter if I do the Huawei integration via the frontend or the config.yaml.

[homeassistant.components.huawei_lte.config_flow] Unknown error connecting to device
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
httplib_response = conn.getresponse()
File "/usr/local/lib/python3.7/http/client.py", line 1344, in getresponse
response.begin()
File "/usr/local/lib/python3.7/http/client.py", line 306, in begin
version, status, reason = self._read_status()
File "/usr/local/lib/python3.7/http/client.py", line 275, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 400, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 734, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
httplib_response = conn.getresponse()
File "/usr/local/lib/python3.7/http/client.py", line 1344, in getresponse
response.begin()
File "/usr/local/lib/python3.7/http/client.py", line 306, in begin
version, status, reason = self._read_status()
File "/usr/local/lib/python3.7/http/client.py", line 275, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

[scop]

Any different with 0.104?

[tomasz8w]

No change after upgrade, same issues like before

[blissarts]

Exactly the same issue on my end.
Last working in 0.102.3 not working ever since

[blissarts]

Unfortunately I'm having still the same issue on 105.2

[scop]

Here's a little script that can be used to debug the issue. It simply tries to log in and out of the router, and dumps some ugly info about the requests and responses:

#!/usr/bin/env python3

import http.client
from huawei_lte_api.AuthorizedConnection import AuthorizedConnection
from huawei_lte_api.Connection import Connection
from huawei_lte_api.Client import Client

URL = "http://192.168.1.1/"
USERNAME = "admin"
PASSWORD = ""
CONNECTION_TIMEOUT = 10

http.client.HTTPConnection.debuglevel = 1

if USERNAME or PASSWORD:
    connection = AuthorizedConnection(
        URL, username=USERNAME, password=PASSWORD, timeout=CONNECTION_TIMEOUT
    )
else:
    connection = Connection(URL, timeout=CONNECTION_TIMEOUT)

if isinstance(connection, AuthorizedConnection):
    Client(connection).user.logout()

Note that while the output does not contain the password in cleartext, it may contain some info that is not good to post in public. Which unfortunately also makes debugging the issue harder.

Anyway, try it out with similar URL, USERNAME, and PASSWORD settings as you have in HA. DO NOT POST THE OUTPUT HERE because of the above, just look at it and report if it fails for you, and if it fails, how (i.e. if you have USERNAME and PASSWORD set to valid values or if they are empty or None (which means unauthenticated access)).

If it fails with valid configurations which I suspect it does, this will likely need to be reported and handled in huawei_lte_api rather than the HA huawei_lte integration.

It could be also useful to test with different huawei_lte_api versions, e.g. 1.2, 1.3, and 1.4.7 if you know how to do it.

[tomasz8w]

I did some tests with all versions after 1.2 and till 1.4.3 script does not return any error. With 1.4.4 and newer, it throws error with credentials provided.

[scop]

Thanks for testing! This confirms that the fix should be made in huawei_lte_api. Between 1.4.3 and 1.4.4 we changed it to use a persistent session, which is really the right thing to do, and that's what browser also do anyway. But apparently there's something some modems don't like something about it.

Could you file a report about this at https://github.com/Salamek/huawei-lte-api/ along with your version findings and the script, and @ me there? I would report this myself, but as I can't reproduce, I wouldn't be able to provide more details upstream will likely need.

[tomasz8w]

Sure, issue opened. I'm glad I can help in some sort of way.

[tomasz8w]

Is there any chance to upgrade huawei-lte-api to 1.4.11 in current (0.107) release? 1.4.10 throws error because time import is missing ( https://github.com/Salamek/huawei-lte-api/pull/72 )

[scop]

The upgrade was done in https://github.com/home-assistant/core/pull/32791 already, but looks like it's missing still from 0.107.4. I've added it to the 0.107.5 milestone at https://github.com/home-assistant/core/milestone/298, hopefully that's the correct procedure for requesting it.