Core: 0.40.0 broke mqtt

[joel-garnick]

Make sure you are running the latest version of Home Assistant before reporting an issue.

You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:

Home Assistant release (hass --version):
0.40.0

Python release (python3 --version):
3.5.2

Component/platform:
mqtt

Description of problem:
Cannot set up mqtt component or components dependent on it (switch.mqtt, mqtt-eventstream, device_tracker.owntracks)

Expected:
mqtt component should load (it loads fine if i roll back to 0.39.3)

Problem-relevant configuration.yaml entries and steps to reproduce:

mqtt:
  broker: <ip>
  port: <port>
  client_id: <id>
  keepalive: 60
  username: <username>
  password: <password>

mqtt_eventstream:
  publish_topic: <publish topic>
  subscribe_topic: <subscribe topic>

switch:
  - platform: mqtt
    name: Switch 1
    state_topic: water/switch.switch_1/state
    command_topic: water/switch.switch_1/command
    payload_on: 'on'
    payload_off: 'off'
  - platform: mqtt
    name: Switch 2
    state_topic: water/switch.switch_2/state
    command_topic: water/switch.switch_2/command
    payload_on: 'on'
    payload_off: 'off'
  - platform: mqtt
    name: Switch 3
    state_topic: water/switch.switch_3/state
    command_topic: water/switch.switch_3/command
    payload_on: 'on'
    payload_off: 'off'
  - platform: mqtt
    name: Switch 4
    state_topic: water/switch.switch_4/state
    command_topic: water/switch.switch_4/command
    payload_on: 'on'
    payload_off: 'off'
  - platform: mqtt
    name: Switch 5
    state_topic: water/switch.switch_5/state
    command_topic: water/switch.switch_5/command
    payload_on: 'on'
    payload_off: 'off'
  - platform: mqtt
    name: Switch 6
    state_topic: water/switch.switch_6/state
    command_topic: water/switch.switch_6/command
    payload_on: 'on'
    payload_off: 'off'
  - platform: mqtt
    name: Switch 7
    state_topic: water/switch.switch_7/state
    command_topic: water/switch.switch_7/command
    payload_on: 'on'
    payload_off: 'off'
  - platform: mqtt
    name: Switch 8
    state_topic: water/switch.switch_8/state
    command_topic: water/switch.switch_8/command
    payload_on: 'on'
    payload_off: 'off'

1. update to 0.40.0
2. start home-assistant service
3. get persistent notification that mqtt and dependent components can't be loaded

Traceback (if applicable):

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/homeassistant/setup.py", line 184, in _async_setup_component
    result = yield from component.async_setup(hass, processed_config)
  File "/usr/local/lib/python3.5/dist-packages/homeassistant/components/mqtt/__init__.py", line 338, in async_setup
    success = yield from hass.data[DATA_MQTT].async_connect()
  File "/usr/local/lib/python3.5/dist-packages/homeassistant/components/mqtt/__init__.py", line 446, in async_connect
    None, self._mqttc.connect, self.broker, self.port, self.keepalive)
  File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
    future.result()
  File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 686, in connect
    return self.reconnect()
  File "/usr/local/lib/python3.5/dist-packages/paho/mqtt/client.py", line 821, in reconnect
    ciphers=self._tls_ciphers)
  File "/usr/lib/python3.5/ssl.py", line 1069, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python3.5/ssl.py", line 752, in __init__
    self.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 988, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake
    self._sslobj.do_handshake()

Additional info:

[Toonix]

Owntrack is working for me with 0.40

[bdurrer]

The ssl handshake seems to fail, did some default settings change from 39 to 40?

Probably a wrong lead, but according to the docs we better define the protocol:

There is an issue with the Mosquitto package included in Ubuntu 14.04 LTS. Specify protocol: 3.1 in your MQTT configuration to work around this issue.

[mf-social]

Indeed, most likely mosquito, I am using cloudMQTT and have no problems in 0.40

[tklucis]

Specifying protocol does not help.

[printzlau]

It seems that it is no longer allowed to use the default SSL port (8883) without actually using SSL.
I fixed mine by adding "certificate = None" after line 316 in mqtt/__init__.py
I will move away from 8883 at a later point when I have made a firmware upgrade for all my devices.
But I also think that the hass mqtt implementation should be changed to only use SSL if specified by a parameter in the config.

[tklucis]

Yep. Indeed. Your solution works. Thank You printzlau.

[joel-garnick]

@printzlau Yep, that did it...i just switched off of the 8883 port to a different one and now everything is working. Thanks!

[bdurrer]

Well 8883 is the standard port for mqtt tls and 1883 is for non-tls, so I guess it kind of makes sense, but at least there should be a warning somewhere.

For the record: I installed mosquitto today, running on 8883 _with_ ssl and it works fine.

[tklucis]

ROFL

[printzlau]

I disagree. Providing a port number should not effect anything but the port number. If specifying the port number actually specifies the use off SSL or not. Then the parameter should be named accordingly.

[bdurrer]

The source of this issue is this PR https://github.com/home-assistant/home-assistant/pull/6429

While I totally understand and agree to your standpoint (and regardless of the outcome of this issue):
Using non-tls on 8883 is simply a bad thing to do and against the standard. Your browser goes haywire too when you setup SSL on port 80 or non-ssl on 443.

Maybe it needs an "use_bundled_certificates" flag instead of pseudo-detecting TLS

[bdurrer]

I guess @dennisdegreef and @balloob should comment on this 😄

[balloob]

I think I agree here. We should not "upgrade" to SSL based on port. We should allow CERTS to be set to "auto" in which we use built-in certs. It's the same behavior for your browser. You can go to http://some-site.com:443 and use the http protocol.