Core: home_connect uses internal_url and should use external_url as redirect_uri

[golles]

The problem

The setup flow for the new integrated home_connect integration isn't working. The URL that is opened in a new tab is using the internal_url. Instead, it should use the external_url from the config

Environment

• Home Assistant Core release with the issue: 0.110.0
• Last working Home Assistant Core release (if known): -
• Operating environment (Home Assistant/Supervised/Docker/venv): Docker
• Integration causing this issue: home_connect
• Link to integration documentation on our website: https://www.home-assistant.io/integrations/home_connect/

Problem-relevant configuration.yaml

I use the packages setup, 2 relevant configuration files:

cat configuration.yaml

homeassistant:
  internal_url: !secret internal_url # => http://192.168.68.114:8123
  external_url: !secret external_url # => https://hass.example.com
...

cat packages/home_connect.yaml

home_connect:
  client_id: !secret home_connect_client_id # => client id from dev portal
  client_secret: !secret home_connect_client_secret # => client secret from dev portal

Traceback/Error logs

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 418, in start
    resp = await task
  File "/usr/local/lib/python3.7/site-packages/aiohttp/web_app.py", line 458, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.7/site-packages/aiohttp/web_middlewares.py", line 119, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/real_ip.py", line 39, in real_ip_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 73, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 127, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 125, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 136, in get
    return await super().get(request, flow_id)
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 91, in get
    result = await self._flow_mgr.async_configure(flow_id)
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 153, in async_configure
    result = await self._async_handle_step(flow, cur_step["step_id"], user_input)
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 201, in _async_handle_step
    result: Dict = await getattr(flow, method)(user_input)
  File "/usr/src/homeassistant/homeassistant/helpers/config_entry_oauth2_flow.py", line 248, in async_step_creation
    token = await self.flow_impl.async_resolve_external_data(self.external_data)
  File "/usr/src/homeassistant/homeassistant/helpers/config_entry_oauth2_flow.py", line 142, in async_resolve_external_data
    "redirect_uri": self.redirect_uri,
  File "/usr/src/homeassistant/homeassistant/helpers/config_entry_oauth2_flow.py", line 167, in _token_request
    resp.raise_for_status()
  File "/usr/local/lib/python3.7/site-packages/aiohttp/client_reqrep.py", line 946, in raise_for_status
    headers=self.headers)
aiohttp.client_exceptions.ClientResponseError: 400, message='Bad Request', url='https://api.home-connect.com/security/oauth/token

and

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 418, in start
    resp = await task
  File "/usr/local/lib/python3.7/site-packages/aiohttp/web_app.py", line 458, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.7/site-packages/aiohttp/web_middlewares.py", line 119, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/real_ip.py", line 39, in real_ip_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 73, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 127, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 125, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/helpers/config_entry_oauth2_flow.py", line 376, in get
    flow_id=state["flow_id"], user_input=request.query["code"]
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 146, in async_configure
    raise UnknownFlow
homeassistant.data_entry_flow.UnknownFlow

Additional information

Url that is opened: https://api.home-connect.com/security/oauth/authorize?response_type=code&client_id=code_was_here&redirect_uri=http://192.168.68.114:8123/auth/external/callback&state=state_was_here

[probot-home-assistant[bot]]

home_connect documentation
home_connect source
_{^{(message by IssueLinks)}}

[astrandb]

Same problem here! I can fonfirm that wrong URL is used for callback.
When I swap internal and external urls in config the config flow continues and the login page at Home Connect appears. The installation completes but I run into other problems with other integrations because of wrong external URL.

[frenck]

It isn't that simply put. Internal and external URLs have nothing to do with the URL you as a user are visiting. That said, integration have no idea of which URL you are using to visit Home Assistant, thus ask Home Assistant, which returns the first available URL.

If what is done as suggested (using external URL), other problems will occur. For example, people with loopback problems.

While I agree this needs addressing of some sort (I have been testing a solution), it is not the integration at fault here.

[golles]

Hi Frenck, bedankt voor je bericht!

I should've mentioned why I thought it is related to the internal/external URLs. I noticed the first time setting it up, it uses the URL from the IP range of my docker network (172...). From reading the release notes of the 110 version, I replaced the base_url with the 2 new URLs.

From that moment I got the internal_url (192.68..) as callback URL.
Changing the URL manually with the correct endpoint in my browser still fails setting up the component for me. I do get on the login screen and I can grant access, but from there it generates the logs provided in the issue.

For now, I've rolled back to https://github.com/cotwild/home_connect

I'll watch this ticket and if anything changes I'll retry. Keep up the good work!

[astrandb]

Tried to register the internal url (http://192.168.18.10:8123/auth/external/callback) as redirect URI with the Home Connect portal. Everything seems to work now.

[abstrakct]

I'm having the same problems as described in this thread.

Registering internal url as redirect uri didn't work for me. After granting access I get sent back to http://10.0.0.10:8123/auth/external/callback?code=[long string]&state=[long string]&grant_type=authorization_code

But that page only displays:

This page isn’t working
10.0.0.10 didn’t send any data.
ERR_EMPTY_RESPONSE

10.0.0.10 is the correct IP for my server where HA is running, I guess it's in Supervised mode (Core and Supervisor running in Docker containers, used to be hass.io).

Is there maybe something wrong with my setup / network configuration? Thanks in advance for any pointers towards why this happens.

[DavidMStraub]

@abstrakct can you access the UI at http://10.0.0.10:8123/ from you browser or not?

[abstrakct]

@DavidMStraub Yes, I can, that's how I mainly access my Home Assistant.

Strangely, the HA Android app also refuses to connect via 10.0.0.10, it will only connect through my Nabu Casa link, so I suspect something is wrong on my end, but I don't know what.

I might add that I've been using the Home Connect integration for a while now (before 0.110), installed it via HACS, and it worked without problems then.

Edit: I can't access 10.0.0.10:8123 through http, it has to be https, maybe that's part of the problem?

[abstrakct]

I solved the problem, at least for me. I still had

http:
  ssl_certificate: /ssl/fullchain.pem
  ssl_key: /ssl/privkey.pem

in my configuration.yaml. After removing those (and the unused DuckDNS addon I had running) I can access HA via http locally, and adding the Home Connect integration worked perfectly! (I also used the 10.0.0.10 url as redirect in the Home Connect portal.)

[DavidMStraub]

OK, so I think we can conclude that your problem was unrelated to this issue and it still stands that using the internal_url for the redirect URI does work.

[golles]

I've changed the Redirect URIs to my network host address 192.168... and I successfully added my devices to HA.

Maybe the docs should be more clear what to add there, I really assumed it needed the HA address that's accessible through the internet.

[MrUserY]

Hmm. Oke, knowing this. And just following the manual again. I used: http://192.168.47.12:8123/auth/external/callback

Added these to the config yaml (offcourse with all the secret data)...
home_connect:
client_id: CLIENT_ID
client_secret: CLIENT_SECRET

And then reboot, I get this error:

2020-05-25 12:21:22 ERROR (MainThread) [homeassistant.setup] Setup failed for home_connect: No setup function defined.

Oke then I went to Configuration > Integrations > + Home Connect.

I don't get a dialog. The screen just flashes and nothing happends. But in the log I can see this:

2020-05-25 12:28:56 ERROR (MainThread) [homeassistant.config_entries] Error occurred loading configuration flow for integration home_connect: No module named 'custom_components.home_connect.config_flow'

Environment

Frontend version: 20200519.4 - latest
arch | armv7l
-- | --
chassis | embedded
dev | false
docker | true
hassio | true
host_os | HassOS 3.13
installation_type | Home Assistant
os_name | Linux
os_version | 4.19.114-v7l
python_version | 3.7.7
supervisor | 224
timezone | Europe/Amsterdam
version | 0.110.2
virtualenv | false

[MrUserY]

This is how my Home Connect API looks (the personal data is scrambled in this example, it's fake data)

Home Assistant
Created 2020-05-20 | Last Modified 2020-05-25

Access Private
i Private access is limited to a single Home Connect user account as defined.

Home Connect user account for testing [email protected] Add more
i This Home Connect user account has been configured in your profile. You can override it for this application.

OAuth Flow: Authorization Code Grant Flow

Client ID: BLA8D3SECRETSTUFFFEC

Client Secret: (required) 75FSECRET!!!!!!!!!!D28E

Redirect URIs: http://192.168.36.44:8123/external/auth/callback

One Time Token Mode Disabled

Proof Key for Code Exchange Disabled

[DavidMStraub]

Have you installed the custom component? Something seems to be wrong with your installation. It looks for custom_components.home_connect.config_flow, but home_connect is not a custom integration.

Edit: And BTW this seems to be unrelated to this issue here.

[MrUserY]

I’ve used CodWilds Home Connect custom component in the past before this one got official. I’ve stopped using that integration.
Could that be the problem because it’s also called Home Connect? 🤣
More people are gonna have that maybe. I’ll do a clean install. Maybe tomorrow and see if that matters :) Thanks

[DavidMStraub]

That would explain it ...

[pascalsaul]

@MrUserY It is working like you're saying and I changed also from the beta to the built-in home_connect integration. The only thing you need to use the internal url as callback. It's little bit odd but hence it is working :)

[MrUserY]

@DavidMStraub I made some templates wich maybe interesting for other people to put in the documentation:

https://community.home-assistant.io/t/combining-home-connect-sensors-in-template/200864

[DavidMStraub]

Thanks @MrUserY, I think the community forum is exactly the right place for this kind of thing.

[golles]

I think to make this kind of template sensors for something the component should do is overkill. Due to the lack of the status sensor, I rolled back to the custom component (mentioned earlier). I also didn't like there is a switch for every possible program, even if the device doesn't support the program.
@DavidMStraub, if you don't mind I'll raise some improvement issues soon because I do think this integrated component has a lot of potential! For sure thank you for making this!

[DavidMStraub]

@golles OK, but to avoid repetition of discussions, can I ask you to please check the (open and closed!) feature requests over here: https://github.com/DavidMStraub/homeassistant-homeconnect/issues
This should explain many of the choices (e.g. why there are switches for non-existing programs), often governed by API limitations.

[AndruXdev]

Hi,

I'm using hass.io on docker and I have the same issue. I added my internal url as a redirect URL in Developer Portal.
I get login screen and once I grant access I get login screen opened again and I have to grant access again. Once I do that for the second time I get this error in browser:
"500 Internal Server Error
Server got itself in trouble"
and the same error in the log file as it is reported in this issue's description.

[X1pheR]

Hi,

I'm using hass.io on docker and I have the same issue. I added my internal url as a redirect URL in Developer Portal.
I get login screen and once I grant access I get login screen opened again and I have to grant access again. Once I do that for the second time I get this error in browser:
"500 Internal Server Error
Server got itself in trouble"
and the same error in the log file as it is reported in this issue's description.

This fixed it for me! Thank you. I filled in the internal and external link without home assistant gui and the internal link as redirect url on the home connect developer portal (in the application config)

[AndruXdev]

I tried this, but no success - I get same errors.

[nicx]

same error 500 here when using the internal URL as redirect URL, any solutions yet?

[allmoney]

I changed internal to external in URL and got 500 error:(

[Reidar6]

For me it works. You must have the same URL in "Redirect URI : /auth/external/callback" on the HomeConnect Development Application as you normally use to access your HomeAssistant locally. In my case I use "http://10.0.1.251:8123/" and therefore on the Redirect URL I also have stated "http://10.0.1.251:8123/auth/external/callback". Then it works fine to add the Integegration. :)

[AndruXdev]

I tried this. HC Dev App allows multiple redirect URL's. So, I added all possible ones - internal IP, external IP, external DNS name.
No success. This error is returned once I grant access on Home Connect website

[Reidar6]

If I had my external IP/DNS stated in HC Dev and used that one it didnt work and I also got all Errors above including Error 500. But when I cleared up all the URL and used only my internal everywhere then it worked.

[X1pheR]

For me it works. You must have the same URL in "Redirect URI : /auth/external/callback" on the HomeConnect Development Application as you normally use to access your HomeAssistant locally. In my case I use "http://10.0.1.251:8123/" and therefore on the Redirect URL I also have stated "http://10.0.1.251:8123/auth/external/callback". Then it works fine to add the Integegration. :)

This is the way to go. Even though you have HA externally available. Also make sure you are connected to HA through the internal ip/fqdn when setting up the integration.

[nicx]

Unfortunately that is not working for me.

I am using the local URLs everywhere: Browser, Redirect URL, even as external AND internal URL in the config:

I get always a blank white page after the login to the homeconnect-dev portal and accepting the application access. The redirect URL seems to be fine. After doing a reload of this blank white page I get the "500 Internal Server Error".

[DavidMStraub]

Anything in the log? That reload gives you a server error is not surprising.

[nicx]

no nothing in the log, on the integration install UI the "round circle" keeps circling in state "please wait while the integration is installing", nothing happens.

the callback URL which leads to the blank white page looks like that:

http://192.168.0.10:8123/auth/external/callback?code=eyJ4LXJlZyI6IkVVIiwieC1lbnYiOiJQUkQiLCJ0b2tlbiI6ImE0ZTM0MdsfgsdfgsdfgsdfgDBlNy1iZTFiLTNkYjQ5NzkxOTg5NyJ9&state=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.edJmbG93X2lkIjoiMzg3YzVjZWsdfgsdfgsDMxMmU3YTg1OGU1OWYifQ.N0_wfQNDALYAZ0E99ziWHFU_KWmEhSuRaELVdddC-Mg&grant_type=authorization_code

[AndruXdev]

Ok, I solved my issue. Issue was related to the configuration - '!secret' was missing in configuration for 'client_secret' configuration variable

[nicx]

OH MY GOD! --- I solved my issue, too: It was just my impatience. After getting the blank white page I just had to wait about 3-5 minutes, then the integration was correctly installed.

Sorry for that! Maybe the documentation should be updated that the blank page is not an error (and not to reload it). Even the long installation duration should be mentioned for impatient people like me ;)