Espeasy: MQTT Broker not reconnecting using dynamic url (if IP is changed)

[Grovkillen]

Me and my friend has both experienced trouble when the broker IP address has changed behind a dynamic dns url address.

For example homeaddress.dyndns.org redirects to 10.0.0.1 and after a power down the broker get a new ip address 10.0.0.254, this is reported to dyndns but the unit still try to connect to the old 10.0.0.1 address.

Is my suspicion correct; that the ESP Easy code takes the dynamic dns address, after connecting only use the direct IP address and never recheck the dyndns address and see if a new IP is needed.

This should be done if the broker is not responding.

[tedenda]

Is this "related" to this #261 issue?

[Grovkillen]

Yes it probably is related.... seems like it.

[pwrsoft]

The same issue, nearly 6 month passed - could someone knowledgable fix it, please?!

[Grovkillen]

I've added it to 2.1.0 since the 2.0.0 is pretty much closed and only smaller issues are being fixed.

[TD-er]

And also related to #521

I was just thinking, maybe if a TCP request is being performed and the other side is not responding, perform a DNS update every now and then (at most once per 10 minutes or so).
That may be a workable workaround?

[pwrsoft]

@Grovkillen thanks for adding it to the 2.1.0 milestone list.
Actually I have 2 nodes with different firmware, where this bug is present: it's master and mega versions.
It would be great if it will get fixed in both branches.

[Grovkillen]

@pwrsoft the mega will become the master when the FW is released as 2.0.0

[ziolek]

I'm experiencing related issue every time CloudMQTT changes IP for it's service behind the same hostname.
It looks like the the name is being resolved only once after setting hostname in MQTT config.

Version: v2.0.0-dev12

[travellingnashes]

Can we get a patch for this against 2.0.0 dev 12 please. Ive had 9 sensor boards go offline due to this only being looked up once when the settings are stored. I would like to test the patch and ensure it closes the issue we are discussing. If not I need to move from ESPEasy to custom code asap

[TD-er]

I've read all related issues and it seems it is only fixed for C11, the http-controller: #302
But it should be fixed for every network-connection, thus including all controllers, syslog server, etc.
And perhaps there is also some plugin which uses network connections? (not sure)

I will look into it this evening to see how wide spread this issue is.
Best would be to have some cache in memory for every reference to a hostname and thus prevent unneeded writes to flash when an IP changes. This cache would be only 4 bytes in size. Only problem is to combine them with the hostname. So perhaps if we can have some 1 byte (unique) identifier along with every hostname, we can have a lookup table present in memory.

When the impact is clear, a well informed decision can be made.

[Walderbash]

I was just looking into this last night trying to connect to Bluemix IOT. It looks like ESPEasy does a DNS lookup when you save the MQTT config, and when doing an actual mqtt call always uses the saved IP address instead of the hostname.

change to something like this:

Wouldnt that solve the issue? (dont know if there is an other DNS cache)

[travellingnashes]

No thats not forcing a reconnect when public IP changes. I will add some diagnosis and try to understand what it is doing. Logic looks sound though

Mike

On 30 Oct 2017, at 15:39, Walderbash [e-mail] wrote:

I was just looking into this last night trying to connect to Bluemix IOT. It looks like ESPEasy does a DNS lookup when you save the MQTT config, and when doing an actual mqtt call always uses the saved IP address instead of the hostname.

https://user-images.githubusercontent.com/16013900/32179456-bcf59e64-bd8f-11e7-8f16-87c26fb62d9f.png
change to something like this:
https://user-images.githubusercontent.com/16013900/32179511-dd731ee6-bd8f-11e7-94c4-23bd45fc851e.png
Wouldnt that solve the issue? (dont know if there is an other DNS cache)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/letscontrolit/ESPEasy/issues/381#issuecomment-340484364, or mute the thread https://github.com/notifications/unsubscribe-auth/AGCaivmKZZFjb9rBqWynj61LHKKHLxNfks5sxe3GgaJpZM4OB7ul.

[travellingnashes]

Sorry my error. Yes that is working against the test that I am doing. Not sure against the efficiency of this but it is fine for my need as I am only doing this once an hour per device. Great thanks

Mike

On 30 Oct 2017, at 15:39, Walderbash [e-mail] wrote:

I was just looking into this last night trying to connect to Bluemix IOT. It looks like ESPEasy does a DNS lookup when you save the MQTT config, and when doing an actual mqtt call always uses the saved IP address instead of the hostname.

https://user-images.githubusercontent.com/16013900/32179456-bcf59e64-bd8f-11e7-8f16-87c26fb62d9f.png
change to something like this:
https://user-images.githubusercontent.com/16013900/32179511-dd731ee6-bd8f-11e7-94c4-23bd45fc851e.png
Wouldnt that solve the issue? (dont know if there is an other DNS cache)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/letscontrolit/ESPEasy/issues/381#issuecomment-340484364, or mute the thread https://github.com/notifications/unsubscribe-auth/AGCaivmKZZFjb9rBqWynj61LHKKHLxNfks5sxe3GgaJpZM4OB7ul.

[TD-er]

I just looked into the code of the Arduino library.
The code for DNS lookups ( https://github.com/esp8266/Arduino/blob/master/tools/sdk/lwip/src/core/dns.c ) is using a table to cache the requests.
So apart from having to lookup the cached result, there shouldn't be any reason to keep a local cached IP-address in every controller instance.
Only thing is that the DNS_TABLE_SIZE might be too small for global use.

My suggestion is to add a helper class to be used in ESPeasy, which will cache only the number of hosts requested in that instance of the helper and only so long as (global) defined or until a timeout occurs.
Then use this helper class anywhere a WiFiclient is used in the code and thus fix all controllers at once.
This way we have a single instance of code to tweak and tune and all plugins and controllers will profit from those fixes.

[psy0rz]

cant we just do a dns lookup every time, since it already does caching?

[TD-er]

@psy0rz That's what my patch now does.
When the UseDNS flag is set, it uses the hostname, or else the IP.

In later patches, I wanted to change the moment the IP is stored (on save of hostname), but that's something for 2.1 release.
For now it is just to use the same logic for all controllers and use DNS lookup when set as such.
The Arduino code is caching the results and I saw also something about use of TTL values in this caching, which is just like any other computer using DNS.

[psy0rz]

Yeah I like this way a lot, I think we should keep it that way.

So with this PR "Use DNS" actually means: "Resolve hostname everytime" vs "Resolve hostname only on save"?

[TD-er]

Yep, that's actually what's changed :)
And also, it now works like this for all controllers and the Email module and the MQTT import plugin.

There are still things to improve, but that's for later. (like why store IP and hostname)
That will result in change how things are stored in flash, so that's definitely not for 2.0

[psy0rz]

Indeed, thanks! @Grovkillen is there a wiki of the settings page? Maybe we could explain what "Use DNS" does from now on:

"Use DNS": Lookup the IP address every time ESPEasy connects to the host. (Requires v2.0.0-dev13 or later)

Disable this function to only lookup the IP address once when you save the settings.

[TD-er]

@psy0rz When you set the toggle to "Use IP", the IP lookup of the entered (and saved) DNS hostname is displayed and only the IPaddress will be used.
When you switch back to "Use DNS", sadly the original hostname is not being displayed anymore.
That's something to look into for later releases and perhaps redesign of that feature. (why store hostname and IP)

[psy0rz]

ah right i understand.

[Grovkillen]

I will update Wiki this weekend.

[catcam]

Can you please post link to compiled v2.0.0-dev13 ... im not lazy but im on field and this will save me lot of time ... and tnx for great job

[Grovkillen]

@catcam No dev13 is released yet. You find the released packages here:

https://github.com/letscontrolit/ESPEasy/releases

[catcam]

yes ... but looking for version with dns resolve fixed ... again tnx for quick reply

[Grovkillen]

@catcam as i said, the dev13 is not out yet. Psy0rz just mentioned it for my wiki text that I will be adding later today. But again, the dev13 is not yet released.

[catcam]

ok it is clear for me now ... thumbs up and looking forward to dev13