Tasmota: Increase maximum mqtt username and password length.

[deshark]

I am using a QNAP IoT suite called QIoT. It is based on Node-Red. The built in MQTT broker auto-generates usernames and passwords. Both are longer than the Tasmota max of 32 chars.

Would it be possible to increase the max length of accepted MQTT password and username to 40 chars in the default sonoff binary available for download?

[localhost61]

I guess it's easy to change in settings.h and per chance there is free slot of 20 bytes available not far in the settings memory map.
It will only induce a shift of mqtt_host, mqtt__port, mqtt__client, mqtt__user & mqtt__pwd in the settings map.This shift will have to be processed in SettingsDelta() of sonoff.ino so that those settings are not lost after an upgrade (or downgrade) _but they will be lost after a downgrade._
It may be other side effects I'm not aware of.

[localhost61]

I did it and tested on a Wemos, just have to push it on my repository.
@deshark
I made PR #4755, but it may not be accepted. In the mean time, you can test this binary compiled with default settings.

[deshark]

Thank you for such prompt response. I'll check out the binnary.

[localhost61]

@deshark I'm afraid that this solution won't be merged . Settings memory space is too precious.

[deshark]

Yes, I saw Theo's response. I'll have to find another long term solution to
this issue. Anyway, thank you for providing the quick fix.

[ascillato2]

Hi,

Closing issue as it is not going to be added soon. Added the label Feature Request (hold over) for future reference.

Thanks everyone for sharing their ideas and testings :+1:

[DP-M]

This is also a problem when using flespi.io. They generate a 65 character username token. Flespi.io is a high quality public MQTT broker, and the only currently functional broker on the public brokers listing (https://github.com/mqtt/mqtt.github.io/wiki/public_brokers) that offers a free authenticated account supporting web sockets, tcp, and ssl, with a generous connection limit (100). I realize that memory is tight, but could someone reconsider this request, or possibly include this upgrade with one of the more feature/memory rich builds? Thanks.

[localhost61]

After a short survey it seems that flespi.io requires a long token but no pass, so it may be feasible to concatenate Id & Pass memory space (40 bytes for Tasmota, 64 bytes in my fork) and extend to 65.
No way to allocate 65 bytes for both Pass and Id, allocating so much contiguous memory space would break other functionalities and need too much reworking that would prevent any possibility of future updates and support.
--
[Edit]
After looking at the code the current size is (32+32), that I extended to (40+40). It would require no effort to make it (65+25) and just add a #define flespi.io in my_user_config.h

[DP-M]

After a short survey it seems that flespi.io requires a long token but no pass, so it may be feasible to concatenate Id & Pass memory space (40 bytes for Tasmota, 64 bytes in my fork) and extend to 65.
No way to allocate 65 bytes for both Pass and Id, allocating so much contiguous memory space would break other functionalities and need too much reworking that would prevent any possibility of future updates and support.
--
[Edit]
After looking at the code the current size is (32+32), that I extended to (40+40). It would require no effort to make it (65+25) and just add a #define flespi.io in my_user_config.h

Hi Localhost, is this something that you're planning on doing, or that could be done easily? I'm still without an authentication compatible, publicly accessible mqtt broker I can use with tasmota.

[DP-M]

Additionally, I was finally able to get a signup confirmation email from solace.cloud (their signup process was broken for months - I don't want to think about what that means for their longevity)... but their hostname is apparently too long for tasmota. ARGHhh.. This world of mqtt brokers seems to be heavily dependent up on randomly generated, and sometimes long strings. It seems we should be able to allocate memory to these hostnames, usernames, and passwords.

My console says:
01:32:08 MQT: Attempting connection...
01:32:14 MQT: Connect failed to mxxxxxxxxxxxx.messaging.solace.c:20966, rc -2. Retry in 10 sec

the "loud" is getting hacked off the end of the hostname.

[localhost61]

OK, I'll do that fork very soon. Maybe adding a Setoption value to change 40/40 to 65/25 space allocation for Id/Pass but don't expect full support for that.

[DP-M]

would a my_user_config.h option require compiling my own firmware? I'm still learning about tasmota, but my understanding is SetOptions can be set during runtime, which would of course be great.

In either case, thanks for pushing forward on this.

[localhost61]

Yes, my_user_config.h (h for Header) is a code file that is "included" in .ino or .cpp source code file and needs to be compiled to be taken into account. It's the reason why I'm talking about "Setoption" which is a setting that can be overwritten by a user command in the console, but I still didn't investigate and am not sure it can apply there.

[localhost61]

Sorry for the delay.
I made some tests with a Wemos D1 and could connect on flespi.io with Tasmota v6.6.03 with the only change that consists to take 16 bytes from free_1D6 and reallocating into mqtt_user[65] and mqtt_pass[17].
@DP-M I believe that you're more interested in a binary for testing, we could Chat on Discord so that I provide you the binary that will fit your needs. (Language, sensors...)

A binary is available in my repository here.

[quammy]

Hi, I have the same issue with Flespi.. The binary compiled by localhost61 (thank you very much!!) is of v6.6.0.3 while latest is V6.7.1. Any chance to have this fix part of the main release?

Another question while at, Flespi host is "mqtt.flespi.io:1883/messages". How can I not use the tasmota port field? Because the console shows it as "mqtt.flespi.io:1883/messages:1883" and I'm not sure "mqtt.flespi.io/messages:1883" will work (cannot test till I'll burn V6.6.0.3).

Thanks!

[localhost61]

I can compile a version on the demand but prepare to argue if you want it to be included in the main project. Even the idea of a flexible allocation of the USER/PASS/HOST settings space was rejected. :-/

[meingraham]

prepare to argue

...prepare to _justify_ :wink:

[quammy]

hmm, look, I am a believer of the famous 'if it ain't broken, don't fix it' ;)
I'm also not 'native speaker' of github, versioning, branches and so on and so forth.
My argument is always be the most flexible to suit as many as possible. Flespi is an amazing reliable and free (home using) service that proved working with my Shelly-Cloud devices on HomeAssistant.
Apart from that, not sure what else can I say. For sure not to argue the feasibility or the easiness of implementing/maintaining such solution in parallel to current user/pass scheme.

[quammy]

@localhost61 I just burned the "sonoff_(user_sz=64).bin" but the user field is still too short :(
an example for a valid value is "FlespiToken ettnzXlHkEKtreMD2QP7SZqfhCo1f38hlrvRrRfmcOFrSgULWyGCnVHulUmqNiwY" which is 76 chars.
Is this a dead end?

[localhost61]

Are you sure that "FlespiToken " is part of the ID?

[quammy]

yes, with the space between the "FlespiToken" and the tocken itself (weird, no?) but it works like that with my shelly devices.

[localhost61]

It should work without too, they advert 64 not 76 bytes. Try it.
And about the API, the host and port setup seems nothing special I believe you misunderstood the how-to and I can't help I use mosquitto locally.

[quammy]

you are right. it works also without the FlespiToken (though I must emphasize again that it works for me with it on Shelly MQTT device. maybe they use http behind the scene and the Authorization header must use the FlespiToken...)
Anyway, so I set the host to "mqtt.flespi.io" / port to 1883 and I could see the unit is getting connected - both on the local console and on the Flespi web log. But no message arrive (even if I subscribe to "#"). Again, on the shelly the host is "mqtt.flespi.io:1883/messages" (again, maybe http...?) when I set the host on the sonoff to that host, I'm getting on the console: "mqtt.flespi.io:1883/messages:1883"...

So, I think you pointed me to the right direction with Flespi:
host should be only "mqtt.flespi.io"
port: 1883
user: only the token itself
Full topic - maybe this is where the culprit lies. Will try to play with it and will report here if I'll solve it.

Meanwhile, only if not too much a hassle, can you publish latest version with place for 64 bytes on the user field?

[localhost61]

Do you want english tasmota.bin or something different?

[quammy]

English please

On Fri, Nov 8, 2019, 13:38 Xavier MULLER [e-mail] wrote:

Do you want english tasmota.bin or something different?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/arendst/Tasmota/issues/4754?email_source=notifications&email_token=AAX57UBJHFOKI4S6A4M6MCTQSVFSVA5CNFSM4GMRD4I2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDQVCWI#issuecomment-551637337,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAX57UBZK2FTKFTXIM3QEGTQSVFSVANCNFSM4GMRD4IQ
.

[quammy]

update - I had a long online debugging process with Flespi support and we have found the problem - somehow the will topic was set to "flespi/message/gw/devices/sonoff-8060/tele/LWT" which messed things on their side as they do not allow any topic to start with "flespi".
As we are on the end of friday, they have told me they will look in to it on monday - though the will topic is 'ilegal' for them, it should not have prevent the regular messages from being published correctly.
While at, where can I edit the will topic on the sonoff? As is I don't even know how it was set on the first place...

[localhost61]

I'm preparing a release at first from current master 6.7.1 and then to latest dev 7.0.0.4
I'm forced to reduce the amount of mods because more settings have been added since v6.6.0.3 so the mod is restricted to the bare minimum, hence I only reallocate space between User and Pass and don't retrieve the preceding 16 bytes that are no more Free.
Caution, after upgrading, MQTT settings won't match, you'll have to reenter them.

• With Flexi.io version, size(mqttuser)=64, size(mqttpass)=0
• With QIOT version, size(mqttuser)=40, size(mqttpass)=24

To avoid issue with settings, I will reset all MQTT strings and set port to 1883
Concerning topics, they are very configurable in Tasmota, take a look at the huge wiki resource.

[quammy]

Thank you very much.
Perdoname, what exactly is a 'mod'?

>

[localhost61]

A short name for modification ;-)

[localhost61]

You'll find my fork of ~the master release 6.7.1~ previous release 6.6.0.3 here

- edit -
Those patched binaries in v6.6.0.3 do work but were dropped by mistake in the wrong commit!

[quammy]

Great, thank you so much!

1. Any idea about the will topic setting?
2. Is there a quick guide about compilation I can follow for future
   versions?

[localhost61]

v7.0.0.4 ~here~

- edit -
Folder removed in repository because broken binaries

[localhost61]

Guide about MQTT -> wiki
Guide about how to compile with Atom -> wiki

[DP-M]

Hey, just wanted to follow up since I originally brought up the flespi issue. I appreciate localhost61 coming up with a custom build, but my goal was to be a "regular user" of this project, and not have to worry about doing builds, or relying on someone else to do the builds. Compiling the binaries may be easy for the devs who do it frequently, but for someone who only touches this stuff once a year, and doesn't live and breath this project, it represents a steep learning curve, for what should be a relatively simple task (upgrading).

I think that the memory savings of username, password, and hostname have turned into just simply being short sighted and stingy. There are at least 3 examples in this thread of very common use cases which can't happen because hostname, username, and password memory allocation has been neutered to the point of being useless, to save a few bytes.

In my case specifically I was eventually able to setup a solace.cloud account, which had a username and password that met the tasmota guidelines. The problem was that their HOSTNAME was too long (argh!!!!). So my solution was to create a cname on my personal domain which, which is only 4 characters long... That of course means that I can't do SSL though.. So yay, no security for me. But it got me moved on past this ridiculous hump.

[meingraham]

FYI - if you want to avoid installing an IDE at all (Atom, VSC, Arduino), use Gitpod. You just point it to the GitHub repository you want to compile (e.g., https://github.com/localhost61/Tasmota/tree/development). Using Gitpod is documented in the wiki.

[localhost61]

@meingraham yes, but sadly it won't allow him to apply latest changes of the mainstream when my repository will be out of date. Merging the conflicts between versions needs to understand what's under the hood.

[meingraham]

Yes, I know... but while you have your fork and if/when you merge the master then he will have a way to get a new binary on his own.

The issue of making it a permanent option in the master branch... that's a fight for others than me :wink:

[quammy]

Guide about MQTT -> wiki
Thanks, I saw that link but didn't find there anything about will topic, neither on other MQTT pages on the wiki or google 'tasmota+will+topic'...
Guide about how to compile with Atom -> wiki
Thanks, adding to my TODO list ;)

and also thank you @localhost61 for the compiled latest versions and thanks @DP-M and @meingraham for your input.

[localhost61]

Either LWT for Last Will Topic

[quammy]

Ahhh, I understand fast when you explain slowly ;)

[StKob]

You'll find my fork of the master release 6.7.1 here

After using it I'm still seeing it as 6.6.0.3 version.

[quammy]

@StKob , You are right, just checked and mine also shows 6.6.0.3...

[quammy]

BTW, looking the other day on the tasmota Options - is it possible to add another option, one that will allow writing half the flespi token on the user field and half on the pass field and then combine them as user only when authenticating? Maybe it's an easier solution than compile a dedicated version?

[localhost61]

@StKob @quammy You're right, my mistake. I uploaded previous version binaries instead of 6.7.1 I'm afraid I can't correct it because it's a github snapshot and I can only upgrade latest commit.
Here they are for ~v6.7.1(7bafb0f)~:
~sonoff_(user_sz=64).zip~
~sonoff_(user_sz=40).zip~

But you also have the option to upgrade to my fork of ~V7.0.4(538d40a)~.

- edit -
Files removed because broken

[StKob]

@StKob @quammy You're right, my mistake. I uploaded previous version binaries instead of 6.7.1 I'm afraid I can't correct it because it's a github snapshot and I can only upgrade latest commit.
Here they are for v6.7.1(7bafb0f):
sonoff_(user_sz=64).zip
sonoff_(user_sz=40).zip

But you also have the option to upgrade to my fork of V7.0.4(538d40a).

Have you tested 6.7.1? Cause after flashing my sonoff mini cant connect to wifi.

[quammy]

@StKob , yes mine works as expected

[StKob]

@localhost61 Have you changed anything regarding configuration saving? I can't connect to wifi and it seems like provided data are not used after reboot.

[localhost61]

@localhost61 Have you changed anything regarding configuration saving? I can't connect to wifi and it seems like provided data are not used after reboot.

Which version and on what device? Are you talking about v7.0.4?

Look at the source code and you'll see that the changes against Tasmota mainstream are really tinny and only related to MQTT, not WiFi. So, upgrading to my fork shouldn't affect WiFi settings.

Additionally I checked in the binary file, my personal settings are not included.
So, if you erased settings prior to flash my fork then they should get initialized to default ones like mainstream Tasmota does and act like Tasmota version regarding your issue.

If possible, I suggest that you serial flash Tasmota version in order to see if it solves your issue ( I think it won't). If it does then flash my fork again and see what happens.

[StKob]

@localhost61 I did what was told me on discord chat. So basically I have use esptool to erase flash, power cycle and then flash firmware. With your 7.0.4 it was showing "cfg use defaults" and not saving my wifi settings. After I have done it with original 6.7.1 it worked.

EDIT
The same happens trying to flash your 6.7.1 onto original 6.7.1.

EDIT2
Reverting to your 6.6.0.3 fixes problem.

[meingraham]

It's is likely that the code hack for the mqtt credentials doesn't work because other parts of Tasmota have been modified. A hack made for 6.x is a hack and might need to be tested and modified in order to work with 7.x. Since the hack you are using is not a supported feature, you probably should inform us that you are using hacked code to avoid troubleshooting something that is a problem with your code hack and not in the stable Tasmota codebase.

[localhost61]

@StKob I can't explain why. Will do some tests tomorrow.

Frankly I never tested those versions I share because I'm too lazy to test from scratch...
As I can compile my flavored binary and use a local Mosquitto brooker I'm used to:

• integrate my own defaults settings (user_config_override.h)
• remove all sensors and options I don't need in order to get a tiny binary that don't cross the half of flash space boundary (my_user_config.h).
• use standard 32byte strings for MQTT settings

But I have some spare Wemos D1 mini I can use for test...

@meingraham I agree that this follow up is off-topic and should be addressed on my own repository. Once again, the hack has nothing to do with WiFi (as you can check in the source code in v7.0.4 or v6.7.1).

[StKob]

@localhost61 The problem per se is not wifi but not storing provided settings after reboot.

[meingraham]

the hack has nothing to do with WiFi

❔

Did I mention Wi-Fi? I gathered from the subject line that it was related to MQTT credentials.

But yes, once a stable solution is identified through testing in your forked repository, please submit a PR to Theo.

Mike

[StKob]

I though it was wifi problem, but after little troubleshooting it looks like defaults settings are always used instead of provided by the user.

[localhost61]

@StKob bug in my fork confirmed for v7.0.4 :-/

I get the same issue using Erase->serial Flash->serial Monitoring
When I restore the 4 files I changed and compile again it works, WiFi credentials are loaded at boot and settings are initialized.

So I'll have to investigate to understand where and why it fails.
Basics first:

• With original Tasmota I could enter WiFi credentials
• Then I flashed OTA tasmota-minimal.bin
• In tasmota-minimal I OTA Tasmota_(user_sz=64).bin and same issue, at reboot it doesn't find WiFi credentials and start in AP mode the Web server at 192.168.4.1 to get WiFi parameters

...to be continued.

[localhost61]

Arghhh sono stupido!!! I found my silly mistake...

I erroneously introduced an extra byte in the MQTT strings, resulting in the shift of all following parameters in the settings memory block. Of course it couldn't work.

@StKob Sorry for the inconveniency and thanks for alerting me.
The following binary works:

Tasmota_(user_sz=64) v7.04.2019.11.25.zip

00:00:00 CFG: Loaded from flash at F9, Count 3
00:00:00 Project tasmota Tasmota Version 7.0.0.4(lh61)-STAGE
00:00:00 WIF: Connecting to AP1 xxxxxx in mode 11N as tasmota-5153...
00:00:07 WIF: Connected
00:00:07 HTP: Web server active on tasmota-5153 with IP address 192.168.xxx.xxx
13:27:17 CMD: mqttpassword
13:27:17 RSL: stat/tasmota/RESULT = {"MqttPassword":"****"}
13:27:31 CMD: mqttpassword 123456
13:27:31 RSL: stat/tasmota/RESULT = {"MqttPassword":"****"}
13:27:43 CMD: mqttuser
13:27:43 RSL: stat/tasmota/RESULT = {"MqttUser":"DVES_USER"}
13:29:19 CMD: mqttuser 12345678901234567890123456789012345678901234567890123456789012345
13:29:19 RSL: stat/tasmota/RESULT = {"MqttUser":"DVES_USER"}
13:31:27 CMD: mqttuser 1234567890123456789012345678901234567890123456789012345678901234
13:31:27 RSL: stat/tasmota/RESULT = {"MqttUser":"1234567890123456789012345678901234567890123456789012345678901234"}
-> restart

As you can see:

• you can't change the default password because it's length is forced to zero.
• An MQTT UserId greater than 64 bytes is not accepted (changes nothing) but if size is less or equal to 64 bytes you get an acknowledge and the device restarts to take into account the new value.

So it works as intended... now ;-)

[localhost61]

Here the correct commit for v7.0.4 in my repository

[localhost61]

BTW, looking the other day on the tasmota Options - is it possible to add another option, one that will allow writing half the flespi token on the user field and half on the pass field and then combine them as user only when authenticating? Maybe it's an easier solution than compile a dedicated version?

@quammy indeed it's the way Theo does with setLongMqttHost() for AWS_IOT implementation were he needs a 64 bytes long mqtt_host and concatenates mqtt_host+mqtt_user.
It could be a better long term approach. But in that case it's not a dynamic option, you still have to compile a specific version.

[quammy]

Thanks for replying. Not clear to me why it cannot be dynamic, but if it
really cannot, then this approach loses its charm.... ;)

[localhost61]

@DP-M @quammy @StKob @deshark
No need to use my ugly patch any more!
Tasmota v8.0.0.1 is out and ought to solve this issue in the best way.
Thanks Theo