Pubsubclient: Not handling messages above almost 150 characters.

[arihantdaga]

I tried to send this message through my mqttbroker. I have tested that this message is recived by my other clients(Laptop Client (Mqttlens)). But the same message is not recieved in my ESP8266.

The length of message is almost 146 characters. All the messages which are shorter than 140 characters in length are received and handled very well by ESP.

{"device":"5ccf7f81ad0e","sb":"584922e3b6de4b7de1c6f9a5","act":21,"key":"sahdgasvajshdbajhsbchaaaaaaabbbbcc","act_type":"switch","mobId":"9015555"} .

Is this an issue with this library or there is any config which we need to change for this to work.

[Suxsem]

You could try to increase the MQTT_MAX_PACKET_SIZE directive in PubSubClient.h
The standard value is 128 bytes

[arihantdaga]

Thank you @Suxsem . It worked. But just to know, can it create a major effect if i change 128 bytes to 5000 bytes for my ESP8266.

[Suxsem]

ESP8266 has a 64 Kbyte or RAM, so you are using less then 10% of it for the buffer. But consider that your program (plus the esp8266 operative system that includes tcp/ip and 802.11 stacks) could easily demands those free bytes as it becomes more and more complex.

If you are using the Arduino IDE, after compiling your sketch you will se the percent of RAM occupied by global variables (see this screenshoot: http://blog.startingelectronics.com/wp-content/uploads/2015/06/arduino-ide.jpg, where RAM is indicated as dynamic memory). You should not exceed a value of 60% or you could not have enough space for local variables.

In summary, for ESP8266, a buffer of 5000 bytes is fine (if you really need it) but always take a look at the percent of RAM used by global variables.
Note that this is a very general advice and your buffer size must be determined after a careful inspection of your program, unless your program is quite simple.

Have fun!

[arihantdaga]

@Suxsem Thank you very much.

[brycemcd]

I'm a noob with respect to Ardunio and C++, and I happened across this thread when debugging a very similar issue I was having with my ESP8266.

Following the advice here, I updated my Arduino sketch to the following but it did not work:

#define MQTT_MAX_PACKET_SIZE 512

// Broadcast readings
#include <ArduinoJson.h>
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <PubSubClient.h>

// some other includes
// Arduino setup and loop logic

That failed to publish a JSON string I was creating to the topic. A key debug line:

 Serial.print("publishing topic: ");
  Serial.print(client.state());
  Serial.print(" ");
  Serial.print(sizeof(postable)); // size of JSON string created
  Serial.print(" ");
  Serial.print(MQTT_MAX_PACKET_SIZE);
  Serial.print(" ");
  Serial.println( client.publish(weather_topic, postable, sizeof(postable)) );

revealed that MQTT_MAX_PACKET_SIZE had been set to 512.

It wasn't until I went into the Arduino library files where this pubsubclient was being referenced and updated the value in PubSubClient.h (the update was changing the value of MQTT_MAX_PACKET_SIZE to 512) did the message begin publishing to the broker.

Is changing the value in the library's source the idiomatic way to change the value? I would have thought that defining the constant as I had would have worked and been more idiomatic.

[knolleary]

@brycemcd due to the way the arduino build works, you cannot put your own #define in your code to override the one in the library; it rewrites your sketch to put all #includes at the start, above anything else.

So yes, you have have to edit the .h file manually.

[brycemcd]

Thanks so much for the quick reply! Hope it helps the next Arduino/ESP8266 noob that reads this

[ThomasBj]

What is the correct way to calculate how large buffer size is needed?
I feel that im just entering random high numbers but i want to relate the to the size of the payload.

[knolleary]

@ThomasBj it is:

length of the payload + length of the topic being published to + roughly 10 to allow for the fixed/variable headers of the packet.

[ThomasBj]

Thanks for you reply.
I guess i need to analyse the longest number of bytes of both outgoing and incoming JSON payloads .
After finding the longest, applying your formula.
Is this correct?

[knolleary]

Yes - and remember that buffer needs to fit within the limited memory of whatever device you're using.

[sreeramtkd]

Tried bumping up the packet size to 512, 1024 etc in the header file and no luck while I'm subscribing from esp8266, message is limited to 118 bytes I guess. Any suggestions ?

What's send from source: {'weather': [{'Dew Point': '24', 'Barometer': '1012.00 mb', 'Temperature': '25', 'Visibility': '2 km', 'date': '2018-11-21 19:36', 'Feelslike': '25', 'Wind': '18 km/h', 'Condition': 'Rain', 'Humiditiy': '94'}]}

The max message I can get in esp8266: {'weather': [{'Dew Point': '24', 'Barometer': '1012.00 mb', 'Temperature': '25', 'Visibility': '2 km', 'date': '2018-

[arihantdaga]

@sreeramtkd . I think you are using an older version of this library. I think earlier this library wasn't handling the macro definition properly. But in the latest version it should work fine.
You mentioned packet size to 512, 1024 etc in the header file. Which header file did you define it in ? Did you change the library's header file or you are talking about a header file in your own program.

In your own program you can define this before including the library, i think you can try doing this -

#define MQTT_MAX_PACKET_SIZE 512
#include <pubsubclient>

If you are using multiple files in your program than there is a possibility that pubsub client may have been included before your macro definition. In that case you can simply #undef and define again. But first thing, update your library to latest version.

[sreeramtkd]

@arihantdaga - Guess its matter of updating the lib versions... for some reason updating the library from the IDE didn't solve it initially for me, had to remove the folders in lib location and reinstalled the lib and changed the packet size in lib's header file solves the issue. Thanks much.

[sanchosk]

Not sure when it changed, but now the library itself includes #ifndef statement.
The only problem is that the new Arduino ESP package "precompiles" the libraries.
Therefore changing the value using #define in the code does not help if it was already compiled.
However, exiting Arduino IDE, starting it again and just hit compile helped...
Thanks for the tip, btw. I was searching for 2 days for a solution to why my GPS tracker cannot upload the information to mosquitto ;)