Tasmota: Invalid JSON if Sensor (DHT11) is not connected

[Tommy-LSA]

IMPORTANT NOTICE
If you do not complete the template below it is likely that your issue will not be addressed. When providing information about your issue please be as extensive as possible so that it can be solved by as little as possible responses.

Describe the bug
If a DHT11 Sensor is defined but not connected (or defective) MQTT returns the following JSON
{ "Time": "2018-10-23T23:47:58", "DHT11": { "Temperature": nan, "Humidity": nan }, "TempUnit": "C" }
This is no valid JSON. Using null instead of nan would be valid. Also 0 as numeric value would be okay

_Also, make sure these boxes are checked [x] before submitting your issue - Thank you!_

• [x] _Searched the problem in issues and in the wiki_
• [x] _Hardware used_ : sonoff basic
• [x] _Development/Compiler/Upload tools used_ : PlatformIO
• [x] _Provide the output of command_status 0 :

STATUS 0 OUTPUT HERE
23:58:01 CMD: Status 0
23:58:01 MQT: stat/stadtweg/esp01/STATUS = {"Status":{"Module":1,"FriendlyName":["Sonoff"],"Topic":"stadtweg/esp01","ButtonTopic":"0","Power":0,"PowerOnState":3,"LedState":1,"SaveData":1,"SaveState":1,"SwitchTopic":"0","SwitchMode":[0,0,0,0,0,0,0,0],"ButtonRetain":0,"SwitchRetain":0,"SensorRetain":0,"PowerRetain":0}}
23:58:01 MQT: stat/stadtweg/esp01/STATUS1 = {"StatusPRM":{"Baudrate":115200,"GroupTopic":"sonoffs","OtaUrl":"http://sonoff.maddox.co.uk/tasmota/sonoff.bin","RestartReason":"Software/System restart","Uptime":"0T01:05:21","StartupUTC":"2018-10-23T20:52:40","Sleep":0,"BootCount":20,"SaveCount":56,"SaveAddress":"F4000"}}
23:58:01 MQT: stat/stadtweg/esp01/STATUS2 = {"StatusFWR":{"Version":"6.2.1.18","BuildDateTime":"2018.10.23 22:08:26","Boot":31,"Core":"2_3_0","SDK":"1.5.3(aec24ac9)"}}
23:58:01 MQT: stat/stadtweg/esp01/STATUS3 = {"StatusLOG":{"SerialLog":2,"WebLog":2,"SysLog":0,"LogHost":"","LogPort":514,"SSId":["FRITZ!Box Turbo Cable",""],"TelePeriod":300,"SetOption":["00008009","55818000","00000001"]}}
23:58:01 MQT: stat/stadtweg/esp01/STATUS4 = {"StatusMEM":{"ProgramSize":480,"Free":520,"Heap":18,"ProgramFlashSize":1024,"FlashSize":4096,"FlashMode":3,"Features":["00000407","0F8AE794","240083A0","23B617CE","00003BC0"]}}
23:58:01 MQT: stat/stadtweg/esp01/STATUS5 = {"StatusNET":{"Hostname":"stadtweg/esp01-6645","IPAddress":"192.168.178.96","Gateway":"192.168.178.1","Subnetmask":"255.255.255.0","DNSServer":"192.168.178.74","Mac":"5C:CF:7F:33:79:F5","Webserver":2,"WifiConfig":5}}
23:58:01 MQT: stat/stadtweg/esp01/STATUS6 = {"StatusMQT":{"MqttHost":"192.168.178.46","MqttPort":1883,"MqttClientMask":"DVES_%06X","MqttClient":"DVES_3379F5","MqttUser":"DVES_USER","MqttType":1,"MAX_PACKET_SIZE":1000,"KEEPALIVE":15}}
23:58:01 MQT: stat/stadtweg/esp01/STATUS7 = {"StatusTIM":{"UTC":"Tue Oct 23 21:58:01 2018","Local":"Tue Oct 23 23:58:01 2018","StartDST":"Sun Mar 25 02:00:00 2018","EndDST":"Sun Oct 28 03:00:00 2018","Timezone":99,"Sunrise":"08:23","Sunset":"18:46"}}
23:58:01 MQT: stat/stadtweg/esp01/STATUS10 = {"StatusSNS":{"Time":"2018-10-23T23:58:01","DHT11":{"Temperature":nan,"Humidity":nan},"TempUnit":"C"}}
23:58:01 MQT: stat/stadtweg/esp01/STATUS11 = {"StatusSTS":{"Time":"2018-10-23T23:58:01","Uptime":"0T01:05:21","Vcc":3.256,"POWER":"OFF","Wifi":{"AP":1,"SSId":"FRITZ!Box Turbo Cable","BSSId":"F0:9F:C2:DD:2E:79","Channel":1,"RSSI":60}}}
````

**To Reproduce**
_Steps to reproduce the behavior:_ Define a DHT11 Sensor in Web UI. Don't connect or disconnect the Sensor data pin


**Expected behavior**
_A clear and concise description of what you expected to happen._

a valid JSON would be

`"DHT11": {
        "Temperature": "nan",
        "Humidity": "nan"
    }`
or
`"DHT11": {
        "Temperature": null,
        "Humidity": null
    }`
or
`"DHT11": {
        "Temperature": 0,
        "Humidity": 0
    }`
**Screenshots**
_If applicable, add screenshots to help explain your problem._


**Additional context**
_Add any other context about the problem here._
The JSON Object should be valid also if errors occur. Maybe, in case of sensors, an additional error sub object could be included. E.g.:
case no error:
`{"Time":"2018-10-23T23:47:58","DHT11":{"Temperature":20,"Humidity":50},"TempUnit":"C",SensorError:null}`
case error
`{"Time":"2018-10-23T23:47:58","DHT11":{"Temperature":null,"Humidity":null},"TempUnit":"C",SensorError:"sensor not respond"}`

Then other software could check 

if(!obj.SensorError){
console.log(obj.DHT11.Temperature);
} else {
console.log("Sensor error");
}
```

(Please, remember to close the issue when the problem has been addressed)

[ascillato2]

Hi,

Why you said that a JSON like "DHT11": { "Temperature": nan, "Humidity": nan } is invalid?

I have just tested with Home Assistant and it parses that JSON without any error and understand that it is a sensor error. Also I tried to make a math operation in Home Assistant with that (like a correction) and no issue. It understood that is not valid.

Besides that, I think that giving 0 will not be a valid JSON. 0 temp can be understood as a valid value. So, that should be avoided.

[Tommy-LSA]

1. Do you agree, that https://jsonlint.com/ is the standard for testing JSON for validity? I don't use Home Assistant.

2. Open the Console of Chrome (F12) and check the following:

JSON.parse({ "Time": "2018-10-23T23:47:58", "DHT11": { "Temperature": nan, "Humidity": nan }, "TempUnit": "C" }) VM56:1 Uncaught ReferenceError: nan is not defined at <anonymous>:1:71 (anonymous) @ VM56:1

[Tommy-LSA]

If 0 could be missinterpreted, use null as I have suggested too.

[ascillato]

Hey, thanks for the link ( https://jsonlint.com/ ). That is very useful! :+1:

[curzon01]

To be complete with links, JSON has some (historical) standards:

• RFC 4627
• RFC 7159
• ECMA-404

For the literals false null true the null would be the best in that case
0 is a valid value and this is wrong if sensor is not connected because from where is 0 coming if no sensor is available?

[Tommy-LSA]

I totally agree. That's why I suggest the option with adding an additional Error flag in JSON. Another Option would be to totally suppress the MQTT message of a non responding sensor. But this may making it hard to identify an issue on hardware side.

I only have checked the DHT Sensor but the "nan" problem could affect other sensors too. Maybe a global check and replace nan->null before sending Message out would be the smallest fix, instead of review all sensor code.

[mprins]

It should be null

https://stackoverflow.com/questions/1423081/json-left-out-infinity-and-nan-json-status-in-ecmascript

[mikep1998]

Not make this more complicated you can wrap the work nan in quotes this also fixes the issue of parsing test and not a number.
'nan' works with HA.

thanks

[mikep1998]

This is a bug. I have 2 sensors when 1 is not working the HA will not parse the payload correctly and indicates that both sensors are 'unavailable'.

tele/Sonoff_bath_fans_topic/SENSOR = {"Time":"2018-10-24T15:10:29","SI7021-04":{"Temperature":nan,"Humidity":nan},"SI7021-14":{"Temperature":22.4,"Humidity":63.4},"TempUnit":"C"}

When the nan is put in quotes that the information is parsed correctly and HA display the correct temperature and humidity for the working sensor and "nan" for the not working sensor. If you wanted to make this correct for all cases I would replace 'nan' with Not Available "NA" this would indicate that the not working sensor is Not Available while also showing the correct information for the working sensor.

[mikep1998]

null works to HA displays 'None' for the bad sensor and the correct information for the working sensor.

[Tommy-LSA]

The problem is simply, that the JSON is malformed. Everybody uses the standard routines (e.g. JSON.parse()) to get an Object from JSON string for further handling. These routines work aligned to RFC specs and fail or return an error. So it's normal that the complete sensor string is wasted (both sensors)

To test for a valid parsable JSON a developer can use:

1. jsonlint.com
2. Chrome console (F12) by using JSON.parse()

If these tests fail a developer can be 100% sure, that other Programs will fail by using these Messages.

Usage of null would be the safe way. It is parsed correctly and the code of a following application can use this information
var temperature = obj.SI7021-04.Temperature ? obj.SI7021-04.Temperature : "sensor error"

[ascillato]

Please, don't worry. We have tagged this issue as BUG a day ago and we will solve it. Please be patient. Thanks.

[arendst]

Will fix it today. Still wondering if it should return "null" or "Null" ;-)

[ascillato2]

Closing issue as it has been Fixed.

Thanks everyone for sharing your ideas.