Core: DTE Energy Bridge no longer working after firmware update

[joeshaw]

For DTE and AEP customers using branded Powerley Energy Bridge devices, a recent firmware update has removed the endpoint used to query for current power demand, at least on v2 devices.

References for this:
https://community.home-assistant.io/t/dte-powerley-energy-bridge-endpoint-error/91160
https://forums.homeseer.com/forum/energy-management-plug-ins/energy-management-discussion/110557-utility-proviced-energy-bridge-how-to-get-info-into-homeseer
https://github.com/joeshaw/powerley-energybridge-homecontrol/issues/1

It's not clear to me from the implementations out there how the original endpoints were discovered. I'm hoping @kylehendricks (the original author from #2247), @Blender3D (who added v2 support in #9431), or somebody else can chime in how they discovered the endpoints, so we can see if such an endpoint still exists after this firmware update.

[joeshaw]

Looks like the original API info may have come from http://myenglife.blogspot.com/2016/02/reverse-engineer-smart-meter-data.html but where the v2 info came from is still unclear.

[kylehendricks]

I found the v1 API's by pulling the energy bridge firmware files out of the android app's APK and did something like strings firmware.bin | grep '$/'. Not sure how the v2's endpoints were found.

Really bummed out about this. Probably going to return the energy bridge and install something myself.

[joeshaw]

@kylehendricks Thanks for the hint. Unfortunately the APKs from AEP and DTE no longer contain any firmware itself AFAICT, and while there are some URL references in the APK that work (/zigbee/se/install and /identity), it doesn't look like there's anything for getting the current usage that I could get to work.

[kylehendricks]

Yea, I'm curious how the v2 endpoints were found. Probably some kind of MITM thing.

I've got a bad feeling they just got rid of the endpoint since it really isn't necessary.

[ronytomen]

@kylehendricks I have to agree with your comment about removing the unnecessary endpoint.

Appears, watching the iOS DTE Insignt Application traffic, that the web service (/identity) is used to retrieve the UUID of the Bridge after discovering it from MDNS. After that I saw MQTT traffic both encrypted and not encrypted between the phone, the bridge and some AWS (Cloud) IPs.

I have not investigated further.

Going to have to look into another solution for measure the entire houses power usage.

[PRabahy]

I just started reverse engineering the DTE app. Looks like DTE uses a service called Powerley. I found this interface that seems useful

    @C7087k(a = {"\u0000,\n\u0002\u0018\u0002\n\u0002\u0010\u0000\n\u0000\n\u0002\u0018\u0002\n\u0000\n\u0002\u0018\u0002\n\u0000\n\u0002\u0018\u0002\n\u0002\u0018\u0002\n\u0002\u0018\u0002\n\u0000\n\u0002\u0010\b\n\u0002\b\u0002\bf\u0018\u0000 \f2\u00020\u0001:\u0001\fJ\u0012\u0010\u0002\u001a\u00020\u00032\b\b\u0001\u0010\u0004\u001a\u00020\u0005H'J\"\u0010\u0006\u001a\b\u0012\u0004\u0012\u00020\b0\u00072\b\b\u0001\u0010\u0006\u001a\u00020\t2\b\b\u0001\u0010\n\u001a\u00020\u000bH'¨\u0006\r"}, b = {"Lcom/powerley/blueprint/network/PowerCoreApiClient$EnergyBridgeService;", "", "sendBindRequest", "Lio/reactivex/Completable;", "request", "Lcom/powerley/blueprint/domain/customer/energybridge/EnergyBridgeBindRequest;", "update", "Lrx/Observable;", "Lokhttp3/ResponseBody;", "Lcom/powerley/network/models/EnergyBridgeUpdate;", "energyBridgeId", "", "Companion", "app_dteRelease"})
    /* compiled from: PowerCoreApiClient.kt */
    public interface EnergyBridgeService {
        /* renamed from: a */
        public static final C4303a f12415a = C4303a.f12414a;

        @C7087k(a = {"\u0000\u0014\n\u0002\u0018\u0002\n\u0002\u0010\u0000\n\u0002\b\u0002\n\u0002\u0010\u000e\n\u0002\b\u0003\b†\u0003\u0018\u00002\u00020\u0001B\u0007\b\u0002¢\u0006\u0002\u0010\u0002R\u000e\u0010\u0003\u001a\u00020\u0004X†T¢\u0006\u0002\n\u0000R\u000e\u0010\u0005\u001a\u00020\u0004X†T¢\u0006\u0002\n\u0000R\u000e\u0010\u0006\u001a\u00020\u0004X†T¢\u0006\u0002\n\u0000¨\u0006\u0007"}, b = {"Lcom/powerley/blueprint/network/PowerCoreApiClient$EnergyBridgeService$Companion;", "", "()V", "BASE", "", "ENERGY_BRIDGE_ID_ARG", "PATH_ENERGY_BRIDGE_ID", "app_dteRelease"})
        /* compiled from: PowerCoreApiClient.kt */
        /* renamed from: com.powerley.blueprint.network.PowerCoreApiClient$EnergyBridgeService$a */
        public static final class C4303a {
            /* renamed from: a */
            static final /* synthetic */ C4303a f12414a = new C4303a();

            private C4303a() {
            }
        }

I really liked this being a local service, but I wouldn't mind it pulling the information from the cloud if needed.

[cscheib]

I used mitmproxy on this a bit earlier... the Powerley REST API that the iOS app connects to doesn't look incredibly complicated, so some of the same data may become available that way...

I haven't had a chance to look into MITM solutions (most likely an RPi running as an AP/Bridge) that would allow me to sniff the local RFC1918 address traffic yet.

While using the app, it still seems to be pulling instantaneous data directly from the unit, since I'm not seeing new requests pop up in the mitmproxy session.

[puddly]

Subscribe to its undocumented MQTT broker:

mosquitto_sub -h 1.2.3.4 -p 2883 -u admin -P trinity -t '_zigbee_metering/event/metering/instantaneous_demand'

You can monitor all topics via # to see what other stuff it sends out.

@joeshaw @kylehendricks @ronytomen @PRabahy @cscheib Please, contact/call Powerley and ask them to provide a public API for their energy bridge (they designed the hardware, the app, and the code running on it). They're probably unaware (I hope) people use the data locally so until they provide an API, you're again going to be one update away from being completely locked out of a device you may be paying for.

Until they release an API, have your router block requests to bothan-auth.pwly.io or disconnect it from the internet entirely to prevent further updates.

[kylehendricks]

Crazy. Had no clue that existed. How the heck did you get the username
and password?

I've actually tried emailing them once last year and never got any response.

On Sun, Mar 10, 2019, 23:28 puddly [e-mail] wrote:

Subscribe to its undocumented MQTT broker:

mosquitto_sub -h 1.2.3.4 -p 2883 -u admin -P trinity -t '_zigbee_metering/event/metering/instantaneous_demand'

You can monitor all topics via # to see what other stuff it sends out.

@joeshaw https://github.com/joeshaw @kylehendricks
https://github.com/kylehendricks @ronytomen
https://github.com/ronytomen @PRabahy https://github.com/PRabahy
@cscheib https://github.com/cscheib Please, contact/call Powerley
https://www.powerley.com/about/contact/ and ask them to provide a
public API for their energy bridge (they designed the hardware, the
app, and the code running on it). They're probably unaware (I hope) people
use the data locally so until they provide an API, you're again going to be
one update away from being completely locked out of a device you may be
paying for.

Until they release an API, have your router block requests to
bothan-auth.pwly.io or disconnect it from the internet entirely to
prevent further updates.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/home-assistant/home-assistant/issues/20170#issuecomment-471392612,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA77xpsBgjhUBoLSELddSdtaXmSN8Dpvks5vVc12gaJpZM4aEDin
.

[bjdiss]

@puddly Thank you! This works perfectly. I too contacted them via email with no response.

[DrDeke]

@puddly Thank you so much; this works perfectly for me, too.

I wonder whether contacting Powerley would be more likely to get them to provide a public API, or to change the password on the MQTT server :/.

[eddyg]

@puddly I had not opened the "DTE Insight" app on my phone in a while. When I connected to instantaneous_demand topic, I didn't get any data. 😞

I then subscribed to _zigbee_metering/# and all I saw was messages like:

_zigbee_metering/request/metering/polling_mode/get {"request_id": "ble_data2"}
_zigbee_metering/response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}

and once a minute I'd see:

_zigbee_metering/summation {"type":"minute","time":1552321920000,"value":884.1077583333333}
_zigbee_metering/event/metering/summation/minute {"type":"minute","time":1552321920000,"value":884.1077583333333}

After I opened the app on my phone, I did start seeing instantaneous_demand messages. The event stream continued after I closed the app. However, after leaving the app closed for 5 minutes, I stopped and restarted mosquitto_sub and once again didn't receive any instantaneous_demand messages... until I re-opened the app.

Any idea what the app does to trigger the Energy Bridge to start sending the "instantaneous" messages?

[puddly]

I'm not sure, mine has been pushing out readings every couple of seconds for over 12 hours now. I did force-quit the app on my phone so maybe it sends something to the bridge to ask it to stop sending out data if the app is gracefully closed/inactive? I'll look around some more and see what the phone sends to trigger this behavior.

[DrDeke]

Mine has been having the same behavior as @eddyg. That is to say, it does not push out instantaneous_demand messages unless I run the phone app. (It stops sending them a few minutes after I regular-quit the app on iOS.)

For my use case, it is actually less work to use the per-minute summation readings than the instantaneous_demand readings. The only reason I was using those before was because I did not know how to access the summations. I can see why the instantaneous_demand readings would be better for other use cases, so it would be interesting if you could find out what the app does to trigger their generation.

[puddly]

@DrDeke Try putting the phone into airplane mode with the app open and then quit the app (I'm not sure if it's possible to completely kill an app with iOS). The Android app has a ton of code so if you can confirm my theory on an iOS device it'll give me some place to look.

[DrDeke]

@DrDeke Try putting the phone into airplane mode with the app open and then quit the app (I'm not sure if it's possible to completely kill an app with iOS). The Android app has a ton of code so if you can confirm my theory on an iOS device it'll give me some place to look.

I will try that (as well as "regular" quit, and "double-click button and swipe up force" quit) later this afternoon, and pay careful attention to the MQTT behavior after doing so.

Note: If you saw my previous comment in the first minute or so it was posted, I mistakenly said that I had force quit the app, when what I actually had done was regular-quit the app. I edited the comment to reflect this, but wanted to point it out here also to avoid confusion.

[joeshaw]

@puddly Great job finding the MQTT stuff. I have tried tweeting at Powerley and sent a private message to an engineer there, but haven't heard anything back.

I tried both a normal shutdown and a force-quit of the app and in both cases the instantaneous_demand events stopped after about 280 seconds.

I'll try mitmproxy on the app to see if there is an obvious correlation from it, but my guess is that the bridge has a 5m timeout after some message is received from the server.

[joeshaw]

mitmproxy doesn't show any ongoing HTTP traffic, so I'm guessing the app connects via MQTT as well and any notifications are probably happening on the backend between the energy bridge and its upstream MQTT connection directly

[DrDeke]

@DrDeke Try putting the phone into airplane mode with the app open and then quit the app (I'm not sure if it's possible to completely kill an app with iOS). The Android app has a ton of code so if you can confirm my theory on an iOS device it'll give me some place to look.

I did this (and left the phone in airplane mode), and the demand readings still stopped after about 5 minutes.

[joeshaw]

You can enable the instantaneous demand response by sending a MQTT message on the _zigbee_metering/request/is_app_open topic. It will accept an empty JSON body ({}) but it's probably better to send a unique request_id in the body.

mosquitto_pub -h 192.168.1.100 -p 2883 -u admin -P trinity -t '_zigbee_metering/request/is_app_open' -m '{"request_id": "foobar"}'

A message will be published on _zigbee_metering/response/is_app_open/<request_id>:

_zigbee_metering/response/is_app_open/foobar {"ack_at":1552335919494,"expires_at":1552336219494}

That starts the 5 minute clock on instantaneous readings.

[ronytomen]

@puddly Thanks for tracking down the MQTT interactions discovery! I wonder once it gets know the MQTT password is out if Powerley will change it.

@joeshaw I was wondering, up until reading your response if they MQTT instantaneous data had a time out.

I had the random thought, which brought me back to this issue, that I wonder if it would be easier to just pull information from the SmartMeter over Zigbee? I do not know how much efforts are involved in doing so, but I wonder if that is a good alternative to Powerley Cloud based product?

Edit:
Reference: https://www.reddit.com/r/homeautomation/comments/68pvjn/extracting_data_from_this_smart_meter/

[puddly]

@ronytomen it definitely would but you need to purchase a very expensive certificate to communicate with smart meters over ZigBee:

I'm sure it's possible to extract a private key from some poorly-secured device but that's not something that belongs in Home Assistant.

[kidmock]

If I'm following this thread correctly, we could use MQTT instead of the DTE Energy Bridge component?

@joeshaw Is this how the sensor would be configured if I'm understanding properly?

sensor dte:
  - platform: mqtt
    name: "DTE Energy Bridge"
    state_topic: "_zigbee_metering/event/metering/instantaneous_demand"
    unit_of_measurement: 'W'`
    value_template: "{{ value_json.demand }}"

[joeshaw]

@kidmock I'm not familiar with the MQTT module, so I can't be of much help, sorry. But what you have there looks good based on the structure of the response. I assume you will need to specify the host and port, however. One thing to note is that their MQTT broker runs on port 2883 instead of the normal 1883.

[puddly]

I think the mqtt platform assumes that the messages will be published to Home Assistant's broker, not to an external one. A simple solution that would work with your above config would be to use Mosquitto as a MQTT bridge between the Energy Bridge and Home Assistant's configured broker, although a better solution would be to just update this component to connect to the Energy Bridge's broker.

[kidmock]

@joeshaw @puddly I managed to be able to get use of the energy bridge again. Here's what I did It atl least looks right so far

If you are already using Mosquitto you can setup a bridge to the MQTT broker on the energy bridge by adding:

connection dte
address <IP_OF_BRIDGE>:2883
remote_username admin
remote_password trinity
clientid homeassistant-1
try_private false
start_type automatic
topic # both 0

Restart mosquitto

If you aren’t using a MQTT broker of any sort you can define the energy bridge as a broker in your configuration.yaml

mqtt:
  broker: <IP_OF_BRIDGE>
  port: 2883
  client_id: home-assistant-1
  username: admin
  password: trinity

In order to keep the data populating, I added a cronjob to run every 5 minutes

*/5 *  * * *   mosquitto_pub -h <IP_OF_MQTT_BROKER> -p <PORT_OF_MQTT_BROKER> -u <USERNAME_OF_MQTT_BROKER>  -P <PASSWORD_OF_MQTT_BROKER>  -t '_zigbee_metering/request/is_app_open' -m '{"request_id": "ha-monitor"}'

No matter which option you choose above, you can then define a new sensor in your configuration.yaml like:

sensor energy:
  - platform: mqtt
    name: "DTE Energy Bridge"
    state_topic: "_zigbee_metering/event/metering/instantaneous_demand"
    unit_of_measurement: 'W'
    value_template: "{{ value_json.demand }}"

[DanielWinks]

Looks like this has been changed again, at least for AEP users.

_zigbee_metering/event/metering/instantaneous_demand has been changed on my device to event/metering/instantaneous_demand. Updating my MQTT topics accordingly re-fixes my sensors.

[dalklein]

Anyone else noticed changes to how the energy bridge messages work?

The one minute interval messages still are sent by the energy bridge:
summation {"type":"minute","time":1560107520000,"value":-861.6829583333333} remote/summation {"type":"minute","time":1560107520000,"value":-861.6829583333333} event/metering/summation/minute {"type":"minute","time":1560107520000,"value":-861.6829583333333} remote/event/metering/summation/minute {"type":"minute","time":1560107520000,"value":-861.6829583333333}

One minute is ok, But the 3 second would be nice. The 3 sec interval instantaneous demand messages seem to only happen if the Insight app is open and active on the phone screen. Close the screen and the messages stop in a minute or two. Until a couple weeks ago, the bridge was publishing the demand messages, without the app open and without me sending anything special, just subscribing to them.

When the app is opened, there are quite a series of messages and the instantaneous messages commence. (funny, it had my lat & lon in there, I guess the energy bridge knows?)
But I'm not able to figure out how to spoof the app being active by sending a mosquitto_pub message, like the request/is_app_open message suggested by @kidmock

`request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}          App not open above, now opened it:

remote/request/announce {"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-0","timestamp":1560108289149}
remote/response/announce/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-0 {"device":"fa904b7f-32c1-49cb-adc4-80cc9c80883f","eb_os_channel":"stable","eb_os_version":"v1.9.3","nucleus_version":"1.9.3","private_address":"0.0.0.0","serial":"073459d4e6b58396","status":"online"}
remote/request/timezone/set {"options":{"timezone":"America/New_York","lon":-83.0,"lat":42.0},"timezone":"America/New_York","lon":-83.0,"lat":42.0,"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-1","timestamp":1560108289253}
remote/response/timezone/set/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-1 {
  "code": 0
}
remote/request/ha_device/device_list {"options":{"authorized_accounts":true},"authorized_accounts":true,"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-2","timestamp":1560108289254}
remote/request/is_app_open {"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-3","timestamp":1560108289256}
remote/response/ha_device/device_list/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-2 {"authorized_accounts":{"ecobee":false},"device_list":[]}
remote/response/is_app_open/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-3 {"ack_at":1560108290037,"expires_at":1560108590037}
remote/request/is_app_open {"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-4","timestamp":1560108289258}
request/metering/polling_mode/get {"request_id": "ble_data2"}
remote/response/is_app_open/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-4 {"ack_at":1560108290157,"expires_at":1560108590157}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
remote/request/wifi/current {"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-5","timestamp":1560108289258}
remote/error/wifi/current/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-5 {"error":"could not find ssid"}
remote/request/demand_response/enlisted_devices {"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-6","timestamp":1560108289262}
remote/request/diagnostics/zigbee {"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-7","timestamp":1560108289265}
remote/response/demand_response/enlisted_devices/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-6 []
remote/request/ha_device/metering/summation {"options":{"start":1557547200000,"end":1560398399999},"start":1557547200000,"end":1560398399999,"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-8","timestamp":1560108289512}
remote/response/ha_device/metering/summation/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-8 {}
remote/response/diagnostics/zigbee/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-7 {"counters":{"Mac Rx Bcast":1,"Mac Tx Bcast":0,"Mac Rx Ucast":37,"Mac Tx Ucast":36,"Mac Tx Ucast Retry":4,"Mac Tx Ucast Fail":0,"APS Rx Bcast":0,"APS Tx Bcast":0,"APS Rx Ucast":17,"APS Tx Ucast Success":18,"APS Tx Ucast Retry":0,"APS Tx Ucast Fail":0,"Route Disc Initiated":0,"Neighbor Added":1,"Neighbor Removed":0,"Neighbor Stale":0,"Join Indication":0,"Child Moved":0,"ASH Overflow":0,"ASH Frame Error":0,"ASH Overrun Error":0,"NWK FC Failure":0,"APS FC Failure":0,"ASH XOff":0,"APS Unauthorized Key":0,"NWK Decrypt Failures":0,"APS Decrypt Failures":0,"Packet Buffer Allocate Failures":0,"Relayed Ucast":0,"Phy to MAC queue limit reached":0,"Packet Validate drop count":0,"NWK retry overflow":0,"CCA Failures":0,"Broadcast table full":0},"link_stats":{"averageRSSI":-64,"averageLQI":255,"lastRSSI":-64,"lastLQI":255},"last_message_received":1560108289728,"network_state":3,"stack_status_history":[{"status":"90","statusEnum":"EMBER_NETWORK_UP","time":1560108249945}],"mac_address":"04214c00000096de","install_code":"f1fc3c17f16fd2bddb33","fastpoll_period":5,"has_authorized_link_key":true,"stack_network_state":2,"last_rejoin_reason":0,"last_leave_reason":0,"discovered_networks":[],"network_info":{"node_type":3,"pan_id":"1481","channel":11,"extended_pan_id":"87114a0100810700","mac_address":"00:07:81:00:01:4A:11:87"}}
remote/request/metering/summation/minute {"start":1560107520000,"end":1560194692055,"compress":false,"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-9","timestamp":1560108292057}
remote/response/metering/summation/minute/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-9 �kW�J{����o��x�`�0�H�-
                    >��c;3}C�   >�X�
�$i�
event/metering/instantaneous_demand {"time":1560108292808,"demand":-1441}
remote/event/metering/instantaneous_demand {"time":1560108292808,"demand":-1441}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
event/metering/instantaneous_demand {"time":1560108295849,"demand":-1530}
remote/event/metering/instantaneous_demand {"time":1560108295849,"demand":-1530}
event/metering/instantaneous_demand {"time":1560108298890,"demand":-1482}
remote/event/metering/instantaneous_demand {"time":1560108298890,"demand":-1482}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
summation {"type":"minute","time":1560108240000,"value":-1254.7981781679161}
remote/summation {"type":"minute","time":1560108240000,"value":-1254.7981781679161}
event/metering/summation/minute {"type":"minute","time":1560108240000,"value":-1254.7981781679161}
remote/event/metering/summation/minute {"type":"minute","time":1560108240000,"value":-1254.7981781679161}
event/metering/instantaneous_demand {"time":1560108301928,"demand":-1455}
remote/event/metering/instantaneous_demand {"time":1560108301928,"demand":-1455}
remote/request/metering/summation/minute {"start":1553041920000,"end":1554955199999,"compress":false,"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-10","timestamp":1560108301495}
remote/response/metering/summation/minute/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-10 �,�D
remote/request/metering/summation/minute {"start":1549947600000,"end":1552363199999,"compress":false,"request_id":"Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-11","timestamp":1560108302434}
remote/response/metering/summation/minute/Ys9bzVuUSLassY3QZ1NNmsEwO345AToOZkOUqE1ZW9aMnxncjN1m93TwGfzY2jng-11 (null)
event/metering/instantaneous_demand {"time":1560108304968,"demand":-1425}
remote/event/metering/instantaneous_demand {"time":1560108304968,"demand":-1425}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
event/metering/instantaneous_demand {"time":1560108308008,"demand":-1395}
remote/event/metering/instantaneous_demand {"time":1560108308008,"demand":-1395}
`

[joeshaw]

@dalklein the message has changed since @kidmock’s post. it’s now the remote/request/is_app_open message you need to spoof instead.

[dalklein]

Thanks @joeshaw I was using the newer topic message, but.....

I tried some more, And got it to work. My line in crontab file was missing the username field to run it as, so crontab was not sending it.

`# Example of job definition:
# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * user-name command to be executed
*/5  *  *  *  *  pi     mosquitto_pub -h <bridge_ip> -p 2883 -u admin  -P trinity  -t 'remote/request/is_app_open' -m '{"request_id": "ha-monitor-ha-monitor-ha-monitor"}'

`

Sort of curious why, sending from the command prompt did not work, but from cron it works? Does it require the multiple repeated requests? I had done it manually several times in a row.

Sending the is_app_open request manually does something, it causes a set of event/diagnostics/zigbee messages to show up. but no 3 sec demand data messages.

`response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
remote/request/is_app_open {"request_id" "ha-monitor"} <<<<<<<< should work?

request/metering/polling_mode/get {"request_id": "ble_data2"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
event/diagnostics/zigbee {"counters":{"Mac Rx Bcast":1,"Mac Tx Bcast":0,"Mac Rx Ucast":17,"Mac Tx Ucast":17,"Mac Tx Ucast Retry":4,"Mac Tx Ucast Fail":1,"APS Rx Bcast":0,"APS Tx Bcast":0,"APS Rx Ucast":8,"APS Tx Ucast Success":9,"APS Tx Ucast Retry":0,"APS Tx Ucast Fail":0,"Route Disc Initiated":0,"Neighbor Added":1,"Neighbor Removed":0,"Neighbor Stale":0,"Join Indication":0,"Child Moved":0,"ASH Overflow":0,"ASH Frame Error":0,"ASH Overrun Error":0,"NWK FC Failure":0,"APS FC Failure":0,"ASH XOff":0,"APS Unauthorized Key":0,"NWK Decrypt Failures":0,"APS Decrypt Failures":0,"Packet Buffer Allocate Failures":0,"Relayed Ucast":0,"Phy to MAC queue limit reached":0,"Packet Validate drop count":0,"NWK retry overflow":0,"CCA Failures":0,"Broadcast table full":0},"link_stats":{"averageRSSI":-64,"averageLQI":253,"lastRSSI":-64,"lastLQI":255},"last_message_received":1560114459338,"network_state":3,"stack_status_history":[{"status":"90","statusEnum":"EMBER_NETWORK_UP","time":1560114448030}],"mac_address":"04214c00000096de","install_code":"f1fc3c17f16fd2bddb33","fastpoll_period":5,"has_authorized_link_key":true,"stack_network_state":2,"last_rejoin_reason":0,"last_leave_reason":0,"discovered_networks":[],"network_info":{"node_type":3,"pan_id":"1481","channel":11,"extended_pan_id":"87114a0100810700","mac_address":"00:07:81:00:01:4A:11:87"}}
remote/event/diagnostics/zigbee {"counters":{"Mac Rx Bcast":1,"Mac Tx Bcast":0,"Mac Rx Ucast":17,"Mac Tx Ucast":17,"Mac Tx Ucast Retry":4,"Mac Tx Ucast Fail":1,"APS Rx Bcast":0,"APS Tx Bcast":0,"APS Rx Ucast":8,"APS Tx Ucast Success":9,"APS Tx Ucast Retry":0,"APS Tx Ucast Fail":0,"Route Disc Initiated":0,"Neighbor Added":1,"Neighbor Removed":0,"Neighbor Stale":0,"Join Indication":0,"Child Moved":0,"ASH Overflow":0,"ASH Frame Error":0,"ASH Overrun Error":0,"NWK FC Failure":0,"APS FC Failure":0,"ASH XOff":0,"APS Unauthorized Key":0,"NWK Decrypt Failures":0,"APS Decrypt Failures":0,"Packet Buffer Allocate Failures":0,"Relayed Ucast":0,"Phy to MAC queue limit reached":0,"Packet Validate drop count":0,"NWK retry overflow":0,"CCA Failures":0,"Broadcast table full":0},"link_stats":{"averageRSSI":-64,"averageLQI":253,"lastRSSI":-64,"lastLQI":255},"last_message_received":1560114459338,"network_state":3,"stack_status_history":[{"status":"90","statusEnum":"EMBER_NETWORK_UP","time":1560114448030}],"mac_address":"04214c00000096de","install_code":"f1fc3c17f16fd2bddb33","fastpoll_period":5,"has_authorized_link_key":true,"stack_network_state":2,"last_rejoin_reason":0,"last_leave_reason":0,"discovered_networks":[],"network_info":{"node_type":3,"pan_id":"1481","channel":11,"extended_pan_id":"87114a0100810700","mac_address":"00:07:81:00:01:4A:11:87"}}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
summation {"type":"minute","time":1560114420000,"value":356.3693676850017}
remote/summation {"type":"minute","time":1560114420000,"value":356.3693676850017}
event/metering/summation/minute {"type":"minute","time":1560114420000,"value":356.3693676850017}
remote/event/metering/summation/minute {"type":"minute","time":1560114420000,"value":356.3693676850017}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}
request/metering/polling_mode/get {"request_id": "ble_data2"}
response/metering/polling_mode/get/ble_data2 {"mode":"continuous"}`

[pokemane]

Drop the remote part from remote/request/is_app_open so you're just publishing to request/is_app_open edit unless you're using another broker (that isn't the energy bridge itself) as a bridge, in which case it depends on your remote/local prefixing configuration

[gitmayer]

The summation topic seems to be very useful and doesn't require continuous polling (which seems insane for HA to record all of these events). You can check it out yourself at
mosquitto_sub -h 192.168.X.X -p 2883 -u admin -P trinity -t 'summation/#' -v

Every minute you'll get data like this
summation {"type":"minute","time":1560253740000,"value":797.1413916666667}

Too bad HA can't do more than one MQTT connection.

[DrDeke]

Sigh. As had been predicted, it seems that the new MQTT API has stopped working. I no longer get any output from my v2 energy bridge with the command:

mosquitto_sub -h [IP] -p 2883 -u admin -P trinity -t '#'

nor from the command:

mosquitto_sub -h [IP] -p 2883 -u admin -P trinity -t 'summation/#' -v

The TCP connection does open; I am wondering if they have changed the password or something.

[puddly]

@DrDeke I've not received the new upgrade but I think you need you remove authentication entirely for it to accept your connection now.

[DrDeke]

@DrDeke I've not received the new upgrade but I think you need you remove authentication entirely for it to accept your connection now.

Oh, thanks! That seems to have done the trick! Thanks again :)

[eddyg]

I don't know where we'd be without you, @puddly! Thanks for keeping us in the loop.

How are you preventing firmware updates? (I thought when I looked in to this before, it was doing DNS queries directly to Google's 8.8.8.8 DNS server, so just blackholing some DNS names locally wasn't going to work...)

[puddly]

@eddyg Hopefully you won't need to worry about that any more. As far as I can tell, Powerley isn't planning on providing any official integrations (yet?) but they are aware of the dozens of people desperate to make numbers show up on their Home Assistant dashboard. The local MQTT broker seems like it will be around for a while.

[mtl010957]

I've also received the update and removed authentication to get back to working order, thanks! I've also noticed that with this new setup it is no longer necessary to run any cron job to keep the data coming, so that makes the configuration much easier. The only topics that are coming through are the event/metering/# topics, which is perfect for our needs.

[stale[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue now has been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[joeshaw]

I think we should keep this issue open. It probably makes sense to remove or deprecate the existing energy bridge integration and direct people to using the MQTT integration.

[bdimcheff]

yeah the existing integration shows no signs of ever working again, and the mqtt integration seems to work well for me. It'd be nice if this integration could switch to using mqtt vs. being removed entirely, but I'm not sure if that's practical.

[DanielWinks]

It'd be nice if this integration could switch to using mqtt vs. being removed entirely, but I'm not sure if that's practical.

That's not practical. Current integrations are stand-alone. There's nothing that requires MQTT to be set up in advance for the integration to work.

A good solution for this would be to remove the integration and leave up the documentation page for the current integration, but edit it to state that the integration has been removed and that the new way to bring this data into Home Assistant is to set it up via MQTT. Include some basic directions for accomplishing this and examples, so people could get it set up.

[kidmock]

I think this should be closed the component really can't be fixed at this time. In order to make things clear to the newbie the component should removed and the integration paged updated with documentation on the options (Direct MQTT; and Proxied MQTT) for continued functionality.

I'd be happy to write this up If I knew the format and how/where to submit.

[puddly]

There might be some use in making a component that connects to the Energy Bridge's MQTT broker independently and only emits useful info, after parsing the timestamps and filtering out invalid values. The Energy Bridge is discoverable with Zeroconf so it should be possible to have everything "just work".

[stale[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue now has been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[joeshaw]

Poking this to keep it not-stale. Same reasoning as https://github.com/home-assistant/home-assistant/issues/20170#issuecomment-543161199.

[stale[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue now has been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[joeshaw]

Poking this to keep it not-stale. Same reasoning as https://github.com/home-assistant/home-assistant/issues/20170#issuecomment-543161199.

[balthisar]

Energy Bridge on the way, so, following.

[eddyg]

My Energy Bridge has been working great via MQTT... I have Mosquitto subscribed to the event/metering/summation/minute messages and HA is receiving the data. But here's my question:

Should the units for this sensor actually be "Wmin" (watt-minutes, a unit of _energy_) and not "W" (watts, a unit of _power_)?

I'm thinking event/metering/instantaneous_demand is "watts right now", but the _summation_ event is the actual reported _energy_ used for that minute.

In other words, if I received 60 event/metering/summation/minute messages of 1000.0000, that would total 60,000 Wmin of energy, or 1 kWh.

The reason I'm asking is I'm trying to set up the Utility Meter component in Home Assistant to generate "daily" and "monthly" usage meters, and it needs values with _energy_ units, not _power_ units.

I haven't tested this yet, but my thinking is something like this (where sensor.energy_usage comes from mqtt):

utility_meter:
  monthly_energy:
    source: sensor.energy_usage
    cycle: monthly
  daily_energy:
    source: sensor.energy_usage
    cycle: daily

sensor:
  - platform: template
    sensors:
      daily_energy_kwh:
        friendly_name: Daily Energy
        unit_of_measurement: kWh
        value_template: "{{ states('sensor.daily_energy') | float / 60 / 1000 }}"

      monthly_energy_kwh:
        friendly_name: Monthly Energy
        unit_of_measurement: kWh
        value_template: "{{ states('sensor.monthly_energy') | float / 60 / 1000 }}"

to graph it by day...

[rgviewer]

I was wondering if someone could help me get this working. I have a DTE Bridge v2. I am using MQTT Mosquitto broker to control some Tasmoto plugs and lights. In lovelace I see the entity DTE energy bridge but it shows Unknown.

In my configuration.yaml I have
mqtt:
broker: 192.168.1.208
port: 1883
client_id: homeassistant-2
username: mqtt
password: mqttpassword

sensor:

• platform: mqtt
  name: "DTE Energy Bridge"
  state_topic: "event/metering/instantaneous_demand"
  unit_of_measurement: 'W'
  value_template: "{{value_json.value}}"

and under Mosquitto Broker under configuration I have the following:
logins: []
anonymous: false
customize:
active: false
folder: mosquitto
certfile: fullchain.pem
keyfile: privkey.pem
require_certificate: false
connection: energy_bridge
address: '192.168.1.235:2883'
clientid: homeassistant-2
try_private: false
start_type: automatic
topic: 'event/metering/# in 0'

[heytcass]

I think I found something. Did a port scan with nmap on my Energy Bridge and discovered that port 8888 is open. Navigated to {IP}:8888 on my web browser and got a JSON response. Unfortunately, the response is { "code": "ResourceNotFound", "message": "/ does not exist" }, but at least it's something. Tried {IP}:8888event/metering/instantaneous_demand, and got the same response.

Not sure where else to start poking at for endpoints, but maybe @puddly will have some insight. Or if someone else can use that info to start poking around, it may do something.

[puddly]

There isn't really anything that interesting left in recent versions provided by the built in webserver, just about everything has moved to MQTT.

[heytcass]

@puddly Hmmmm. Okay. Is there a current, up-to-date writeup on how to get the Mosquitto add-on to talk to the device? I'm having similar issues to @rgviewer

[ronytomen]

I finally gave up and returned the energy bridge. This issue was one of the factors, but not the sole reason for the devices demise in my house.

[stale[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue now has been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[joeshaw]

Bumping again, though my local utility (AEP Ohio) is no longer required by law to provide energy efficiency programs 🤦 and are discontinuing their app associated with the energy bridge at the end of November. It remains to be seen whether the device itself will be a brick or if I can still use it through MQTT, so this might be the last time I bump it to keep it alive.

[puddly]

As long as you keep your Energy Bridge offline it should continue to work (unless they start kicking it off of the Zigbee network). I think I ran mine for close to year (when I still had one) without ever letting it go online.

[chrisbalmer]

Adding to what @puddly said, if you want accurate timestamps, allow DNS and NTP access but deny everything else. Otherwise after a power loss, the time will not be correct. That is how I've been running mine for the last 6 months. No issues (once I figured out the time problem).

[combative-american]

Just got a v2 device and came here after I was unable to pull any data from my energy bridge. Is there anyway to get this to work? Otherwise this integration should be removed to so others know its not supported.

[balthisar]

Although I'm happy with MQTT and it doesn't lock me into the stupid HA no-more-YAML ecosystem, I still wonder if there's a way to update this integration to use MQTT, just for discoverability purposes? I never would have discovered that the MQTT exists if not for this broken plugin.

[puddly]

Should be pretty easy to write a client with hbmqtt. You just connect to the bridge's MQTT broker and subscribe to the appropriate topic, which doesn't require any broker configuration on the host or bridging. I believe the DTE energy bridge can also be auto-discovered on the network as well.

[esolutions4ev1]

Just got a v2 device and came here after I was unable to pull any data from my energy bridge. Is there anyway to get this to work? Otherwise this integration should be removed to so others know its not supported.

V1 bridges are still able to use this integration which is why the integration is still around.