Gadgetbridge: Pebble 2 support

[danielegobbetti]

Pebble 2 support is work in progress.
This is an umbrella issue for various remaining Pebble 2 issues.

• [x] Connection only works when paired through the official app or weird workarounds are being used #433
• [x] Reconnect does not work #432
• [x] Installation of watch apps sometimes gets stuck #434

For the history until before we finally got our hands on a Pebble 2, click on details.

tl;dr There is no ETA for the support of Pebble 2. No core developer has a pebble 2 yet, despite having backed the kickstarter and having asked politely if the shipment could be somehow expedited. Pebble Inc. did not document the changes in the protocol despite having been asked politely if they could do so. Last the kickstarter shipment in Europe will probably fall behind the in-store availability, so if you plan buying a pebble 2, you should probably wait a bit or be prepared that it will not work with Gadgetbridge initially.

Long story follows:

1 - The kickstarter: At the time of the kickstarter announcement one of the core developers backed a pebble 2.

2 - "free samples" handed over to developer: later on, the pebble subreddit was invaded by developers who received free samples to prepare their watchapps for the new platform (including HR support and the like). At this point we reached out to Pebble asking whether we could either be treated the same way, or if at least the (already paid!) watch could be shipped early to us. We did get an answer, but not the one we were hoping for. We were also told that black watches usually are sent first, so our dev could choose the black option for the Pebble 2 to get it as soon as possible. At this point we also became aware that the new watches come with a change in the transport protocol (from classic BT to BLE) that requires us to rewrite part of our code, initially only the pebble time 2 was described as a BLE-only watch.

3 - The "please document the thing" phase. Having failed getting our hand on a watch but knowing that old pebbles (and pebble time) also support BLE, we asked if the protocol documentation (available here on github) could be updated to document how we could get our watches working with the new protocol. Again, we did have an answer that this could happen "soon", but eventually it didn't happen. It became apparent that despite _some_ people inside Pebble Inc. are doing what they can to keep the pebble ecosystem as open as possible, this is not a company priority anymore. To confirm this feeling, consider that all the health-related information are completely "guessed" by us (thankfully we can confirm our findings with the reading on the watch screen) as the official documentation does not describe this kind of messages at all.

4 - The "blind testing" phase. We did try to establish a BLE connection with both pebble og and pebble time, we did so numerous times and with lot of effort. Eventually we failed to exchange even the more basic information. The official pebble app exchanges data with the watch only via classic BT on android, so we couldn't even rely on some other pebble user to provide us logs to understand how we should operate. We hope that we will be able to do so with the Pebble 2 as soon as we can get our hands on one, as that watch should use exclusively BLE also on android.

5 - now. Some backers in US received their Pebble 2 (white/acqua pebbles watches were first! Oh the irony). An update was sent to kickstarter backers that watches are going to be distributed to retail shops and this distribution channel has somehow a higher priority. We therefore preordered a second pebble 2 from a big online retail shop, which should be shipped at the end of the month (and was cheaper than early bird kickstarter pledge!). We have no idea when the kickstarter pebble 2 units will be sent to European backers. We are a bit frustrated and wanted to let everyone know about the current standings.

[IzzySoft]

Thanks for the update, Daniele! I bear with you – especially as my question I directed at the Pebble team a couple of months ago to send you one unit remained completely unanswered. I was positively surprised at the initial helpful contacts I had to them – just to be overly disappointed when it suddenly stopped, out of the blue. Company politics can be a quite horrible thing; I can only guess it has to do with that, and the 2 team members having communicated with me initially have been told to stop that (we all have an idea why: they cannot collect our data if we use GB instead of their app).

[danielegobbetti]

Update: the 31/10 shipping date declared initially by the big online retailer has been changed to "will ship as soon as available" so we are expecting further delays.
Thanks IzzySoft for your support!

[IzzySoft]

@danielegobbetti That was natural for me. I'd thought the usual deal back then: writing a good (and honest) article on the device, getting the device for writing the article (to enable me testing it). I did my part (bought one as they were undecided on when they could send me one), and wrote the article. As I already had bought mine, I told Pebble to fulfil their part by sending the device to you, or at least support you with the upcoming firmware details (that was well before 4.0); I'd profit from that as well then, while being able to give something back to the project I've already profited from. Looks like they did none of it – and that was about when they stopped answering me, too. Which is why I can completely understand your frustration: I'm through a similar thing.

And well, I'm tempted to say that delayed shipment was what I've expected already. But I don't say that. Or did I already? :see_no_evil:

[ashimokawa]

Also they dropped the preorder price making the price gap even bigger (paying about 15€ more as early bird kickstarter when comparing to a big online shop in uk, including shipping to my country)

When complaining earlier that it is now cheaper and probably delivered earlier for non kickstarter orders in Europe pebble answered with a canned reply tailored for US customers stating it is still 20-30 dollars cheaper than retail price....

WTF

[IzzySoft]

That will make some potential backers for the next version think twice before jumping on the train. Maybe Pebble forgot about those consequences? Apart from which, that lets me think again whether Pebble really still gives much on community. Saturation point reached? :hushed:

[ashimokawa]

Based on my experience: Europeans should never back anything from any US company on Kickstarter. It will almost certainly cost more than in retail (if planned to retail in Europe).

Pebble cares a lot for their community (we are just not part of it). They have some loyal and almost religious followers. Some even attack people questioning Pebble's policies:

https://www.reddit.com/r/pebble/comments/58mc0o/pebble_time_2_and_gadgetbridge_support_the_end_of/

We will tackle this issue without Pebble's help. I never considered them really open. libpebble was started by @Hexxeh as an independent reverse engineering effort. I used that to start Gadgetbridge and also my own findings. @Hexxeh was contracted as a developer by Pebble later and his library (and the successor libpebble2 developed in-house ) became part of Pebble's SDK.

This openness was a coincidence it is just the legacy which came from the outside Pebble Inc and was then absorbed by the company.

I can't remember when I got valuable information out of libpebble/libpebble2 after development was taken over by Pebble.

So ... fixing this issue is just business as usual. Nothing changed. We were hoping for a change but that did not come.

[IzzySoft]

Agreed, @ashimokawa – "caring for the community" and "being open" are not necessarily the same. Though when combined… that makes the force I love :) And still, community is no one-way road. They might consider us privacy-concerned a "too unimportant minority" to work on our wishes ("not worth binding that much resources"). But then they should be thankful that there is a project dealing with exactly that, so they don't have to. Supporting them with hardware every now and then would be an easy and cheap way to fill that gap. Guess I'm not the only one who bought a Pebble just because there was Gadgetbridge – and wouldn't have bought it otherwise. Simple calculation that. Plus polishes the image :)

OK, [/rant] :innocent:

[Hexxeh]

There's no need for new hardware here. The new BTLE transport can equally be developed and tested with a Pebble Time series watch.

[ashimokawa]

@Hexxeh

Yes, this is what we already assumed, thanks for confirming.

Unfortunately I only own a classic, and that won't work I guess.

So I am waiting for mine to arrive. And then the fun begins. You know best what I am talking about ;)

Thanks again for libpebble, your independent reverse engineering effort was what made me buy a pebble and start Gadgetbridge (and a number of other people who followed).

I hope are doing well on "the other side" ;)

[danielegobbetti]

If you read step 4 in my first message you can see that we tried. The problem is that there's no way (that we found) to get dumps from the official app using ble on android. We did try, we did find the trigger pairing characteristic etc. We weren't successful in establishing a connection though. Difficult tasks are what make things fun, but people need to know what to expect.

[ashimokawa]

I got my kickstarter shipping notification. But in contrast to what pebble promised it did not get it sent from within the EU but from the US.

I only backed the pebble under that false premise. In a long message exchange on kickstarter I was personally reassured that it will be sent from the EU. That is why I took part in kickstarter.

Speechless...

[pseudotechy]

I received my Pebble 2 in the mail yesterday. If there is testing or logs I can generate that will help please let me know and I'll do what I can.

[ashimokawa]

@pseudotechy

At the moment you can only do two things:
1) tell us how shipping and taxes worked if you are from the EU. (Where was it shipped from, through which service, did you pay extra tax)
2) In bluetooth settings, confirm that you can only find a Pebble LE and how the exact name is (you can change the four digits into XXXX for privacy)

Thanks!

[danielegobbetti]

@pseudotechy

3) IF you find also a Pebble without LE in the name (android bluetooth settings), you may try to pair it and see what works :)

[pseudotechy]

1) I'm in USA and it shipped via US Post from California

Phone One Plus One Android 6.0.1 Cyanogen Mod
2) Bluetooth settings only shows up as "Pebble XXXX"
This will pair and the Pebble asks for confirmation that I want to pair.
There is no LE in the name
In Gadgetbridge the Pebble is listed and says "not connected"
I tap on it and it says "connecting" but it never completes

BLExplorer (app on F-Droid)
Shows as "Pebble XXXX"
Indicates LE
this is able to read some of the attributes

I have not paired it with the official Pebble App. Not sure if that would make a difference.

[ashimokawa]

@pseudotechy
Thank you very much for the information.
Since the Pebble 2 only supports LE and we do not support that yet it is totally expected that you could not connect via Bluetooth Classic.
The Pebble2 not being identified as such will require some extra quirks from our side. That was a bit unexpected, since Pebble Time identifies itself as such.
Anyway, development auf Pebble2 support will kick off as soon as I have my watch and some spare time.

[pseudotechy]

Thank you for your work. Let me know if I can help with any testing in the future.

[ashimokawa]

My Pebble 2 arrived an hour ago. Apart from being shipped from the US and not EU(*), it was painless and without any additional cost.

So far I only checked if it is working and could confirm @pseudotechy's findings.

It looks like a good upgrade from my old OG. Smaller in every dimension, better band, updated SoC, current firmware.

Now I hope to get Gadgetbridge working in the not too distant future.

(*)I rechecked my Kickstarter message inbox and yes - pebble promised shipping within the EU in a private conversation.

[ashimokawa]

Before you get too exited, this is all a bad hack and no code will be published in the next days, but ...

Since yesterday I am able to connect to my Pebble 2, receive data from the watch and send phone call notifications.

Apart from missing resource cleanups and tons of code ugliness and hacks there are three major problems that I noticed.

1. Connecting to the pebble only works if I pair device during connection though Gadgetbridge. It will then work until turning BT off on the watch. After turning it on again I have to unpair the Pebble on the Phone and vice-versa. This was a major blocker preventing further research till yesterday until I found out by accident yesterday. This needs a hack in Gadgetbridge since we wont show devices that are not paired and have not been connected yet. For further research and development I can live with it but fixing this is a must.
2. I fail at sending bigger protocol chunks to the pebble, that is why I could only get phone calls working. Also the protocol layers seem to get out of sync so that I cannot send anything meaningful anymore after a while
3. Incoming data eventually also gets out of sync so that we fail to parse anything meaningful. But I could ready pretty much before that happens.

Reasons for 2) and 3) are most probably that I do not use the BLE stack properly. Also I do not fully understand some control packets that sneak between the regular protocol chunks. I figured out some control sequences but not everything.

At the moment I am not stuck, and I am confident that I can improve on 2) and 3) in the next few days (given I can allocate time). For 1) I will try something as soon as 2) and 3) are solved.

[ashimokawa]

More progress:

The good:

I am able to send pretty much anything to the watch and communication in both direction works for a longer time (probably until disconnect or out of range). Notifications, phone calls, taking screenshots etc seem to be no problem anymore. 2) and 3) seem fixed.

The bad:

• Still a lot of hacks in place
• Still 1) from the previous post is unsolved and even worse than I thought. In addition to what I already knew it seems to that I also need to be inside the Bluetooth menu on the watch to be able to connect. And on a Galaxy S4 mini I also have to unpair after every disconnect!
• I could not set a sane MTU so the number of packets sent is ridiculous. EDIT: solved

[ashimokawa]

I have pushed my code to master as is.
It will probably not work for you.
The pairing and connection problem is so bad and fuzzy that I needed a combination of different workarounds on different devices. Could not be worse.

The best chance for a connection is obviously when connecting to an unpaired device and let Gadgetbridge do the pairing after that. Problem is that this won't work without database hacks because Gadgetbridge only lists paired and devices in the database. Not of that would apply to a new not paired pebble.

This needs a lot of work unfortunately 90% of my time was eaten by the unresolved pairing and connection problems.

[ashimokawa]

@pseudotechy
Are you able to do some tests from master or do you prefer waiting till 0.14.0 is released?

[pseudotechy]

I'm happy to, but I'll need some basic instructions building the apk. I have installed Android Studio 2.2.2 on Windows, but lost from there. I also can use an Ubuntu server or Desktop if necessary.

[pseudotechy]

Here is what I have done:
Downloaded the zip of the code
Used Android Studio 2.2.2 to import Gadgetbridge-master\app\build.gradle
Built unsigned APK with these instructions http://stackoverflow.com/questions/16709848/build-unsigned-apk-file-with-android-studio

It generated the two files
app-release-unsigned.apk
app-debug.apk

I copied them over USB and app-release-unsigned.apk would not install, but app-debug.apk did.

I uninstalled the Pebble App and unpaired Pebble on both the phone and the watch
I paired restarted both the phone and the watch
I paired in the Bluetooth menu > Gadgetbridge lists the Pebble, but is stuck on "connecting"

I unpaired in the Bluetooth menu and clicked the + in Gadgetbridge
My watch showed up and the watch and phone allowed me to pair
It returned to the main screen where it is again stuck on "connecting"
I can repeatably unpair from the bluetooth menu and repair in Gadgetbridge, but it will not connect after that.

I turned on logging, but no log file appeared in
Android/data/nodomain.freeyourgadget.gadgetbridge/files/gadgetbridge.log

Let me know what else to try or report on.

[ashimokawa]

@pseudotechy
Thank you for testing. I was expecting this.
But the fact that you have no log file is weird, if you have turned the logging option on.
Pairing a Pebble 2 really seems to be a super complicated process and using the android settings won't be enough it seems. Depending on how it pas paired it is in a different weird state.

I would like to ask you testing the following depending on your current setup:

1) IF you did not reinstall the official Pebble App yet

Go to the Bluetooth menu on your Pebble 2 and while being there try to connect though Gadgetbridge again.

2) IF you already reinstalled the official Pebble App and paired your pebble with it:

Do not unpair, uninstall the Pebble App, install Gadgetbridge again and try to connect without ever having unpaired. If it does not connect, try in the Bluetooth menu like in 1)

Optional if you are brave:

After having paired and successful connected in the official app, turn bluetooth off on the phone. Switch on hcisnoop as explained here:

https://github.com/Freeyourgadget/Gadgetbridge/wiki/Log-Files

Turn bluetooth on, connect to your pebble with the official app, then turn hcisnoop off and send me the hcidump to my email address (you can use a hex editor to check if it contains personal information, it should not as the Pebble App transfers them on the first connect right after pairing that you should not log (eg. addressbook))

PS: in android studio just press the green play button, to install and run, you don't have to mess with apks yourself.

[pseudotechy]

@ashimokawa
I couldn't get my phone to talk to Android Studio so I'll just keep generating APKs in the same way I described above.

2) Per instructions - uninstalled Pebble Official and installed Gadgetbridge. Gadgetbridge was able to connect.
I was able to use the debug menu to send notifications to the watch
I received a real text and emails which were sent to the watch
I was able to see the stock apps on the watch, but it did not see the installed watch faces or installed non-stock apps
I walked away from the phone and watch disconnected. Gadgetbridge still showed that the watch was connected, but the watch showed it was not. I disconnected the watch in Gagetbridge and then requested it to connect and it did.

1) I unpaired the watch and rebooted the watch and my phone. I went to the bluetooth menu on the watch and paired it in Gadgetbridge. I got stuck on "Connecting" again.

I'm working on the hcisnoop log. I'll comment here if it doesn't work.

[ashimokawa]

@pseudotechy
1) I assume you did paired it by connecting to the unpaired pebble right? Not by using the "add new device feature" which would not work. I can connect after pairing by connecting to Gadgetbridge, but only if I turn BT off and on again, then switch to a watchface, press connect in Gadgetbridge, wait 6 seconds then quickly to into the BT menu on the pebble. Magic happens. ;)

2) These information are really interesting, thanks!

What was really unexpected is that Gadgetbridge can connect always if it was paired through the official app. The conclusion is that if we do pairing inside Gadgetbridge right, it should always work.

The rest was expected. We cannot reconnect yet. Disconnect is not properly recognized either. I will fix that later.

About not being able to see the installed apps, that's normal, I think I explained about 5 times already in various issues:

There is no way to ask the pebble for installed apps, the official pebble app just knows what you installed and deletes everything else(!). So if you install apps with Gadgetbridge and then connect with the official app they will be gone. We do the same. Then the App Manager would show what was installed though GB and we would kill everything else. But that would make people angry who just want to try GB once.

Thank you for all the valueable information!! :+1:

[ashimokawa]

@danielegobbetti
Shall we update OP and make this an umbrella issue with checkboxes or shall we close this and just make some more specific issues?

[danielegobbetti]

@ashimokawa I'd change this one but up to you :)

[cpfeiffer]

@pseudotechy Weird that no log file is created for you. Can you make sure that the configuration option is turned on and the restart Gadgetbridge and check the output of Android Studio's "Android Monitor" or the output of adb logcat? We try really hard to find a place where to store the log file, but if all fails, we report that to logcat and continue logging there.

[ashimokawa]

@pseudotechy
You can try current master and press the (+) button in control center and then pair your Pebble 2 through there, it works here, no crazy workarounds needed.

[pseudotechy]

I built the apk with tonight's code.

@ashimokawa I keep running into the same issue that is described in #412 where the phone locks up and reboots after pressing the (+) button.
I completely uninstalled the commercial Pebble app, forgot the phone in the watch and rebooted both.

I tried pairing in the bluetooth menu which worked. After doing that the Pebble shows up in Gadgetbridge, but stalls at connecting and never completes.

Got It!! - OnePlusOne with CyanogenMod (not rooted) comes with Privacy Guard which automatically was blocking access to something. I turned off Privacy Guard for Gadgetbridge and it almost instantly found my Pebble after pressing (+) and connected right up!!!

[pseudotechy]

@cpfeiffer I found the log file exactly where it was supposed to be tonight. Not sure what happened before.

[ashimokawa]

One item to go - but I might release this before marked as experimental.

[ashimokawa]

0.14.0 got released yesterday and we are waiting for f-droid to build it.
Feedback is welcome.

[IzzySoft]

That's faster than I dared to hope – congrats! Not that I'm able to test the Pebble-2 stuff, but I've followed it with much interest – good job!

[ashimokawa]

@IzzySoft

Well... theoretically you could though I didn't tell anyone because I do not recommend it.
It is now possible to pair your "Pebble LE" (inside Gadgetbridge not BT settings) and then use that with BLE!
The reason you should not do it is that you will then distribute your (eg health) data between two devices since Gadgetbridge does not yet know it is one device. I am working on supporting that as one device then you can freely switch between LE and classic connections - a Gadgetbridge exclusive! :stuck_out_tongue:

[itmike]

Super happy for your hard work!

I upgraded gadgetbridge and after removal of offical app and disconnecting my OG Pebble Steel from my phone, I managed to pair my Pebble2 thru gadgetbridge (needed to be really quick to accept the pair on both devices) and it connects fine.

But, the watch only shows it as connected to my phone and no watchface or menu is available. Only shows a message that I should start the app, non of the debug messages is working to it. :(

Could I do something to get the watch to start the menu/watchface?

Edit: I think I found my answer here: https://github.com/Freeyourgadget/Gadgetbridge/issues/433#issuecomment-262205751 So, I am of trying some firmware updates.

[ashimokawa]

@itmike

Yes you have to install a firmware. Unfortunately for me that fails every time :(

Please try anyway, won't brick anything.

You might be forced to use the official app for fw upgrades for now.

[itmike]

@ashimokawa tnx for confirming. Is the 4.3 firmware available for download yet? I noticed it said 4.0.1 on my pebble2.

[ashimokawa]

4.0.1 you saw is the recovery firmware version.

4.3 is available for the Pebble 2, however we only linked 4.2 at this time since 4.3 is untested for Gadgetbridge compatibility:

https://github.com/Freeyourgadget/Gadgetbridge/wiki/Pebble-Firmware-updates

(You might guess the correct link for 4.3 ;) )

[ashimokawa]

I made a minor change to the code and managed to flash fw 4.3

Also latest research revealed new details that will allow me to properly fix the code.

Will release soon with the temporary ugly fix.

[itmike]

Thanks for linking 4.2, it "just worked". :) Verrry slow bt transfer, but worked. Rocking my p2!

Auto-reconnect have some issues, but reconnecting manully works everytime.

Edit: I am using the fdroid version.

[ashimokawa]

@itmike
Thanks for the feedback. I could only install with the changes done after the latest f-droid release (0.14.0). We are still waiting for them to build 0.14.2.

Firmware installation is also much faster now, but still far from being optimal. That is because I do not properly do synchronization and just wait to be sure instead of "optimizing" the data transfer.

Maybe you were just lucky that flashing worked, or maybe your phone behaves differently. You should wait for 0.14.1 or 0.14.2 to pop up on f-droid to flash FW 4.3 which seem to work also fine with Gadgetbridge.

Please also report back if it worked or failed then.

Also 0.14.2 tagged today includes an important fix for BLE disconnects when using more than the Pebble via BLE.

[itmike]

Ok. Got the new fdroid version installed today and then tried an firmware upgrade to 4.3 from the firmware update page. Seems to work as before. Some reconnection issues, as mentioned earlier. Only issue was as before the transfer speed, now ~16 min. How long is an upgrade with the offical app?

But, of course it is better that the update do not fail. :) :+1:

[ashimokawa]

@itmike
The update with the official app should be faster but still lot slower than Bluetooth classic on the "old" pebbles.
We wait 0.5s after each 2013 bytes (transferred in multiple chunks of normally 339 bytes), I assume a firmware update should not take more than 5 min when done correctly.

[pseudotechy]

@ashimokawa @itmike
I updated to 0.14.4 this morning and then tried the firmware update to 4.3
Successful upload in just under 7 minutes on the first try.

The first try comment is important because when I used the official Pebble app it took several tries to update firmware.

[ashimokawa]

@pseudotechy
Thanks for the feedback. I consider the P2 working at an experimental level (while it is stable enough for me personally on a daily basis), hence I will close this issue.