Gadgetbridge: Amazfit Bip: Support Export of gpx

[bioschaf]

Before opening an issue please confirm the following:

• [x] I have read the wiki, and I didn't find a solution to my problem / an answer to my question.
• [x] I have searched the issues, and I didn't find a solution to my problem / an answer to my question.
• [x] If you upload an image or other content, please make sure you have read and understood the github policies and terms of services

Your issue is:

Is it possible to export GPS data from the Amazfit Bip? That way, it could be used as GPS tracker.

Your wearable device is:

Amazfit Bip

Your android version is:

lineage OS

Your Gadgetbridge version is:

latest

[ashimokawa]

THIS FEATURE HAS BEEN RELEASED IN 0.26

We have that working in bip-wip branch. It needs some cleanup but basically mostly works

[salvisdagis]

Is there any posibilities to export GPX or TCX data from Amazfit Bip?

[danielegobbetti]

The mentioned branch exports data in gpx format.

[salvisdagis]

I compiled "bip-wip" but failed to retrieve workouts, so now I use edited a different app edited, which can greatly export TCX

[danielegobbetti]

@salvisdagis sharing logs of our branch showing the errors of Gadgetbridge would help more in fixing issues, but it's good - if off topic - that you found a working solution.
The merge is not released exactly because it needs more work and polish, having said that it works for me. Which firmware were you on when you had issues?

[salvisdagis]

I'm use Firmware v0.1.0.86.
After push refresh button I see only "Fetching activity data", but after that Activities frame always is empty. I don't know why. I'm also run in Debug mode, but breakpoint never stops in "lucky place".
It would be very nice if I could get GPX with bip-wip, because I have found that the Amazfit bip GPS attitude (after export to TCX) is very inaccurate / noisy and in my case (running on a flat area) it's better not to use it at all or to put a strong filter, a mask for an attitude, etc.

[danielegobbetti]

Try going to the debug screen and pushing button "test new functionality"

[andrewheadricke]

Just received my Amazfit Bip and was surprised GPS log exports has so little support across the different Bip related apps.

Initially, I was just running the vanilla GadgetBridge on F-Droid and somehow there existed a couple of BIP_XXXX.log files which contained GPS log data.. but i'm not sure how that happened, and was unable to recreate them.

So I managed to checkout bip-wip branch and get it compiled... reading the comments above I was able to use the debug screen to download the activity into the "Activities" tab. However from there I am stuck.. any advice on how to export to GPX from there?

[danielegobbetti]

The gpx files are in the same folder where you saw BIP_XXX.log entries, the prefix is gadgetbridge-track- followed by the date/time.

Additionally, if you tap an entry in the "Activities" tab it will fire an Intent that should list all apps on your android device which is capable of displaying gpx files (Osmand works well for me).

Of course there is no gpx file for "treadmill" activities.

[andrewheadricke]

I must have screwed something up, cause GadgetBridge is no longer creating those BIP_xxx.log files and doesn't appear to be creating the gpx files either.

The gadgetbridge.log and db exports still appear there though.

[danielegobbetti]

Does tapping an activity work?

[andrewheadricke]

Tap does nothing, a long press brings up the delete which works.

I have breakpoints on the "gadgetbridge-track-" and its not even hitting the breakpoint.

Looks like handleActivityFetchFinish success = false, so the files are not being written.

[danielegobbetti]

Do you see a track plotted on the watchscreen for these activities?

[andrewheadricke]

Yup

[danielegobbetti]

Then please share your firmware version, as I guess there is something different/new in the message contents.

[andrewheadricke]

Its the last stable firmware 0.1.0.86

[andrewheadricke]

I'm not an Android dev so forgive me for not just fixing the issue.. i'm struggling to debug this properly.. but on line 158 of AbstractFetchOperation.java it cheks that the Metadata = MiBand2Service.RESPONSE_FINISH_SUCCESS which is [16,2,1], but my MetaData = [16,2,4]

Which is causing the success = false.

Update: So after going to bed, the next day I recorded another activity and it appears to sync normally now. The correct Metadata code [16,2,1] is coming through. Not sure whats changed.

Update 2: It looks like the activity recorded the previous day sync'd fine with Metadata [16,2,1], however the new track just recorded now does NOT sync with Metadata [16,2,4]. I wonder if theres some post processing that the watch performs that needs to happen before the Metadata changes to [16,2,1]?

[andrewheadricke]

I have been doing a lot of debugging trying to get to the bottom of this.. I have 3 saved activities on the watch, it can download the first 2 and not the last, however activity number one is in the correct geographical location but has activity number 2's track log.

I wonder if this "off by one" bug is also the reason the tracks for the most recent log don't download.

Also, I have a hunch Daylight savings may be the culprit.

Update: After tweaking the syncTime passed into FetchSportDetails I can successfully download all the activities.

Update 2: Ahh, this line Line 187 has a FIXME to include timezone.. thats probably the reason :)

[andrewheadricke]

Ok, at the line 187 mentioned above I have made these changes..

Calendar cal = Calendar.getInstance();
int dstoffset = cal.get(Calendar.DST_OFFSET);

cal.setTimeInMillis(BLETypeConversions.toUnsigned(buffer.getInt()) * 1000);
cal.add(Calendar.MILLISECOND, -1 * dstoffset);
summary.setStartTime(cal.getTime());

Also, did the same with setEndTime too.

And it appears to be working fine. I'm not sure why the BIP is returning values not inclusive of DST, maybe thats just how it is. Hope this helps.

[andrewheadricke]

Sorry to spam this thread.. I also had to convert the GPX Exported track point times into UTC format to be compatible with sites like Strava.

[piggz]

I tried out the bip-wip branch, but I cant get this version to do the application level auth/pair step. If I checkout master or 0.25.0 and compile that one, it does do the auth ok. I tried rebasing the branch on master (and fixing the conflicts) but it too wouldn't do the auth step. Is this a known issue? Am I doing something wrong?

Out of interest, I was wondering if I could get some tips on doing the GATT auth steps. Im looking to write an app to support the bip in C++ for the SailfishOS and linux desktops. I think ive worked out that there is a characteristic write to the auth uuid, handle 0x004c by looking at your source code, but would appreciate any insight you've got on this.

Cheers

[ghost]

hi, i was able to export tracks to gpx format. However is missing the heart rates in that file.

[ghost]

hi, i dont want to make pull requests etc, so i paste the code here to fix missing HR in GPX. (just notice that Im not a java developer rather .NET :P )

String fileName = FileUtils.makeValidFileName("gadgetbridge-track-" + DateTimeUtils.formatIso8601(summary.getStartTime())+".gpx"); // please add extension to the file

private boolean includeHeartRateByTime = true;

private void exportTrackpointExtensions(XmlSerializer ser, ActivityPoint point, List<ActivityPoint> trackPoints) throws IOException {
        if (!includeHeartRate) {
            return;
        }
        int hr = point.getHeartRate();
        if (!HeartRateUtils.isValidHeartRateValue(hr)) {
            if(!includeHeartRateByTime) { return; }
            Date time = point.getTime();
            ActivityPoint closestPointItem = null;
            long lowestDifference = Long.MAX_VALUE;
            for (ActivityPoint pointItem : trackPoints) {
                int hrItem = pointItem.getHeartRate();
                if(HeartRateUtils.isValidHeartRateValue(hrItem)) {
                    Date timeItem = pointItem.getTime();
                    if (timeItem.after(time) || timeItem.equals(time)) continue;

                    long difference = time.getTime() - timeItem.getTime();
                    if (difference < lowestDifference) {
                        lowestDifference = difference;
                        closestPointItem = pointItem;
                    }

                }
            }
            if(closestPointItem != null) {
                hr = closestPointItem.getHeartRate();
                if (!HeartRateUtils.isValidHeartRateValue(hr)) {
                    return;
                }
            }


        }
        ser.startTag(NS_DEFAULT, "extensions");

        ser.setPrefix(NS_TRACKPOINT_EXTENSION, NS_TRACKPOINT_EXTENSION_URI);
        ser.startTag(NS_TRACKPOINT_EXTENSION_URI, "hr").text(String.valueOf(hr)).endTag(NS_TRACKPOINT_EXTENSION_URI, "hr");

        ser.endTag(NS_DEFAULT, "extensions");
    }

[pietrorim]

I'm sorry, but I'm still a newbie. I use Amazfit bip V0.1.0.86 and Gadgetbridge 0-25-5-5 apk, but exporting the database (using debug mode) I can find in the directory nodomain.freeyourgadget.gadgetbridge only these files: amazfitbip_date_hour.log and Gadgetbridge_date_hour.
Tapping on these files I'm not able to see a gpx path on GPS applications as Osmand or Oruxmaps. Thanks for your suggestions.

[ghost]

are you using Gadgetbridge from google store or compiled from github? if from google then this export will not work for you. If from github you need to check if you using master branch or if bip-wip branch . If you using from master branch that will not work for you as well. You need to download and compile bip-wip branch and then go to debug option and click the latest button. then you will be able to see activities in mobile application.

[ghost]

if you are not able to compile the apk from code. Im sharing already compiled version together with added HR to GPX (my changes) : LINK REMOVED please do not install random APKs, see our blog for our stance on that matter: https://blog.freeyourgadget.org/beware-of-unknown-apks-online.html

[danielegobbetti]

Please remember that this is an issue tracker for the project development/developers, not a user support forum.

[pietrorim]

I installed the compiled version with the added changes, that works perfectly; in particular, I thank swiss5555 for the attention and patience

[pietrorim]

The last thing... I hope. I cannot read or copy in my PC the gpx tracks contained in the directory android/data/nodomain.freyourgadget.gadgetbridge/files (message: file access denied). The same happens for the same gpx tracks contained in the directory android/data/net.osmand/files/tracks. These files are not empty and are correctly completed in the OsmAnd app. Thanks in advance.

[ghost]

maybe the appliction which you are using to browse the files dont have permission to that location. im using Total commander android on the phone. and im able to access the files in both, OSM and gadgetbridge location. however this seems to be rather phone permission issue. eg. i just open endomondo from the browser in the phone, select my profile, and then i see button on the page to upload traning (you need demand PC version of the site). then I just click select traning and navigate to exported file. select it, and i am happy becouse :) becouse I just use phone, no needs for upload to PC first.

[ghost]

there is no possibility to export track anymore after latest upgrade of the watch to version 0.1.1.14

[danielegobbetti]

Does this happen for tracks recorded with new firmware, or also old tracks are not exposed anymore?

[ghost]

there is something strange. becouse after downgraded FW to 0.1.0.86 still is not possible to parse datas.

I tryied to debug that from last 2 nights, without success.
Basically onCharacteristicChanged is triggered by UUID_UNKNOWN_CHARACTERISTIC4, with 15 bits, and that is OK.

Then I receiving notification from UUID_CHARACTERISTIC_5_ACTIVITY_DATA, and buffer is filled up with 127bits.

Then UUID_UNKNOWN_CHARACTERISTIC4 is calling again with 3 bits, 16.2.4 and then UUID_UNKNOWN_CHARACTERISTIC4 is called again with 15bits but expectedDataLength is equal to 0.

When I trying to parse that 127 bits manually with ActivityTrack parse(byte[] bytes) throws GBException {

I get exception (lenght exception).

I wondering is this is only my device, or that also for you is not working?

[ashimokawa]

The data on the device does not change when up or downgrading. So IF the data recorded with .14 is incompatible then it is still incompatible when downgraded. We are aware of two different versions which we both support. I will record data with .14 tomorrow and test.

[ashimokawa]

I can confirm that it is broken with Firmware 0.1.1.17

EDIT: It might have to do with timestamps, like the pervious comments suggest. Will investigate.

[ashimokawa]

It wasn't a firmware issue and should be fixed in master (yep, bip-wip has been merged by now)

Please not that you have to swipe to sync in the "Activities" overview (does no longer work in the debug activity) and that there will be no proper indication whether syncing is done etc.

[sukamare]

Works like a charm, thanks! Though as Swiss5555 noted, heart rate is not included in the gpx file.

[andrewheadricke]

Great work devs.. GPS downloading is working well on master now.

Looks like there is still some issues with the lack of .gpx extension, and the exported XML still has some issues.. I found it really handy to have the option to be able to reset the last sync time in gadgetbridge so that I can redownload past activities if needed. I may look at creating a PR for that --> https://github.com/Freeyourgadget/Gadgetbridge/pull/1057

[andklv]

just got 0.26 from f-droid.
My bip cannot connect to the app (it could in previous versions).

[piggz]

I too can't get this version to connect. Start discover, finds watch, clic!, Says it's creating bond, but never pops up the message on the watch, and gadget bridge just sis waiting.

[danielegobbetti]

@andklv and @piggz unfortunately this release is having issues on older android versions. The fix is already in master and a release should follow soon.

[danielegobbetti]

Basic support was released, hence I am closing this issue.