Omim: F-Droid...?

[arkhan]

This application meets the requirements for repositories added to F-Droid ...?

[Zverik]

It doesn't on two counts:

• Artwork and other assets do not have a clear license (and marked as copyright my.com).
• Releases are not tagged, but done on branches.

But other than that, it does. As far as I understand, we do not object to publishing the app on f-droid, though we'd prefer only release versions to be published.

[lorenzhs]

See https://github.com/WhisperSystems/TextSecure/issues/127 for some possible arguments against distribution on F-Droid. Many do not apply here, I just wanted to list it for completeness.

Quick summary of those that might apply here:

• enabling "allow 3rd-party APKs" is awful from a security perspective
• no stats and crash reporting, you need to roll your own if you want that
• privacy win of avoiding Google Play is not as big as you think, most phones still have GSF
• people who just download the APK from F-Droid don't have an upgrade path
• you should build and sign yourself, using F-Droid only as a distribution channel -- it seems that their signing keys are not kept sufficiently offline to prevent tampering should they get hacked: https://github.com/WhisperSystems/TextSecure/issues/127#issuecomment-26461194

These issues are a lot less critical here than they are in an app that people rely on to keep them secure, so I think it's entirely doable to distribute Maps.ME on F-Droid, but doing it well would probably require some work.

[Zverik]

Well, these are cons of downloading from F-Droid, not of publishing the app there. We definitely won't use it as an official distribution channel.

[ghost]

It's encouraging that devs are open to the idea of supporting F-Droid. Although I use it personally, I realize it is a very niche audience. Thank you for working with the community.

From my perspective I can see a few distribution options.

1) Third party binary repo

E.g. The Guardian Project

This would not be much different than the APKs you currently offer on your website, but with increased security of file downloads.

Pros:

• Relatively simple, minimal maintenance
• Easy transition from Play Store or other distribution methods that are signed with the same key

Cons:

• No verification that binary matches source code

2) Reproducible builds
All proprietary components must be removed/replaced.

Pros:

• Developer can sign with own key
• Developer control over app content

Cons:

• Requires a separate gradle flavor to maintain
• Complexity of build process

3) F-Droid Source-built version

E.g. Fennec F-Droid, Telegram FOSS

May need a generic "white-label" branding to comply with copyright.

Pros:

• Maps.me devs do not have to support the fork
• Source is guaranteed to correspond to binary

Cons:

• Resource intensive to maintain, may be out of date

I recommend setting up a binary repo if you would like to support F-Droid users until a more permanent solution such as a reproducible build can be achieved. The first two have the advantage of being used as official distribution channels.

[pizzamaker]

A thread for its inclusion has been created at https://f-droid.org/forums/topic/maps-me/

[Zverik]

Thanks!

[HalosGhost]

I would just like to throw in my support for a custom repo. This makes it available to users through F-Droid but avoids almost all of the problems that Whisper ran into (frankly, I'd really love to see them do this too)!

Either way, keep up the rockin work!

[mvdan]

Just looked through the source code and it would indeed be easier for everyone if you just set up your own F-Droid repo. Which is just a few files on an HTTP server, really.

Once you do, you can set up a QR code for people to scan like this one: http://grobox.de/fdroid/

[relan]

Unfortunately, the source code of Android app is incomplete. I've found the following dubious modules:

• FlurryAnalytics-6.0.0: blob with unknown license
• gson-altered-2.2.4: blob with unknown license
• myTracker: blob with unknown license
• com.google.android.gms:play-services-location:7.8.0: proprietary blob
• com.google.android.gms:play-services-analytics:7.8.0: proprietary blob
• com.google.android.gms:play-services-plus:7.8.0: proprietary blob
• com.facebook.android:facebook-android-sdk:4.6.0: source code is available under EULA

Blobs are blockers for inclusion into F-Droid. Facebook's EULA will most probably be rejected too because it's unclear whether this is FOSS or not.

[HalosGhost]

@relan, well, that is a blocker for inclusion in the official F-droid repo. But as mentioned previously, hosting a repo specifically for this can allow anything so long as it is legal for distribution. Ideally, I would like to see a version of this lacking the blobs you mention, but they do not actually stop it from being distributable in a custom F-droid repo as far as I know.

[mvdan]

Both @relan and @HalosGhost are correct. I concur with the latter clarification of "inclusion into F-Droid", since that applies to our main repository. Third party repositories can have their own rules.

[relan]

@HalosGhost, you are absolutely right. I was talking about inclusion into the main F-Droid repo because I don't see any point in unofficial repo for MAPS.ME: if you accept proprietary software that tracks you, just use Google Play; if you don't, you cannot use MAPS.ME regardless of the installation source.

I've found another issue: private.h file is missing. It should contain addresses of servers used by the app. Obviously the app is useless without servers because you cannot even download a map.

[Zverik]

To make an empty private.h, run configure.sh from the repo root (see docs/CONTRIBUTING.md). You can copy maps to a device manually, or set up your own server.

[mvdan]

@relan I mostly agree, but there's a difference - you can freely fetch the apks without having to use Google Play and a Google account. You could do that by just uploading apks somewhere, but F-Droid adds the interface, security (index signing and apk checksums) and update notifications.

[HalosGhost]

@relan, I have to agree wtih @mvdan on this point. One of the obvious use cases for F-droid is to have as close to only free software on your device as possible. But another big one that many people forget is the ability to install applications without needing access to Google Play.

[relan]

@Zverik, thanks for clarification. Please consider publishing those addresses. They are not a secret anyway but their absence hinders development for newcomers.

Also, could you publish the source code of gson-altered?

[joncamfield]

Also, one can run f-droid repositories in offline locations; lots of valuable use cases if there's also a way to side-load maps and bookmark data.

[ghost]

@joncamfield Sounds like it would work well in Cuba or anywhere else with expensive or unreliable internet access.

Using F-Droid swap would be a great way to help those users avoid expensive data charges, use and possibly contribute to OpenStreetMap, and be exposed to FOSS.

The upshot for MAPS.ME is increased brand exposure and first-mover advantage. @Zverik what do you think about this?

[joncamfield]

I was actually thinking of things like national parks / hiking trails and off-the-beaten-path tourist spots (or -- with Swap, backpacker hostels where people could swap placemarks) Obviously, it would also be useful in low-connectivity environments around the world!

[utack]

Released as of today, can be closed?

[Zverik]

I wonder how does it do map downloading...

[miclill]

Great. Thanks! (It's >50mb. maybe this could be changed?)

[ghost]

@miclill The F-Droid version is an unofficial fork so don't expect it to be supported by the devs here. Questions about that specific build should be directed to its issue tracker.

[relan]

@Zverik,

I wonder how does it do map downloading...

It uses the same servers as the official builds (commit).

I hope you won't consider this fork as a competitor. It targets a different audience that wouldn't use MAPS.ME otherwise due to proprietary dependencies. It's a win for everyone: F-Droid users get a great navigation app, you get new users (many of which are quite advanced and socially active). I'd like to cooperate with the upstream as much as possible, so feel free to contact me.

Many thanks to the MAPS.ME team and Mail.ru for making all this possible!

[biodranik]

You have removed all analytics from your fork, and still using our servers in not a legal way, without our permission, putting our official users under a risk of servers overloading.

This is gray zone, we definitely can not guarantee anything. Please consider using your own servers for maps as soon as possible.

On Dec 2, 2015, at 08:08, relan [email protected] wrote:

@Zverik https://github.com/Zverik,

I wonder how does it do map downloading...

It uses the same servers as the official builds (commit https://github.com/relan/omim/commit/38e41f90f9be9bde5465c6c7f3017c999a1a6a0e).

I hope you won't consider this fork as a competitor. It targets a different audience that wouldn't use MAPS.ME otherwise due to proprietary dependencies. It's a win for everyone: F-Droid users get a great navigation app, you get new users (many of which are quite advanced and socially active). I'd like to cooperate with the upstream as much as possible, so feel free to contact me.

Many thanks to the MAPS.ME team and Mail.ru for making all this possible!

—
Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161183204.

[relan]

@deathbaba,

You have removed all analytics from your fork, and still using our servers in not a legal way, without our permission

User tracking is a show stopper for F-Droid version because MAPS.ME uses proprietary libraries for that.

I'm quite confused by your statement. Could you please clarify

1. Do users of the official builds loose the right to access your servers if they disable user tracking in preferences?
2. If I make a fork with all the user tracking left but disabled by default (with the ability for a user to enable it easily in preferences), will users of this fork have the right to access your servers?

Please consider using your own servers for maps as soon as possible.

This introduces a very high barrier for F-Droid version. I'll try to find people who would like to run server infrastructure for the project, but most probably I'll have to shut it down. :(

[biodranik]

My user tracking comment was about your statement:

you get new users (many of which are quite advanced and socially active).

Without analytics libraries we can’t count these users as “ours”. If we can’t count, we can’t say that we got new users.
If user installs MAPS.ME from Google Play or AppStore, at least we see download numbers and updates count, even if analytics was disabled by user.

But we actually get higher load on our servers.

We never gave any official permission to use our servers for a forked applications. Even more, you made our private server urls publicly available, without requesting any permission.
Documentation states that you should use your own servers: https://github.com/mapsme/omim/blob/master/docs/INSTALL.md#map-servers

Please, don’t get me wrong. I’m a great fan of Open Source and want to spread good software everywhere. And it was feasible to grant an official permission in your case to use our servers, or even get a separate server, just for f-droid users.

Let’s discuss a possible option which can at least temporarily reduce risks of overloading our servers by f-droid users. If you append string “-fdroid” or “fdroid-“ to ‘res’ variable (after line 45, res = HashUniqueID(res); in file android/jni/com/mapswithme/platform/Platform.cpp ) at least we can block possible DDOS attempts from your fork. This ID is used when client tries to download any map from our servers. If you use your own server(s), it is not used at all.

On Dec 2, 2015, at 11:13, relan [email protected] wrote:

@deathbaba,

You have removed all analytics from your fork, and still using our servers in not a legal way, without our permission

User tracking is a show stopper for F-Droid version because MAPS.ME uses proprietary libraries for that.

I'm quite confused by your statement. Could you please clarify

• Do users of the official builds loose the right to access your servers if they disable user tracking in preferences?
• If I make a fork with all the user tracking left but disabled by default (with the ability for a user to enable it easily in preferences), will users of this fork have the right to access your servers?
Please consider using your own servers for maps as soon as possible.

This introduces a very high barrier for F-Droid version. I'll try to find people who would like to run server infrastructure for the project, but most probably I'll have to shut it down. :(

—
Reply to this email directly or view it on GitHub.

[pizzamaker]

Why not use free analytics tools?

[biodranik]

Btw, Alohalytics was open source and free. But you have removed it too :)

On Dec 2, 2015, at 13:47, pizzaiolo [email protected] wrote:

Why not use free analytics tools?

—
Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161256071.

[mvdan]

As @relan pointed out, the idea was to remove non-free parts of the app, not analytics in specific. And of course no offense/harm was meant toward upstream.

If those analytics are in fact open source, then there is no reason for removing them.

[ghost]

All the maps are available publicly, how can there be a risk of overloading? Or does the app use a different server? Since there are no analytics with the fork, map downloads should be the only communication with MAPS.ME servers. Either way, I wouldn't mind sideloading maps if in app server access was revoked.

P.S. As a user, removal of antifeatures such as tracking is a big advantage.

[biodranik]

This link is just for manual download by users who can’t do it in the app by some strange network issues or provider blocking. It is NOT intended for use in the code.

On Dec 2, 2015, at 17:14, twzkxkan [email protected] wrote:

All the maps are available publicly http://direct.mapswithme.com/direct/latest/, how can there by a risk of overloading? Or does the app use a different server?

—
Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161308786.

[LuccoJ]

And yet it can be used by anyone with a web server. Look, when you release something as open source (under most OSI-approved licenses at least), you are releasing it to be used, modified and redistributed by anyone. It's that simple. Don't want this? Don't release it as open source.
I'm tired of seeing people release things as open source only to later WHINE when the rights that open source grants are actually USED by others. That way, you're trying to make "open source" merely dead letter.
As to "Even more, you made our private server urls publicly available, without requesting any permission", I'm very confused. If the "private" server URLs are listed in the source code, and the source code is open, then they're public. That's also a very linear concept.

[mvdan]

@LuccoJ your understanding of what free software is is plain wrong. Them releasing their software doesn't grant you rights to using their servers. Ranting like this, it seems to me like you'll just get them to regret ever open sourcing this app.

[biodranik]

Our private servers were (and are) NOT listed in our open sourced code. The author of f-droid fork has reverse-engineered urls from the client and put them into an open source without even asking us.

Servers are not an open code. It is a limited resource, which we are paying for and taking care of. And if anything goes wrong with our private servers, millions of our users will suffer. And I want to avoid it by simple preventive measures.

On Dec 2, 2015, at 18:21, LuccoJ [email protected] wrote:

And yet it can be used by anyone with a web server. Look, when you release something as open source (under most OSI-approved licenses at least), you are releasing it to be used, modified and redistributed by anyone. It's that simple. Don't want this? Don't release it as open source.
I'm tired of seeing people release things as open source only to later WHINE when the rights that open source grants are actually USED by others. That way, you're trying to make "open source" merely dead letter.
As to "Even more, you made our private server urls publicly available, without requesting any permission", I'm very confused. If the "private" server URLs are listed in the source code, and the source code is open, then they're public. That's also a very linear concept.

—
Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161330589.

[LuccoJ]

@deathbaba if they were not in the code, then shame indeed on the person who put that in F-Droid. But, apparently, as @twzkxkan pointed out, the maps are available publicly on well-known servers, too...? You say those servers are just for manual download when users have problems, but if the reality is that anyone can point an HTTP client there and download them, I don't see the issue.

@mvdan I understand it fine, and I'm sadly aware that in some countries the law requires one to respect server ToS even when you're accessing an unauthenticated, public HTTP server - which is pretty messed up, if you ask me. But regardless of what a specific jurisdiction is like, I strongly believe that someone telling you that you can only use their (open source) code to access their servers if you don't make certain modifications to it (like removing analytics, or whatever) is blatantly contrary to the free software spirit, and should at the VERY least give you a very big red-letter warning on any F-Droid entry. Furthermore, in my view, you're basically never guaranteed you'll be able to use such a piece of software in virtual perpetuity; instead, it's all in the hands of the original developers, and that is ALSO a deal-breaker for free software ideals.

[mvdan]

@LuccoJ I agree that the situation isn't ideal, but again, ranting at upstream devs will accomplish nothing.

[biodranik]

You can play around and point code to this one public server. And if it will be DDOSed because of that, other servers will continue to work, and users will not suffer.

We intentionally made one specific server publicly available for our users to manually download maps, and for developers to play around with maps.

I repeat again: my only concern is that millions of our users should always have our map servers accessible. If you can guarantee it in any way, you can get our permission to use our private servers.

And just in case if you missed our documentation which says how to set up your own server: https://github.com/mapsme/omim/blob/master/docs/INSTALL.md#map-servers https://github.com/mapsme/omim/blob/master/docs/INSTALL.md#map-servers
And how to generate maps to put on this server: https://github.com/mapsme/omim/blob/master/docs/MAPS.md https://github.com/mapsme/omim/blob/master/docs/MAPS.md

P.S. Free software spirit should not make any harm to real users. They don’t understand it.

[LuccoJ]

@mvdan you see, it's that every time that I have a peek on #fdroid on freenode, it's always about some app that I had noticed the previous day and seemed good news to have in "the free world", being discussed because their developer has gotten MAD about it being released on F-Droid, for one reason or the other. That gets a bit tiresome, as I'm sure you are first in line to understand, and the way I tend to see things at least, I end up preferring software like that not to be released as open source, instead of being "teased" and then realizing it's even harder in practice to keep a hold on it than it is with some proprietary software.

@deathbaba honestly do you really think the small minority of Android users who source their software from F-Droid instead of Google Play will have an impact on your servers' availability?
And in any case, when someone downloads your software, generally they'll start downloading maps (or, exactly one map) almost immediately, so what great gain do you have by having "advance warning" from your analytics in terms of a couple of minutes?

[biodranik]

@LuccoJ I suppose that apk from f-droid can be uploaded into any Chinese store and we easily can get our servers DDOSed, without even seeing these users and without any easy way to block them not harming ‘legal’ users. That’s why I offered at least a patch which can help us to control such situations.

If analytics shows that we have more users from some specific country/app store, we can rescale our server’s load. Or put more efforts/resources to support this new source of users and traffic. We can buy additional servers and traffic to scale it for our new users. And we can use new users in our marketing texts (we just got XXX installs from YYY country!)
And of course it’s not about realtime reaction.

On Dec 2, 2015, at 19:02, LuccoJ [email protected] wrote:

@mvdan https://github.com/mvdan you see, it's that every time that I have a peek on #fdroid on freenode, it's always about some app that I had noticed the previous day and seemed good news to have in "the free world", being discussed because their developer has gotten MAD about it being released on F-Droid, for one reason or the other. That gets a bit tiresome, as I'm sure you are first in line to understand, and the way I tend to see things at least, I end up preferring software like that not to be released as open source, instead of being "teased" and then realizing it's even harder in practice to keep a hold on it than it is with some proprietary software.

@deathbaba https://github.com/deathbaba honestly do you really think the small minority of Android users who source their software from F-Droid instead of Google Play will have an impact on your servers' availability?
And in any case, when someone downloads your software, generally they'll start downloading maps (or, exactly one map) almost immediately, so what great gain do you have by having "advance warning" from your analytics in terms of a couple of minutes?

—
Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161345123.

[LuccoJ]

If you ask me, I'd say that "We subscribe to the free software ideals and we are shipped by F-Droid as transparent open source software" is a more effective marketing text than "We got X installs from country Y", but maybe this is just my own wishful thinking.
I still don't get it, anyway. When people download your maps, your server will see how many IP addresses are downloading and what countries they come from. What's the difference? The only difference I can see is that you can't know that one IP address corresponds to exactly one (and always the same) user, since addresses can be dynamic.

But that's about tracking users, not simply analytics.

[ghost]

@deathbaba 's concerns are reasonable. The reverse engineering risk exists regardless of whether or not the code is present in the F-Droid fork. But it's still best to incorporate the changes proposed if their servers are used and the app is being widely distributed.

It looks like we have three options:

1.)make the changes @deathbaba proposed (unique ID)
2.)setup a community run server
3.)remove internet permissions from the fork and manually download and copy map files onto a device

I'm not sure which server @deathbaba was referencing "You can play around and point code to this one public server", but if they are willing to let one of their servers be used in code that is a fourth option.

[biodranik]

@LuccoJ We can not distinguish between F-Droid users and all other users in the case you described.

And it’s a good question, what is more effective marketing text :)

On Dec 2, 2015, at 19:17, LuccoJ [email protected] wrote:

If you ask me, I'd say that "We subscribe to the free software ideals and we are shipped by F-Droid as transparent open source software" is a more effective marketing text than "We got X installs from country Y", but maybe this is just my own wishful thinking.
I still don't get it, anyway. When people download your maps, your server will see how many IP addresses are downloading and what countries they come from. What's the difference? The only difference I can see is that you can't know that one IP address corresponds to exactly one (and always the same) user, since addresses can be dynamic.

But that's about tracking users, not simply analytics.

—
Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161350276.

[Thrilleratplay]

@LuccoJ Please be respectful. Regardless of your views on what open source software is and interpretation of the Apache License, you must admit releasing the Mape.me code is a monumental gift to the FOSS community. Like Whisper Systems, wanting to have control in the distribution of their software is very reasonable when you think about brand integrity. As far as I know, Whisper Systems issue with Fdroid is that they did not have control over the builds offered by a third party; important for company that sells itself on security. You and I both trust Fdroid implicitly. My phone does not have gApps and install apps exclusively from Fdroid. It is voluntarily run community but, like any online community, its integrity is contingent on its members and kickass moderators which could change one day. Just look at SourceForge. You have the option to uninstall Fdroid but developers may not have the option to remove their app if used for malicious purposes before their name has been tarnished. Also as a software developer, I understand that bugs unfortunately happen. The Maps.me team can test to ensure a logic flaw does not accidentally DDOS their servers but they do not have this level control in a third party fork. Open software is about the openness of sharing of ideas and concepts, that goes beyond just the code but also the people who use and develop it. Open source, open minds.

@deathbaba I will apologize for the rudeness here. As a member of the FOSS Android community, I want to thank you for what Maps.me provided and hope that this has not soured the decision to release the source.

As far as analytics is concerned for Fdroid, if the analytics library is open source I beleive it can be included as long as there is an option to opt-in/opt-out.

Is Github's Large File Storage an option to mirror the map files for Fdroid?

[LuccoJ]

@deathbaba I'm not really sure why you'd want to distinguish F-Droid users if the goal is simply to know how many resources to allocate. But anyway, you could distinguish quite simply: just have a (secret) ID that your proprietary built sends to the servers, and the F-Droid build would have to come up with a different ID, so you could tell which is which. I'm not talking about unique IDs for user, but just an ID for the build.

@Thrilleratplay without implying that this is a similar case, the Trojan horse was a monumental gift, too.
"Having control in the distribution of your software" and "developers having the option to remove their app (from repositories)" are concepts that run directly contrary to software freedom. If you want direct control on, including the ability to REVOKE license to use, your code, then you don't license it under a free software license, because that's EXACTLY what free software licenses are made to prevent.

[patcon]

:-1: @LuccoJ

[biodranik]

@Thrilleratplay Thanks for your support! MAPS.ME has a setting in menu to turn off analytics. About Github LFS, we never consider it because nobody will host petabytes/month traffic for free. But in theory, if they have HTTP links to files and support HTTP partial download it can work. HTTPS probably needs some tuning/testing in the client code, we didn’t tried it yet (see #817).

@twzkxkan @relan Options I see sorted by preference:

1. Use community servers for maps (no any dependency from us, no risks for F-Droid users). Cons: someone needs to generate and update maps on a regular basis. Or at least sync it with our direct link.
2. Do not remove open sourced Alohalytics statistics ( https://github.com/deathbaba/Alohalytics ) so we can count F-Droid users as ours and scale servers if needed.
3. Add “fdroid-“ prefix or “-fdroid” suffix to UniqueClientID so we can block unwanted traffic by user agent in an emergency case.
4. Leave METASERVER_URL empty (I would prefer to clear commit history too), put our direct server into DEFAULT_URLS_JSON and replace OMIM_OS_NAME in Storage::GetFileDownloadUrl to "direct”. At least we lose only one node if DDOS happens.

Some options can also be combined.

And in any case, please add the following flavor into android/build.gradle to make your apk more customized for F-Droid, and build it with ./gradlew assembleFdroidRelease

    fdroid {
      versionName = android.defaultConfig.versionName + ‘-fdroid'
      buildConfigField 'String', 'SUPPORT_MAIL', ‘"[email protected]"'
      android.sourceSets.fdroid.assets.srcDirs = ['flavors/mwm-ttf-assets']
    }

(Yes, I’ve just created [email protected] email for support).

[agilob]

@deathbaba w8 a sec, what do you mean:

in not a legal way,

1. Were there any securities or rules that were broken?
2. Your position is currently against the licence file you published in the repository. You should consider changing the licence.

All the following redistribution rules are satisfied by Fdroid:

 4. Redistribution. You may reproduce and distribute copies of the
      Work or Derivative Works thereof in any medium, with or without
      modifications, and in Source or Object form, provided that You
      meet the following conditions:
      (a) You must give any other recipients of the Work or
          Derivative Works a copy of this License; and
      (b) You must cause any modified files to carry prominent notices
          stating that You changed the files; and
      (c) You must retain, in the Source form of any Derivative Works
          that You distribute, all copyright, patent, trademark, and
          attribution notices from the Source form of the Work,
          excluding those notices that do not pertain to any part of
          the Derivative Works; and

3 In NOTICE file you say:
Please refer to their LICENCE, COPYING or NOTICE files for terms of use.

There is no COPYING file in the repository.

f-droid can be uploaded into any Chinese store and we easily can get our servers DDOSed

You know... I can do the same with any apk by using apk downloader or backuping apk from playstore.

[LuccoJ]

@agilob Use of their servers has legally nothing to do with the license the source code is under.

[mvdan]

Let's drop the licensing discussion. They are not violating their own license, nor are we by publishing it. But they are in their right to ask us to use their servers on their terms.

@deathbaba option 2 can be done. Option 3 can be done along with it, but if 2 is done, is there really any need?

[LuccoJ]

Why don't we call it "tracking" rather than "analytics", since I was on https://github.com/deathbaba/Alohalytics and it describes that's what it does - track potentially any action done inside the app, against unique identifiers such as "basic devices information including IDFA and Google Play Advertising IDs"?
Option 3 seems much better to me, and it would allow F-Droid to publish it without an anti-feature (which, IIRC, would even hide the app from F-Droid's default configuration).

[mvdan]

@LuccoJ as per https://f-droid.org/wiki/page/Antifeatures, any analytics on by default are a tracking anti-feature.

We don't filter out apps with anti-features. We patch play-services-analytics because it is non-free, not because it is tracking. So no reason why we should remove them.

[LuccoJ]

@mvdan I thought to be able to see apps with anti-features, you had to enable that in the options. Maybe that was in an older version of the client, I haven't checked in a while.

[mvdan]

Yes, this was a very old version of the client. Even then, they were shown by default.

[relan]

Options I see sorted by preference:

Good summary, see my comments below.

Use community servers for maps (no any dependency from us, no risks for F-Droid users). Cons: someone needs to generate and update maps on a regular basis. Or at least sync it with our direct link.

I totally agree that this is the best option. After rethinking this situation I've concluded that this is the only way to go because if a fork uses official servers, the upstream may impose arbitrary limitations on it. This is unacceptable for a FOSS project.

Do not remove open sourced Alohalytics statistics ( https://github.com/deathbaba/Alohalytics ) so we can count F-Droid users as ours and scale servers if needed.

It's worth mentioning that MAPS.ME uses 4 (!) user tracking libraries: Alohalytics, FlurryAnalytics, myTracker and Google Analytics. While Alohalytics is open source, the other 3 are proprietary.

Add “fdroid-“ prefix or “-fdroid” suffix to UniqueClientID so we can block unwanted traffic by user agent in an emergency case.

That's interesting. MAPS.ME generates a unique ID for each installation and passes it to the server. So you can track (and block) requests from each particular device even if analytics is disabled or removed.

To be honest, I find your explanation that you need to track users for

• server load balancing
• DDoS attacks prevention

totally unsatisfactory.

Leave METASERVER_URL empty (I would prefer to clear commit history too), put our direct server into DEFAULT_URLS_JSON and replace OMIM_OS_NAME in Storage::GetFileDownloadUrl to "direct”. At least we lose only one node if DDOS happens.

This has nothing to do with DDoS. A DDoS attack is a complex (and costly!) thing that uses custom software.

Anyway, thanks for explicitly stating your position. F-Droid will stop distributing MAPS.ME soon (the request has already been merged, need to wait for the next repo index update). For those who may be disappointed by this I'd like to remind that the fork was reported to be illegal and thus puts its users at risk. I kindly ask everyone to keep calm.

[biodranik]

@deathbaba I'm not really sure why you'd want to distinguish F-Droid users if the goal is simply to know how many resources to allocate. But anyway, you could distinguish quite simply: just have a (secret) ID that your proprietary built sends to the servers, and the F-Droid build would have to come up with a different ID, so you could tell which is which. I'm not talking about unique IDs for user, but just an ID for the build.

@LuccoJ That’s exactly what I mentioned in option 3.

You know... I can do the same with any apk by using apk downloader or backuping apk from playstore.

@agilob Yes, I know. And we can count these users in this case (analytics is present in official apks) - and it is great! But it is not true for current F-Droid apk.
MAPS.ME licensing you are referring to is about the code, not about trademarks (you can’t use MAPS.ME name or our app icon for your own business/another application without our permission) and other finite resources (our servers).

@mvdan option 3 is not needed if 2 is used. And if you add a separate F-Droid flavor I described, it would be even better (you can push it into upstream branch).

That's interesting. MAPS.ME generates a unique ID for each installation and passes it to the server. So you can track (and block) requests from each particular device even if analytics is disabled or removed.

@relan In theory, we can block users one-by-one, but it does not help in the case of apk leak or other unlikely ddos-related event. If you mark all F-Droid build users by a suffix/prefix it is at least feasible to protect our servers. For example, we can redirect users with this suffix to a separate server for maps downloading. And I don’t like this option too, it’s mostly about some kind of compromise.

This has nothing to do with DDoS. A DDoS attack is a complex (and costly!) thing that uses custom software.
I use this terminology to describe the case of uncontrolled/unexpected high load which breaks our service for millions of our users.

To clarify again: I like F-Droid community and ready to support it, with only one concern: any potential harm/risk to our existing users should be reduced to minimum or better avoid at all.

[relan]

@deathbaba, I again have to agree with you that all options except 1 (completely independent fork with own servers) will be a compromise that won't completely satisfy either side.

To all: are there people, not affiliated with MAPS.ME or Mail.ru, who would like to do server-side job for an F-Droid-friendly fork? Please reply here.

[rugk]

As a user of MAPS.me I also would very much appreciate a release on F-Droid. So :+1: for this issue in general.

From reading this thread I see there are mostly tracking issues. But I think @deathbaba has made multiple kind suggestions how they would allow clients to use their own servers. Generally thanks to them for doing such a nice thing for the FLOSS community and for staying calm in this discussion.

I also think that 4 tracking libraries are too much and not neccessary (do you really need to track each user with 4 libraries to prevent DDOS?!), so all properitary ones should be removed. As a question BTW: If you disable tracking in the settings all of them are disabled, aren't they?
So this is what I would suggest: The one FLOSS library may be included and disabled by default, so it is no antifeature. Additionally you may add this prefix/suffix to the user ID so that MAPS.me can identify the users. If they want it I have no arguments against it from a user perspective as it does not make the user significantly more trackable (the prefix is the same for all users of the build). But what I have not understood yet is one thing: Is this UniqueClientID an ID which is hard-coded into the source, so that you can identify a build (which would help to prevent the Chinse-store-uploads you mentioned as an example) or is it a ID for each device/user of the app as the name suggests? In case the latter is true this is obviously tracking (without using any tracking library) and this antifeature should be removed in the F-Droid build. Alternatively you may just replace it with a hard-coded string (fdroid-<authorname> e.g.) so that it becomes only the "build-tracking" feature mentioned earlier.

---

Another way is obviously the self-hosted server. So why not use rawgit or GitHub pages to host the files? They are available in public and if you can even download them via HTTPS this additional secure won't hurt.*

* Note that you should make sure the TLS connection is properly verified (checking for root certs) in this case.

---

Another general note about all the ones talking about the License: At first the license for open source code does not allow you do use the servers of the authors - that's something completly different. Secondly please don't stick to this "But this is the license". Yes the code is licensed in such a way and you have the right to use it in any way allowed by the license, but please also be human - it is very desirable to have a good relation to the authors and for example there is no argument against making the few modifications for a "F-Droid Flavor" @deathbaba mentioned.

I hope I could give you my point on this and I'd also like to ask whether there is any progress on this issue as the last comment is already two months old.

[rugk]

Any news on this?

[Ezwen]

I would also be very interested in having news on that topic :).

After reading the whole discussion, I think that option 3 is still a very good one for having a first version of MAPS.ME in F-droid. I really wouldn't mind manually downloading and installing the maps, as long as it is explained. It is far from perfect, but would make possible a first set of releases.

And once this is done, we can still hope to have a community server running someday.

[willhelm-mueller]

same for me, currently i use osmand and there i also place the maps manually into the right folder.

[Paviluf]

It would be very great to be able to install Maps.me from F-droid.

[rugk]

As I elaborated earlier the files could just be hosted by GitHub pages or we could even just use the official servers - provided the necessary User Agent/ID changes are made in the fork.

[paride]

The project would gain a lot of popularity and respect among the the free software community if the developers agreed in having maps.me distributed via f-droid.

I don't understand @deathbaba when he writes:

I suppose that apk from f-droid can be uploaded into any Chinese store and we easily can get our servers DDOSed, without even seeing these users and without any easy way to block them not harming ‘legal’ users. That’s why I offered at least a patch which can help us to control such situations.

Would this be a "voluntary" DDoS, or just an excess of users? If it's voluntary I don't think that having the app published somewhere makes any difference, they would just try to make the servers unavailable. But I really don't see the reason to. If it's just a "too many users" problem, why should this come from f-droid and not from the official App store? You already distribute the apk anyway (https://maps.me/apk/), so I can't see the difference. The userbase of f-droid is not _so_ big anyway. But maybe I'm missing the point here.

[willhelm-mueller]

hmhm if someone would make this app 'F-Droid compatible' without using their servers for map retrieval everything should be fine?
An additional, offline OSM Map -> maps.me converter is already described here:
https://github.com/mapsme/omim/blob/master/docs/MAPS.md
So we could use this tool to generate our own map files

Since somebody already cleaned this app from 3rd party trackers / statistic sites it should be doable.

[paride]

The .mwm files http://direct.mapswithme.com/direct/latest/ sum up to about 33GB, so it's not so much. Still GitHub asks for repositories not to exceed 1GB, so using GitHub for hosting is not an option.

[Ezwen]

Why not package a version for which one has to download and install map files manually, for a start?

Le 5 juillet 2016 16:16:20 CEST, legovini [email protected] a écrit :

The .mwm files http://direct.mapswithme.com/direct/latest/ sum up to
about 33GB, so it's not so much. Still GitHub asks for repositories not
to exceed 1GB, so using GitHub for hosting is not an option.

---

You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/mapsme/omim/issues/85#issuecomment-230490747

[paride]

I think that a better solution would be to have a user configurable server address in the app. In this way I can host the map files on my home server and then let friends and family use it.

Ideally this should be added upstream, with the official all coming preconfigured to use the official server. In this way people can experiment with custom maps or self-hosting even with the official app. And of course the official app and the f-droid one would diverge less.

Then, if one day a stable hosting for the map files is found, the f-droid app could come preconfigured to use it.

[willhelm-mueller]

and add

1. Add “fdroid-“ prefix or “-fdroid” suffix to UniqueClientID so we can block unwanted traffic by user agent in an emergency case.

Whereas the UniqueClientID should be erased or be the same for all F-Droids ;)

[Thrilleratplay]

@legovini It isn't the size of the repo that is so much the problem as it is the size of the individual files. Even with Github's large file storage, where you are allowed 1GB of free storage, the bandwidth limit is 1GB/Mo.

I have not found a free for open source hosting solution that can be seamlessly integrated into an app that did not have severely limiting storage and bandwidth restrictions. Perhaps a few Google drives accounts or Mega, but those would be difficult to securely administer for a group. Before @relan deleted his fork, there was the idea of using a bit torrent client or other P2P protocol but no one had the time or development skills to assist with that feature. If it were added, it would likely need to be leech only by default and have an option to only sync over wifi. I can only image someone's phone bill if those went unrestricted. I think this is an optimal solution it eliminates the need to use Maps.Me's server or any server for that matter. I would happily seed this from my home computer.

[paride]

@Thrilleratplay the "torrent" solution is too complex to implement and manage on may points of view, it will not be done.

What I suggest is the server address to be user configurable in the app, thus enabling small scale self-hosting.

[juanitobananas]

@relan, do you happen to still have a git repository somewhere with the official-F-Droid-repository-compliant maps.me app? I am thinking of creating a fork that uses a different server (I want to test it at home at first), and it would be great if I could start with your "freed" version of the app. Thanks!

[relan]

@juanitobananas, nope, I've removed the local copy too because I didn't need it. Anyway, those patches would be heavily outdated now, so you'll hardly miss them.

[juanitobananas]

@relan OK, thank you very much anyway!

[paride]

Great news @juanitobananas, keep us posted. Just for you to know: @mvdan, one of the core f-droid developers, recently mentioned the possibility to host the map files in the form of OBB data on f-droid itself. He said they could handle the traffic. But this would significantly alter how maps.me downloads and updates maps...

[bootofood]

@juanitobananas any news on this?
It is really important to get this on f-droid!

[juanitobananas]

Oh! I forgot about this... :blush:

It is really important to get this on f-droid!

I'd love to see it in F-Droid too, but I took a look at it but it was a little more complicated than I expected so decided not to do it for the moment.

Sorry!

[rugk]

Oh, that's bad. So what's about the offer of @mvdan? Is it still valid?

[tuxayo]

@juanitobananas No problem, time is limited. Can you write about what remains to be done? Especially the complicated stuff, so maybe someone else can continue your work.

[juanitobananas]

@tuxayo :smile: I am very much afraid I didn't implement anything, I just took a look at the code to see if I could find the places where it'd have to be changed. I wasn't very successful at that.

[rugk]

Also here some nice analysis information of the tracking functions in Maps.me by @mnemonicflow: https://github.com/mapsme/omim/issues/5122#issuecomment-270690016

Direct link to gist: https://gist.github.com/mnemonicflow/41c6b7fbf9794a663d2bb7293bb6135e

[ghost]

1. User delete "maps.me" and install "LibreMaps.me"(internet connection permission completely removed edition).
2. I download MWM files from http://direct.mapswithme.com/direct/latest/ to my WWW server(by manual or auto script).
3. You point the URL to my WWW server.
4. You download the map.
5. You use it.

Why Maps.me must track all people just using this app?
Can't this be this simple?

[Zverik]

With open source, "someone" almost always means you :)

[Zverik]

We haven't got any questions in mail about that, but you don't have to consult us about installing a server. There is no complex procedures to configure it, just publish a directory with mwm files, and that's all. You can do with any cheap hosting solution, although you would need at least 35 GB for the whole world.

[rugk]

Also there are many free CDNs, maybe there are also some hosters available, which offer free hosting for FLOSS projects.

[ghost]

According to maps.me support, "we don't support fdroid" "we don't need that niche audience".

Also, they don't(never) add proxy support to Fdroid client(e.g., Orbot support).

I think this issue can be closed.

[Zverik]

So what? We don't support F-Droid, that's true, but the code is open-source and anyone can make a custom build.

[rugk]

Indeed. This issue is only about where (& how) to host the files.
One could maybe use BinTray - it's free for FLOSS projects (1TB download per month, 10GB storage, CDN & HTTPS).

And the files could just be pulled from the official server once day or so.

The only thing we then need is a F-Droid-acceptable fork, which uses the new server.

[Thrilleratplay]

@paride mentioned in August that there is the possibility of hosting the map files on F-Droid's servers.

[paride]

@Thrilleratplay but as an OBB, which would require modifying how maps.me deals with map data. It would be better to find a way to serve the mwm files directly via http, staying near to upstream.

cloudatcost.com is currently offering an 80% discount on their biggest VPS, with 80GB of storage and unmetered monthly transfer. It costs 224$, one time. Of course you get what you pay for, still it could be good enough for serving map data, as occasional downs won't affect the user experience too much.

If somebody is willing to buy the server I'm willing to administer it for free, keeping a mirror up to date. I won't work on the Android app. I can't offer more as I'm currently working on another project, which is already draining the resources I can devote to FLOSS stuff.

[bonanza123]

Frankly, I guess that finding someone that hosts the files is not the problem (I personally can offer ~20TB traffic and ~ 100GB data storage). The "real" work is making the fork.

[tuxayo]

Does a fork is really needed? These F-Droid related changes might not be the core devs priority (and that's fine) but it's totally possible that they would accept well written patches. Can we have a statement on the subject?

[rugk]

It's necessary, already for removing all tracking "features". (or at least making it possible to shut them off)

[Zverik]

I don't think you would need to fork the repo: without a set of private keys all of third-party tracking would be disabled, and the built-in statistics can be disabled in app settings.

[biodranik]

It’s always good to have binaries without any proprietary 3rd party code:

• You never know what are they collecting even without valid initialization keys.
• You waste less CPU/battery (and potentially network traffic).
• Binary size is smaller.
• Some 3party libs can crash at runtime if you don’t provide valid keys to them.

[rugk]

without a set of private keys

Which private keys?

In any case disabling the option in the settings is not enough as @mnemonicflow showed.
So as even @deathbaba argues against 3rd party tracking libs, I assume none are used in Maps.me, are they?

[Zverik]

Check out the build dependencies

[rugk]

So @deathbaba if you dislike these 3rd party tracking libs, why not remove them from Maps.me?

[biodranik]

@rugk I don’t have time to do that once, and definitely don’t want to support it later with every new release.

Btw, it should be relatively easy for any developer.

[rugk]

No, your understanding me wrong. Why don't do it in the upstream version (here is this repo).

But, ah, you do not work for Maps.me any more?

[biodranik]

No, I don’t.

[PanderMusubi]

As https://f-droid.org/forums/topic/maps-me/ has been closed, a new issue can be created at https://gitlab.com/fdroid/rfp/issues?scope=all&state=all in order to continue effort on the side of F-Droid packagers.

[rugk]

Done: https://gitlab.com/fdroid/rfp/issues/87

[rugk]

So @axet is currently removing (proprietary) (tracking) libraries (see gitlab.com:fdroid/rfp#87 (comment)). In case he might need some servers for the maps, you can now offer some… :wink:

[realmain]

Do we have any progress on this?

[rugk]

This issue can be closed! Hooray! :tada:

A fork is available on F-Droid: https://f-droid.org/packages/com.github.axet.maps/ (own servers, adjusted settings, …)

Thanks @axet!! :+1:

[paride]

@rugk good news, the last commit is 8 months old, I'm not sure the fork is actively maintained...

[realmain]

@paride the last commit of the fork is 28 Jan 2018

[LuccoJ]

Yes, I recently tried the fork as it landed on F-Droid, it works fine as far as I could see, aside from a problem with the TTS which I believe to be my own Android problem and not the program's, and it's modern enough to be a viable alternative to OsmAnd if you don't need all the power but do need ease of use and more snappiness for car navigation.

The "live traffic" feature doesn't work, but I assume that's a proprietary service that the Maps.me people wouldn't want to offer to a fork. I guess the button should be removed to avoid confusion. Maybe one day we'll have a decent, free, crowdsourced version of such a service.

[realmain]

@LuccoJ There is OpenTraffic, but I don't know if that project is actually going anywhere. Unfortunately, live traffic is a must have for me due to major traffic jams & construction where I live.

[dimqua]

I assume that's a proprietary service that the Maps.me people wouldn't want to offer to a fork

You are probably right, but we do not know for sure.

[LuccoJ]

@dimqua I don't know for sure if the universe exists or is just an illusion my mind creates, but Maps.me developers have made it abundantly clear throughout this thread that they do not appreciate F-Droid versions of the app (ones that respect F-Droid inclusion rules by disabling analytics by default, at least) using their servers with the theoretical possibility they might impact their servers' performance for other users of the app.

[PanderMusubi]

More external issues to monitor or help progress regarding traffic information are https://github.com/osmandapp/Osmand/issues/1432 and https://github.com/graphhopper/open-traffic-collection/blob/master/README.md

[WPFilmmaker]

Could a compromise be found?e.g. an official version is added to f-droid with a donation page that redirects to this project?

edit: I agree with those who said that the two versions should not be seen as competing, f-droid ones target a niche of users who don't want to deal with Google (also look at numbers google play vs f-droid).

[PanderMusubi]

Donation urls can be added to the listing in F-Droid, that is very common.

[TheCapsLock]

Please ; take care that this app has at least 20 trackers in it : https://reports.exodus-privacy.eu.org/en/reports/100886/ I DO NOT expect to see this app in FDROID regarding this point.

relates to #10576

[WPFilmmaker]

Wow I didn't know it had so many trackers.....one wonders whether google maps is more private than this one then :(

[biodranik]

It is possible to remove all trackers and compile a "clean" version.

[LuccoJ]

@TheCapsLock @WPFilmmaker and @biodranik technically the application isn't in F-Droid. What F-Droid has is a fork called Maps, and its description includes

Removed ads and binaries (peace and freedom)

Now that may not be very specific, but I've run it with TrackerControl enabled, performed some simple actions, and it found no evidence of any tracking. Unfortunately, Exodus can only analyse Play apps, or I'd try with that. In any case, many of the trackers listed look like they are probably based on proprietary libraries, so they are certainly not included in the F-Droid version.

[dimqua]

@LuccoJ https://reports.exodus-privacy.eu.org/en/reports/com.github.axet.maps/latest/

[LuccoJ]

Oh, thanks @dimqua. Honestly the thought it was on Play didn't even cross my mind. I'd say that answers the concerns...

[TheCapsLock]

@TheCapsLock @WPFilmmaker and @biodranik technically the application _isn't_ in F-Droid. What F-Droid has is a fork called Maps, and its description includes

Removed ads and binaries (peace and freedom)

Now that may not be very specific, but I've run it with TrackerControl enabled, performed some simple actions, and it found no evidence of any tracking. Unfortunately, Exodus can only analyse Play apps, or I'd try with that. In any case, many of the trackers listed look like they are probably based on proprietary libraries, so they are certainly not included in the F-Droid version.

I checked an apk of Maps from fdroid using exodus standalone and found no evidence of having trackers embedded in it \o/

[WPFilmmaker]

Any update on this? Any chance to, at the least, switch to FLOSS alternatives thus reducing the number of trackers? The app is great but it kinda sucks that despite being opensource it seems more privacy invasive than closed-source alternatives (google maps,2gis etc.) :/

[tuxayo]

@WPFilmmaker I think there are two option:

• Use the app "Maps" from F-Droid and fund the work of the maintainer: https://liberapay.com/axet/
• Create a bounty here for MAPS.ME to have an official F-Droid flavor

[tuxayo]

As humble attempt to move this forward and contribute financially directly to MAPS.ME, a bounty has been started:

[tuxayo]

• terrible stuff happened to maps.me: https://telegra.ph/What-happened-to-the-old-MAPSME-12-20
• it got forked: https://organicmaps.app/
• it's in F-Droid: https://f-droid.org/en/packages/app.organicmaps/

We can close this issue now :)

Thank you to those who bought Maps.me and ruined it. It actually unblocked the situation and allowed the app to get a new start. A horrible situation can actually be better than a medium terrible.