EDIT: tl;dr: We are working an an open implementation of the Google/Apple protocol (I call it PPCP, which may be wrong). You can find and contribute to our repository here: https://github.com/theScrabi/CoraLibre-android-sdk
Dear Corona-Warn-App developpers we have read your code and documentation. We like your effort and your open mind for the community. As there is a small but willing comunity of people who refuse to use closed source Google or Apple dependency and so can not use the Corona-Warn-App, I'd like to discuss the posibility of a Google Free FLOSS solution.
What we know so far is that it's currently hardly possible to create the corona app without the use of the closed source GMS. This comes along with the nearby android exposure notification API, which implements the PPCP protocol. From what we can see in the code so far, the contact points between the android exposure notification API and the Corona-Warn-App are few as for the most common calls to the API a wrapper class is used. That's great! Also there are only about 14 code files that include parts from the GMS library. We therefor think it might be feasible to create a version of the app that does not require GSM services.
As statet in this issue the current maintainer have no plans to implement an own exposure notificatoin API, but might be open to support an alternative if available.
What we understand so far is that the part that runs the actual BLE contact tracing as well as providing a highlevel API for exposure likelihood and key handling would need to be reimplemented. We had several thoughts about how this might be possible.
If a community based solution is possible what would be the next steps to go?
I agree, there should maybe some abstract wrapper class or interface that encapsulates the Google Play Services stuff. Especially as Google already mentioned that the play services stuff may move to core Android (maybe AOSP). Also Huawei may have another variant of the same API without play services on their newer phones.
So I tend to think that another abstraction here would be a good idea, so it's easy to add more exposure notification API code.
On the other hand: If Google moves the stuff away from Google Play Services on newer devices, I am sure the API will stay the same and Google Play services will delegate to the Andoid layer.
Here is a good example of a GSM free fork:
https://f-droid.org/de/packages/org.telegram.messenger/
While the original Telegram app uses GSM, in example for location sharing, the fork is 100% GSM free, without losing any functionality (in this case they use Open Street Maps iirc).
Removing the Google dependency could also improve the user acceptance by wiping concerns against US big data companies.
Personally, I can only encourage you to build a libre version of the ENA API for Android.
But to be crystal clear:
In the foreseeable future the Corona-Warn-App project will rely on the official Google and Apple APIs!
My personal advice to you: Fork this repo, give it a new name (e.g. CoraLibre or CoronaLibre) and integrate it with your libre ENA API. Try to change as few as possible of the upstream code so you can stay really close to this upstream repository. If you have a working fork, completely libre, and some minor changes in the upstream code base would help you to integrate the upstream commits (e.g. by doing some Dependency Injection here and there) feel free to create PRs. We will definitely accept no code changes that have impact on the UX (e.g. the mentioned Android notification hack).
All the best,
Malte
PS: Please be careful when publishing your app: Afaik you may not use the Corona-Warn-App name, brand, logo and designs.
In the foreseeable future the Corona-Warn-App project will rely on the official Google and Apple APIs!
Then in the foreseeable future, it will not be possible for everyone to use the app. I'm using a Fairphone 2 with their Google free Fairphone Open OS, no GSM available.
So the question is: Why is it necessary to use the Google APIs for this app? Are there any technical reasons?
I think the most important feature for such an app is, to make it available to the biggest user spectrum possible. But when using Google APIs this target can't be reached. :-(
@MalteJ That is great! Thank you for the reply. So I guess the next thing is to somehow be able to write a libre ENA API that is compatible to Privacy-Preserving Contact Tracing. The question was can it be pulled off and has someone experiment with it already?
In the foreseeable future the Corona-Warn-App project will rely on the official Google and Apple APIs!
Then in the foreseeable future, it will not be possible for everyone to use the app. I'm using a Fairphone 2 with their Google free Fairphone Open OS, no GSM available.
So the question is: Why is it necessary to use the Google APIs for this app? Are there any technical reasons?
I think the most important feature for such an app is, to make it available to the biggest user spectrum possible. But when using Google APIs this target can't be reached. :-(
I think using Googles / Apples solution is the only option for now, since the development of an own "Google free" solution would cost time and money.
Please read the Notification Exposure Documentation in this project for more information.
@MalteJ That is great! Thank you for the reply. So I guess the next thing is to somehow be able to write a libre ENA API that is compatible to Privacy-Preserving Contact Tracing. The question was can it be pulled off and has someone experiment with it already?
I do not know any project that is trying to create a compatible API. But I know a few folks that would be very interested in such a project and maybe could also help. @theScrabi Are you an Android developer? Can you start developing such an API?
I am an android developer, however I am not sure if I could write such an API as I have little experience with bluetooth and that API (yet). I started to read my self into the documentation and prestandard DP-3T and I think to see some parallels, so maybe by modifying prestandard DP-3T android sdk it might be possible to create a compatible API, but I don't know for certain right now.
If I tried it I'd need help.
@theScrabi you will find most of the documentation here:
https://www.apple.com/covid19/contacttracing/
I think when you start developing a lib there will appear some helping hands from the community 🙂
Feel free to post a link to your project in this thread!
So I've been researching a bit more, as I try to persue the idea of modifying the prestandart dp-3t android sdk. So here is a comparison of both based on what I found out so far (no assurance for accuracy, please correct me if I am wrong):
DP-3T and PPCP are both decentralized and generate a new Token every ~15 minutes. In both protocolls that token derives from a general key that will be newly generated every 24h. On DP-3T that key derives from a $SK_0$ which will be generated only once when the app is installed. Then every 24h a new $SK_i$ is derived from it by applying a HMAC function on $SK_{i-1}$. At PPCP this equivalent is the Temporary exposure key $Tek_i$. It will be generated by a random function so no deriving here.
For DP-3T $SK_i$ and a Constand string are then Hashed and put into a stream cipher function. From this stream every 15 minutes 16 bytes are taken and used as Ephermal ID which is the id that gets broadcasted via Bluetooth.
The equivalent for the EPhid for PPCP is the Roling proximity identifier. First the Rolling Proximity identifier key $RPIK_i$ is derived from $Tek_i$ together with the string "EN-PRKI" using an HKDF function.
About every 15 minutes a rolling proximity identifier $RPI_{i,j}$ is then derived from it by encrypting the $RPIK_i$ together with $PaddingData_j$ padding data contains several information including a counter that will represent the time that $RPI_{i,j}$ was generated. The $RPI_{i,j}$ is then concatenated together with the Associated Encrypted Metadata $AEM_{i,j}$ which contains encrypted information about the sending power and the protocol version. The concatinated data is then broadcasted via Bluetooth.

The Broadcasting in both cases is send as Payload of the BLE Advertising packages.
If someone was found to be Corona positive he/she must upload all $SK_i$ and Timestamps $i$ of the past 14 Days for DP-3T, and all $Tek_i$ for the past 14 Days for PPCP.
One thing that concerned me a little is that the App apparently can not see the received $RPI_{i,j}$. The app can only give the infected $Tek$s it received from the server to the framework, and ask how high the risk is that the user got infected. That the app has no access to the $RPI_{i,j}$ is ok, but here it says that an algorithm calculates the risc based on the exposure duration, however I could not yet find any information about this algorithm.
The next thing I will do is frok the prestandard DP-3T android sdk and try to get it working so we can start modifying it :)
Here is the repository of the CoraLibre-android-sdk. So far it still contains the prestandard DP-3T android sdk.
@MalteJ you said we can not use the name or the package name of "Corona-Warn-App" which is ok. We will not, but can we leave the paths of the package names of the java files? If not pulling changes from this repo will get harder as files are going to be stored in different locations.
You just shouldn't use the name for any marketing stuff. I don't care about path names. :)
We are making progress. The prototype of the encryption core for the Apple/Google is done. It needs to be verified by a security expert:
https://github.com/theScrabi/CoraLibre-android-sdk/tree/master/coralibre-sdk/sdk/src/main/java/org/coralibre/android/sdk/internal/crypto/ppcp
@MalteJ According to this file, page 11, Apple/Google hand out test vectors if you kindly ask them. Do you know if these testvectors are public available or who I can kindly ask to get them?
Great project and amazing work so far, thanks for the effort @theScrabi. One question: Does the work need to have location services on all the time to ensure the background scanning?
Good question. I'd guess if the app uses a "priviliged extention" then no, if you install it on a phone without root access that probably yes.
@theScrabi Unfortunately I cannot provide you any data that we got under NDA.
But you can install the Apturi Covid app and receive their BLE beacons:
https://www.apturicovid.lv/
@theScrabi
but here it says that an algorithm calculates the risc based on the exposure duration, however I could not yet find any information about this algorithm.
The Apple documentation on ENExposureConfiguration explains that algorithm.
@theScrabi
According to this file, page 11, Apple/Google hand out test vectors if you kindly ask them. Do you know if these testvectors are public available or who I can kindly ask to get them?
I don't have official test vectors either, but when I wanted to verify that this implementation is correct, I used a "live" Android device (Android 9, GMS 20.18.17) to generate TEKs, RPIs and AEM (and the Raspberry Pi setup to receive RPIs and AEM).
Malte asked me to provide some of these values for reference:
TEK: 2765f41dbaa6306a264391913bf48723
Derived RPI Key: 34fbbd748ee78bb386d1bab2df7b4165
Derived AEM Key: e01962eba7d4b5f40226e91ef9f6c050
RPI: 31fc2cf219fe6dc6765d860e979ba9e5 (decrypted: 454e2d525049000000000000bb772800) --> interval: 0x002877bb == 2652091; * 600: 1591254600 == GMT: Thursday, June 4, 2020 7:10:00 AM
AEM: 10413a43 (AES-ECB-encrypt of RPI using the AEM Key: 50b33a43150b72c90c1d8756eb9f864b - 50b33a43 XOR 10413a43 == 40f20000 --> TX Power Level 0xf2 == -14dBm
Another valid combination is this
TEK: 7921b817fdb92074df5345594273756f
RPIK: eae8956644770f952871daf549c0ce7e
AEMK: a1d0bd6f94b053cf622ca88194e20611
padded data: 454e2d5250490000000000005b6d2800 --> RPI: 5811408cf8d88d2b33f73773a7c6d45f
metadata: 400c0000 --> AEM: a3e9517f
Hope this helps - any questions, let me know :)
Great project and amazing work so far, thanks for the effort @theScrabi. One question: Does the work need to have location services on all the time to ensure the background scanning?
I found the answer. Seems like you have to enable location mode in order to get Android to locate the Bluetooth device. It is a privacy orientated feature. if its disabled, every Android device will send the same Bluetooth-ID 02:00:00:00:00:00
Seems like you have to enable location mode in order to get Android to locate the Bluetooth device. It is a privacy orientated feature. if its disabled, every Android device will send the same Bluetooth-ID
02:00:00:00:00:00
You are right, it's a privacy oriented feature. What Google wants to prevent is this scenario: An app does _not_ get permission from the user to access location, and then still tracks the user's location by BLE-scanning for fixed-position BLE beacons (iBeacons in stores, museums, etc). Totally different use case, but leads to the requirement to ask the user for location permission, even though the location isn't used within the app.
Regarding the BDADDR: I found that my Android device (MTK chipset) advertises random BDADDR like 4f:31:14:1c:d7:20, 59:7b:48:03:e7:c9, 60:a5:86:6f:9a:d3, 77:11:8d:93:f9:d4, ...
Generally it starts with 4, 5, 6, or 7, i.e the most significant bits are 01. This is a _resolvable private address_, see e.g. Bluetooth Core Spec 4.2.
Whereas iPhones advertise these beacons with a random BDADDR where the msbs are 00 (_non-resolvable private address_).
BTW, if you advertise with an _undefined_ random BDADDR (msbs 10, such as 80:...), Android devices will happily scan the beacon, but iPhones will ignore them, so be careful with that.
Seems like you have to enable location mode in order to get Android to locate the Bluetooth device. It is a privacy orientated feature. if its disabled, every Android device will send the same Bluetooth-ID
02:00:00:00:00:00You are right, it's a privacy oriented feature. What Google wants to prevent is this scenario: An app does _not_ get permission from the user to access location, and then still tracks the user's location by BLE-scanning for fixed-position BLE beacons (iBeacons in stores, museums, etc). Totally different use case, but leads to the requirement to ask the user for location permission, even though the location isn't used within the app.
That's unfortunately also different to the "official" Google Play Services: There your app is NOT allowed to have location services.
This is one reason why the official CWA app can't implement both things at same time:
Both at same time wont work as you app would not be accepted for Play Store.
Actually the approach with Google Play Services is in my opion better for privacy! It sounds strange, but actually the app itsself can't track the location of users if you separate the BLE tracking from the COVID19 app (what Apple and Google do). The play services are a separate service implemented on a lower level in the operating system. Those have all required permissions to do the BLE scanning, but don't expose any privacy critical information to the COVID19 app. That's well thought!
@uschindler While I think you are right regarding the standard approach "Corona-Warn-App on a GMS Android Device",
this thread is about an approach where people compile everything from source. They could easily see what the software does with their privacy critical information.
I think it may be good to consider possible consequences of publishing open implementations of EN, normally I'm on board with fully open-source solutions, but it seems that in this particular case (distributed approach) it will enable easy "nerd attack":
The DP3T app collects as little information as possible by design, but operates
with an open protocol. This means that anyone can develop his own DP3T-like client, and
possibly decide to collect more data than what DP3T meant. There could be “enriched apps”
which collect more information for each encounter such as the geographic location, the exact
time, more information about the Bluetooth message. The app would make its best effort
to link changing EphIDi, which would not be too hard from Bluetooth metadata and signal
strength if there are not many neighbors in proximity and they are all static. The app could
further invite the user to enter more data such as if he knows the person, their gender,
approximate age, visible ethnicity, etc, or in which circumstance this encounter occurred (e.g.
in bus line x, in the elevator of building y). The enriched app could easily create a huge
database. With this, an isolated malicious user could start identifying many reported cases.
It's already easy to build an app on e.g. Raspberry PI zero W which collects RPIs together with exact timestamp and GPS location, and then matches these RPIs with published TEKs (which can be continuously fetched with simple curl cron job since it seems that there will be no safeguards like Device Attestation). Then adversary will be able to determine exact time and location of contact with infected person, which in turn may result in easy identification of this person. This capability could be limited to developers who can implement this from scratch, but with open implementation I guess this attack vector may be much more popular.
Keep in mind that users of official app are told that:
Die App ist so konzipiert, dass so wenig personenbezogene Daten wie möglich verarbeitet werden. Das bedeutet zum Beispiel, dass die App keine Daten erfasst, die es dem RKI oder anderen Nutzern ermöglichen, auf Ihre Identität, Ihren Gesundheitsstatus oder Ihren Standort zu schließen.
and right before turning on broadcasting of RPIs further assured that:
Die verschlüsselten Zufallscodes geben nur Auskunft über das Datum, die Dauer und die anhand der Signalstärke berechnete Entfernung zu Ihren Mitmenschen. Persönliche Daten wie Name, Adresse oder Aufenthaltsort werden zu keiner Zeit erfasst.
This is true if only official apps will be in use, but once we have enriched apps (developed based on open EN implementation) which provide fine-grained information about the encounters then above statement no longer holds.
I'm a big supporter on fully open-source / libre software and I think a solution where users don't have to create Google account in order to participate in contact tracing would be much better, but unfortunately current approach does not ensure full privacy by design and open implementation may enable more adversaries to violate privacy of others.
For those who don't want to use Google products one possible solution is to get cheap used Android phone from eBay, create fake Google account and carry it in another pocket (e.g. Motorola G2 which is capable of running EN costs 25-30 EUR).
If you decide to go forward with publishing EN implementation just make sure that it does not violate any licenses from Google / Apple (e.g. Beacon Simulator app developer had to remove iBeacon format implementation from public source code, despite the fact that iBeacon specification is open)
@kbobrowski One opposing view would be "Theoretically, it would be possible to follow a user, collect the user's RPIs, connect them with person-identifying data, and then check if the person is ever marked as infected. Yet, in practice, this attack vector to deanonymize a user requires a high amount of effort just to gain little additional information compared to the one already gathered while following the user." (quoted from here)
Also we should consider that the CoraLibre-android-sdk discussed above aims at replacing the GMS EN API, not the corona-warn-app. A "nerd attacker" could already today use either GMS EN, or other cheap hardware like you mentioned, and doesn't need to wait for this open/libre implementation of the API.
@mh- I don't fully agree with this quote from documentation, adversary just need to get a single RPI to determine if person who broadcast this RPI was within infectious period during encounter, there is no need to follow this person with "high amount of effort", couple of seconds in the same room is enough.
You are of course right that already today "nerd hacker" could use custom implementation of EN, but as long as there is no popular open EN implementation this attacker would at least need to know how to implement it from scratch. This is not black or white situation, I'm sure some people would be doing these "nerd attacks" anyway, but with easily accessible open implementations of EN the situation may have a bit darker shade of grey.
I just wonder about risk-reward ratio of developing alternative EN implementation as open-source, it's fine if someone who can do it does it for own responsible use, it's simple enough that it does not require community effort to develop. Developing it in open will enable more Google-free users to participate in contact tracing effort but at the same time will increase number of adversaries who otherwise would not have enough coding skills to perform "nerd attacks", potentially undermining trust in this system. Difficult to judge net outcome.
@kbobrowski "nerd attacker" could already today use the original Google EN implementation, once an official app is installed and activated, it will start collecting RPIs.
@mh- I don't exactly see how attacker could use official app for "nerd attacks", maybe by turning Bluetooth only next to the person whose infectious status attacker wants to determine but it would be very error-prone and require "high amount of effort" due to longer time required to stay in close contact. With custom EN implementation it's much more precise and require much less effort. Or do you mean reverse-engineering like using Frida to hijack some functions which interact with Google's EN? Or disabling signature check such that custom app can interact with Google's EN? But then again it requires higher level of skill which limits potential attackers.
Well, you are talking about nerds... Google makes efforts to access-control the RPIs that its EN implementation stores, not to encrypt them. Nerds will likely be able to override access control on their own device.
OK guys, thanks for your thoughts regarding nerd attack and Open Source philosophy. But please let us focus again on the tech, here. If you like you can create an issue in the documents repo and discuss this topic there. Here in the Android repo all discussions should be tech-only :slightly_smiling_face:
It looks to me that this is also related to https://gitlab.com/LineageOS/issues/android/-/issues/1949 and https://github.com/DP-3T/dp3t-app-android-ch/issues/59
Well, you are talking about nerds... Google makes efforts to access-control the RPIs that its EN implementation stores, not to encrypt them. Nerds will likely be able to override access control on their own device.
@mh- right, there is no obfuscation in GMS data. So unless Google is not encrypting / decrypting TEKs with some hard-coded / rolling key on the fly when sharing it with CWA and when matching with stored RPIs then it's easy to implement and validate EN framework. I have no doubt that nerds are able to access whatever on their phones and build EN, making implementation open just significantly lowers the bar ;) but perhaps it's inevitable anyway, and maybe "enriched" apps won't be a big issue after all.
Perhaps a good idea for open-source implementation would be to keep the same leveldb/protobuf structure as original one.
Speaking about storage. We might need help with securely storing the keys, as non of the current contributors know how to do this the proper way on android. Can/Should Android KeyStore be used for this?
Speaking about storage. We might need help with securely storing the keys, as non of the current contributors know how to do this the proper way on android. Kan/Should Android KeyStore be used for this?
You might want to have a look at the original Google implementation; at least in GMS 20.18.17 Google stores them in a LevelDB database in the /data/data/... section of the file system. They are access controlled, i.e. other applications cannot access this file system area. And if the user chooses File Encryption or Full Disk Encryption, they are also encrypted by that.
If you want to add your own encryption layer, you should carefully consider how this would really protect from which attacks; I think the only additional value that an Android Keystore could bring, would be to require a separate user approval for access - this could be used for the TEK storage, but not for the RPI storage, because you want the RPI matching to happen at night in the background.
Also the TEK storage should be accessible to the app without user interaction. You need to store the daily keys.
Also the TEK storage should be accessible to the app without user interaction. You need to store the daily keys.
@MalteJ I'm glad you asked :)
You can't store the TEKs themselves in an Android Keystore, because the Keystore will not return the raw key material to you, it will protect it from extraction (and you might need to upload the raw key material later).
So you would store the TEKs somewhere else (like a LevelDB) and encrypt them using a storage key. You can put this storage key into an Android Keystore. And you can do it twice: Put the storage key into a Keystore with the restriction that it can only be used for encryption, but always accessed (for the storage of a new TEK at 2am). And put it there a second time with the restriction that it can only be accessed for decryption if the user explicitly agrees (for the upload in case the user is infected).
@kbobrowski When looking at artefacts like Diagnosis Keys or Android Scan Records, this script might help: https://github.com/mh-/diagnosis-keys
@mh- thanks, nice job :) I'm running my own stack for reading leveldb, parsing export.bin etc., in C++ but nice to see similar stack in Python
@mh- thanks for your vectors, we tested them against our implementation and it seems to work :)
Also what do you think about using SQLcipher?
Seems to help progress with #490 too
See also discussion "Exposure Notification (Corona) #1057" in MicroG project.
The difference between DP-3T and Apple/Google framework is also discussed in DP-3T/documents#128.
All right so people keep asking me whats the current situation with our implementation.
We finished the crypto part, that part that handles the generation of the private keys and the keys that get send off. Also the comparison, meaning the part that checks weather ans exposed key was collected or not has been finished (not yet completely tested). We did tests with some test vectors from @mh- and it seems to work.
We are currently stuck with the development of the database, I hope we can finish this by the end of the weekend.
I've tried to integrate our crypto implementation into the bluetooth part here. However I found out that my knowlage about Bluetooth is very limited, and I have the fear that I do things the wrong way. I don't have the time right now to work myself into the Bluetooth stack, so we will need someone with bluetooth experience on android.
The API that later connects the app with our sdk is currently implemented by @haitrec.
Eventually the google contact tracking implementation needs to be striped form the app, so we can integrate our sdk.
EDIT: The algorithem that decides wather you got exposed or not is also not yet implemented, but it is described here in this podcast where @MalteJ and @tklingbeil, two of the official developer of the Corona-Warn-App where interviewed by @timpritlove.
All in all we do need more support. Weather its just about the code of the app itself, things like setting up a website, do cooperate identity things, documentation or comunity management.
If someone is willing to help feel free to open an issue or write me a mail on schabi (at) schabi (dot) org
Currently a lot of people are angry at the government because the warn app does not work with older Android versions. For example here: https://www.merkur.de/politik/coronavirus-warn-app-dorothee-baer-handy-smartphone-hartz-iv-aussage-kritik-richtigstellung-twitter-zr-13805451.html
"Mega Shit-Storm" it says.
This is because this app is just a front end for an API that was developed by Google. And Google of course wants to sell more phones, so they only implemented it for newer Android versions, not for the old ones. Also the dependency on Google API is a major point of critique: https://www.nachdenkseiten.de/?p=62113
The solution is quite simple: Implement your own API, then you can make it available for older phones as well.
Here, a single person now tries to fill this gap in his spare time. The motivation is different, he wants to get rid of Google API to make a true FOSS solution. But I see a lot of synergy here. A true FOSS solution of course also would allow to back-port the algorithm to older phones. There will be no limits any more, everything can be customized then. I am surprised that this has not been realized so far.
Telekom and SAP should team up with this project discussed here. This would be a win-win for all. SAP and Telekom can make their employer happy (the employer is the German government, and the government are the people) and FOSS will be strengthened.
@HerrBert233 older phones just don't have BLE (Bluetooth Low Energy), which is required for this whole concept to work, since standard Bluetooth consumes a lot more battery and requires pairing for exchange of information, something unfeasible for when you just meet other people around.
I do not believe it's a conspiracy by Google to sell more phone, and I don't think a FOSS alternative would possibly solve this intrinsic hardware issue (although I definitely root for one).
Seriously? Let me describe it with a quote from Telekom ( https://www.t-online.de/digital/id_88070154/corona-warn-app-darum-werden-aeltere-handys-nicht-unterstuetzt.html ):
Die hier benötigte Android-Version 6.0 wurde im Herbst 2015 vorgestellt. [...]
Damit dürfte es wohl einige Geräte aus 2014 geben, auf denen die Corona-Warn-App funktioniert. Viele günstigere Geräte aus 2015 und vermutlich selbst einige aus dem Jahr 2016 dürften hingegen nicht mehr geeignet sein.
Warum haben SAP und Telekom nicht an ältere Geräte gedacht?
Tatsächlich liegt die Ursache nicht bei SAP und der Telekom, die die App im Auftrag der Bundesregierung entwickelt haben. Sie sind letztlich auf die sogenannte Softwareschnittstelle angewiesen, die Google und Apple dafür bereitgestellt haben.
[...]
Die Corona-Warn-App an sich setzt keine leistungsfähige Hardware voraus und Bluetooth Low-Energy ist bei Smartphones vielfach auch schon länger Standard-Ausstattung.
To summarize: Bluetooth Low-Energy was introduced a long time ago, 2009 it was officially part of the standard, but Android 6.0 (minimum required) was introduced much more recently in 2015, and given the slow update cycle, even phone from 2016 may not have received that update.
So the Android version requirement is a far stronger issue than Bluetooth LE.
Secondly, what about the many phones without Google Play service? Just think of Fairphone. some use a custom Android without Google play service. Owners of Fairphone 3 are currently also locked from the Corona app. (not because of mussing Bluetooth LE)
Yes, a FOSS alternative would possibly solve those issues. Please get the facts right before downvoting me.
To give you also some examples of phones where Bluetooth 4.0 is present but Android 6.0 not, here is a list:
When a small company like Fairphone is able to update Snapdragon 801 phones to Android 9, why on earth are others not able to update their old phones to newer software if not to sell more phones? This was already irresponsible behaviour in context of sustainability but now it is risking people's health.
From an economic perspective, you sell more phones by _not_ updating your old phones already on the market but at the same time implement new features only for the most recent firmware. This is called planned obsolescence. You should not blame the companies, they just follow the path which maximizes profit. You should blame the system. If you think this is a bad behavior, then you should let your government pass anti planned obsolescence laws like France did. If we had such a law, then yes I would agree, then is no need to support older operating systems. But in Germany, politicians do not have the courage to implement such laws. So that's the situation we now have to deal with.
Instead, our politicians say things like people are "too lazy" to buy a new phone. (Dorothee Bär)
This cannot be. They cannot blame the citizens. I do not want to be called lazy just because I refuse to buy a new phone each year. This is a very arrogant statement I cannot believe how this can be tolerated.
The solution is clear and simple: Either back-port the algorithm to older phones, or pass laws that require all companies to update their phones. Both of those alternatives would be possible. But I think the first alternative is much more realistic and given that theScrabi already started the implementation, this could even be solved in a realistic time frame.
Here, a single person now tries to fill this gap in his spare time. The motivation is different, he wants to get rid of Google API to make a true FOSS solution. But I see a lot of synergy here. A true FOSS solution of course also would allow to back-port the algorithm to older phones. There will be no limits any more, everything can be customized then. I am surprised that this has not been realized so far.
Telekom and SAP should team up with this project discussed here. This would be a win-win for all. SAP and Telekom can make their employer happy (the employer is the German government, and the government are the people) and FOSS will be strengthened.
What I can't believe is, that the SAP earns 9.5 mio € for this App and is not able to support developers to build a Google API free version. This amount of money is roughly equal to 100 man years working capacity (with a german based salary). I can't see this amount of work in this repo (and dependency repos), so there should be a huge part of the cake available for supporting the Google API free FOSS development.
@qoheniac they are able to but don't want to. Selling new phones brings in
money, providing updates doesn't. Sounds stupid, is stupid but that's the
current state. In addition, some companies have a highly adapted Android
which complicates them updating their phones even further.
This update fragmentation is IMHO one of the biggest problems in Android.
It is in fact one of the reasons why the contact tracing API was released
as part of Google play services before adding it to the OS. Play services
can be updated solo, without requiring a system update (which, as you might
already guess, might never be released for a lot of phones). There have
been mitigations, e.g. Google requiring some years of guaranteed OS updates
to have the phone "Android certified", but unfortunately this does not fix
the problem completely.
On Sun, 21 Jun 2020, 11:28 qoheniac, notifications@github.com wrote:
When a small company like Fairphone is able to update Snapdragon 801
phones to Android 9
https://www.fairphone.com/en/2020/06/18/fairphone-2-gets-android-9/,
why on earth are others not able to update their old phones to newer
software if not to sell more phones? This was already irresponsible
behaviour in context of sustainability but now it is risking people's
health.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/corona-warn-app/cwa-app-android/issues/75#issuecomment-647103484,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAM6YFACYQ772HBGT7ETBVDRXXHFNANCNFSM4NPVHKHQ
.
You should not blame the companies […]. You should blame the system.
There is no contradiction. I can ask politicians to pass such a law and meanwhile blame companies for their behavior. There are lots of things I am legally allowed to do, but I don't because I think I shouldn't and companies have choice too.
To summarize: Bluetooth Low-Energy was introduced a long time ago, 2009 it was officially part of the standard, but Android 6.0 (minimum required) was introduced much more recently in 2015, and given the slow update cycle, even phone from 2016 may not have received that update.
So the Android version requirement is a far stronger issue than Bluetooth LE.
The first Android version that supported Bluetooth Low Energy was Android 4.3, released 24 July 2013. Phones earlier than that can't even be supported with a open source version, so the only thing we gain here is a minimal extension of period of support. Given the fact that Android 4.x phones were seldomly updated to 4.3 and marketshare of 4.3 was low, there is not much impact of back porting.
The example Galaxy S3 never officially got 4.3, because Samsung had problems with updating. Some later variants of it got 4.3, but not in Europe.
According to

this are still a few people. (It's worldwide so maybe not relevant for germany)
Let's not forget Android 4.4, Android 5.x
It is not true that only Android 4.3 is missing this feature. There is a 14.5% market share for Android 5.x
I'd first focus on supporting android 6.0+ here. I'd recommend opening a separate discussion about a "legacy" version. Adding legacy support to our main approach adds another layer of complexity which we should avoid in our first implementation. Our goal at the moment should be to at least be able to have full FLOSS version of the app.
I want to emphasize again, that SAP and Deutsche Telekom do not work on an open source implementation of the ENA protocol and are also not planning to do so.
We keep this issue open to give the community a platform to discuss about the technology. As you maybe have seen, I am personally also trying to promote this community project because I like the idea. But please stay focused. This is no platform for politics.
Thank you! 🙂
So its currently getting reeeealy slow, as I've started at a new job and have to prepare for my tests. In other words unless we are getting more help CoraLibre might take a long time or get to full halt soon. Especially Bluetooth, the exposure algorithm and the Database we will need help.
I found out that dp3-t uses the old bluetooth api (before version 26), which is apparently not compatible with the Google/Apple protocol as it can not be set precise enough. Therefore we need to rewrite the gatt part using API level 26.
At the moment I am trying to set the right AdvertisingSetParameters. Especially the tx power level is something I am concerned with. To how much db should I set it?
This value should depend on the phone hardware.
So its currently getting reeeealy slow, as I've started at a new job and have to prepare for my tests. In other words unless we are getting more help CoraLibre might take a long time or get to full halt soon. Especially Bluetooth, the exposure algorithm and the Database we will need help.
I found out that dp3-t uses the old bluetooth api (before version 26), which is apparently not compatible with the Google/Apple protocol as it can not be set precise enough. Therefore we need to rewrite the gatt part using API level 26.
At the moment I am trying to set the right AdvertisingSetParameters. Especially the tx power level is something I am concerned with. To how much db should I set it?
txPowerLevel int: Transmission power of Bluetooth LE Advertising, in dBm. The valid range is [-127, 1]
The next question would be: which TX Power Level should you report in the AEM? The same value, or something else, calibrated to the device? Depending on the detected situation (in pocket, in car, ...)? I think Google put some effort into this question, and I don't know the exact outcome.
All I can tell you is that my OUKITEL C16pro Android 9 device _reports_ -14dBm in the AEM that it sends out together with the RPIs. I don't know if Google Play Services also sets txPowerLevel to -14dBm.
I'm using TX_POWER_LOW for now. If it's good or not I don't know. Maybe we can find out more about this.
@theScrabi
Google has just published a new document:
https://developers.google.com/android/exposure-notifications/ble-attenuation-overview
Also have a look at the linked pages on the left navi:
https://developers.google.com/android/exposure-notifications/ble-attenuation-computation
and
https://developers.google.com/android/exposure-notifications/ble-attenuation-procedure
They also provide a calibration CSV:
https://developers.google.com/android/exposure-notifications/files/en-calibration-2020-06-13.csv
I'm using TX_POWER_LOW for now. If it's good or not I don't know. Maybe we can find out more about this.
Looks like the correct choice :)
from https://developers.google.com/android/exposure-notifications/ble-attenuation-computation:
"the Exposure Notifications framework uses the ADVERTISE_TX_POWER_LOW setting"
Nice thank you :) I'll take a look at it later.
Ah nice. So TL;DR (badly accumulated) about https://developers.google.com/android/exposure-notifications/ble-attenuation-overview
There is a value called attenuations. It is used as a proxy for the distance of two devices. It's calculated with the formula Attenuation = TX_power - (RSSI_measured + RSSI_correction) whereas TX_power is the transmission power and RSSI is the receiving power. In order to measure a persistent RSSI across different devices, each device needs to be calibrated. Thats no problem because google hands out a cvs file with calibrations for different devices, which apparently is constantly updated (this link is a snaptshot).
How to calculate/use the clibration: https://developers.google.com/android/exposure-notifications/ble-attenuation-computation
Rather less interesting for us I guess, unless we want to contribute devices to the calibration list: https://developers.google.com/android/exposure-notifications/ble-attenuation-procedure
Devices scan every 5min for 4 seconds.
If a capture is scanned multiple times, all captures are saved, whereas the minimum and average attenuation is stored (Note for me: Save all. Calculate average later).
If you scan a RPIs once, and scan the same RPIs (or RPIs that belong to the same TEK) five minutes later put all of these into a ExposureInformation. (Gets important for the risk calculation. keywoard: exposure duration).
Disclaimer: please correct me if assumed something wrong
If a capture is scanned multiple times, all captures are saved, whereas the minimum and average attenuation is stored (Note for me: Save all. Calculate average later).
That's correct, you can't calculate attenuation before you get a match with a TEK, as you need to decrypt the AEM to get the TX Power. So you have to save all RSSIs.
I'm using TX_POWER_LOW for now. If it's good or not I don't know. Maybe we can find out more about this.
Looks like the correct choice :)
from https://developers.google.com/android/exposure-notifications/ble-attenuation-computation:
"the Exposure Notifications framework uses the ADVERTISE_TX_POWER_LOW setting"
Aha, uff. So ADVERTISE_TX_POWER_LOW is used as a constant for AdvertiseSettings.
TX_POWER_LOW on the other hand is used for the AdvertisingSetParameters. I'm not even using AdvertiseSettings, so I may need to check this once more.
Here is the current state of my Ble integration: https://github.com/CoraLibre/CoraLibre-android-sdk/pull/14
It is by far not done yet, as you can see here, however you are welcome to take a look at, review or contribute to it.
Disclaimer: yes my code is not very good. This is why I'd really appreciate review.
@ttobsen "SAP earns 9.5 mio € for this App and is not able to support developers to build a Google API free version. This amount of money is roughly equal to 100 man years working capacity (with a german based salary)."
Wow, that's what I would call ONE big discount! If you would really offer 100 man years for just 9.5 million, your company would be a smash hit on the market. Or bankrupt VERY soon. No one can offer that much for 9.5 million. Or maybe you have never made calculations for a business case and simply "forgot" all extra costs like taxes, social insurance, rent, equipment, holidays of your work force etc. pp. et. al. ?
And last but not least, even if you forgot all this extra cost, you calculated from a raw salary of 100 K per year. Good luck trying to find SAP programmers for less than 350 K per year, seniors much higher...
@hman2 Actually, it is considered way too expensive, see this: Corona Warn-App: Ziemlich teuer. It's an open secret that the Corona warn app is just a bailout for German IT companies. The whole point why every European country is doing its own app (instead of an European joint development) is that now governments can spend 27 times the money instead of only once. They just want to have a reason to give taxpayers money to the industry. That's it. If you understood this, then I think it is absolutely legit to ask something in return for it. And I think supporting an open source exposure API would be a great way to spend all the excessive tax payers cash.
Especially since it is a long term goal of many German strategists to become independent from US IT companies. An alternative open source exposure API working on LineageOS or similar w/o Google services would very well fit into this strategy. To summarize:
1) SAP got way too much cash from German government
2) It is a very important economic, political and social goal to become independent from US IT companies
Now we just need someone with a great vision. A manager who does not just look at monthly cash flows but at the long term well being of the company. Then it is absolutely clear that SAP and Telekom must support theScrabi and his colleges to create something independent from the big US IT companies.
SAP doesn't need any bailout! SAP is not dependent on consumers. SAP is the world's third biggest IT company and NO large and very large business worldwide runs without it. Just look at the balance of SAP.
I work in a very large computing center in Germany as a quality manager. 20 million is an absolutely reasonable price for an app that has to be tested on hundreds of devices. Most people forget that testing costs as much as developing software. If you do right, of course.
Do not forget that measuring distances with Bluetooth is nothing that Bluetooth was built for and results are highly dependant on antenna gain, emission characteristics, casing etc. pp. et. al. No one has done it before, so it had to be tested deeply, because the least the government would want are high rates of false negative or false positive results. For testing the App they called for the armed forces of Germany...
Ok, maybe "bailout" is a bit too strong translation of "Industrieförderung". But I think you get the idea.
Do not forget that measuring distances with Bluetooth is nothing that Bluetooth was built for and results are highly dependant on antenna gain, emission characteristics, casing etc. pp. et. al. No one has done it before, so it had to be tested deeply, because the least the government would want are high rates of false negative or false positive results. For testing the App they called for the armed forces of Germany...
Well, but all this was _not_ part of the app. This has all been developed by Google. The Corona app developed here is just a front end. A graphical user interface for some Google service. The app "receives" exposure notification events from the Google API (which does all the complex, underlying mechanics), then it writes this event into a database, besides that it provides nice graphics to the user. That's basically it.
So if your assumption was that all of the underlying mechanics have been developed by SAP and Telekom and you think "20 million is an absolutely reasonable price" then yes, I agree. But the point is: They charged full development price for just developing the hull. That's what I criticize and why I think it is fair to ask supporting the development of a free open source exposure API.
Again, please focus on the technology!
Otherwise we will close and lock this issue.
@HerrBert233 please allow @theScrabi et al to leverage this issue to find coworkers for their project!
Thank you!
@HerrBert233
Do not forget that measuring distances with Bluetooth is nothing that Bluetooth was built for and results are highly dependant on antenna gain, emission characteristics, casing etc. pp. et. al. No one has done it before, so it had to be tested deeply, because the least the government would want are high rates of false negative or false positive results. For testing the App they called for the armed forces of Germany...
Well, but all this was _not_ part of the app. This has all been developed by Google. The Corona app developed here is just a front end. A graphical user interface for some Google service. The app "receives" exposure notification events from the Google API (which does all the complex, underlying mechanics), then it writes this event into a database, besides that it provides nice graphics to the user. That's basically it.
So if your assumption was that all of the underlying mechanics have been developed by SAP and Telekom and you think "20 million is an absolutely reasonable price" then yes, I agree. But the point is: They charged full development price for just developing the hull. That's what I criticize and why I think it is fair to ask supporting the development of a free open source exposure API.
I know this is off topic to this issue but in the official press meeting a SAP employee said they helped Google and Apple to improve the distance measuring a lot so I assume we don't really know what's included in the price. I also asked the finance ministry about that yesterday so let's see if I get a response. I also think there is a more appropriate issue in this repository regarding the cost of the project.
Again, please focus on the technology!
Otherwise we will close and lock this issue.
@HerrBert233 please allow @theScrabi et al to leverage this issue to find coworkers for their project!Thank you!
Do you have any advice on how and where we could get more workforce to work on this project? Who could we try to communicate with?
Thanks
Do you have any advice on how and where we could get more workforce to work on this project? Who could we try to communicate with?
Maybe this discussion can be splitted and this topic could be moved to another thread?
How about https://GNU.org, https://www.linuxfoundation.org/, https://www.dyne.org/ (devuan), https://digitalcourage.de/, https://fsfe.org/, https://www.gatesfoundation.org/, Huawei (I mean it! Since they need their own software universe and Google is not allowed to play with them) ? Maybe they have some workforce or money?
Huawei made their own solution already.
Huawei made their own solution already.
The interface is similar but the corona apps need to be modified:
Dafür ist allerdings zusätzlicher Aufwand nötig: Die Anwendungen müssten auf die Huawei-Schnittstelle und die Veröffentlichung in Huaweis eigenem App Store angepasst werden.
https://www.heise.de/news/Huawei-entwickelt-eigene-Schnittstelle-fuer-Corona-Warn-Apps-4779422.html
MalteJ: Am I right, that the last statement is something like: "the CWA will be only published for Google and Apple. For other systems the work has to be done by other", which is possible because the CWA is open source?
Again, please focus on the technology!
Otherwise we will close and lock this issue.
@HerrBert233 please allow @theScrabi et al to leverage this issue to find coworkers for their project!
Thank you!Do you have any advice on how and where we could get more workforce to work on this project? Who could we try to communicate with?
Thanks
I'm currently trying to make more publicity for this project. I think the more people know about this the more developer will join.
Do you have any advice on how and where we could get more workforce to work on this project? Who could we try to communicate with?
Maybe this discussion can be splitted and this topic could be moved to another thread?
How about https://GNU.org, https://www.linuxfoundation.org/, https://www.dyne.org/ (devuan), https://digitalcourage.de/, https://fsfe.org/, https://www.gatesfoundation.org/, Huawei (I mean it! Since they need their own software universe and Google is not allowed to play with them) ? Maybe they have some workforce or money?
Maybe no workforce but as I said in my previous commend it might spread awareness of this project.
FYI about the exposure risk algorithm: https://developer.apple.com/documentation/exposurenotification/enexposureconfiguration
@theScrabi If you want to test if your custom implementation of EN is complying with original one, then this project may be useful: https://github.com/kbobrowski/en-i13n
It allows anyone to become whitelisted and use EN framework with custom app. Developed it mainly for debugging current issues with Android, but may be also useful for validating open-source implementations of EN
Hei we have some things we might need some support with. @Stypox implemented the exposure risk calculation algoritm: https://github.com/CoraLibre/CoraLibre-android-sdk/pull/24
He has a question regarding the calculation of the transmissionRiskLevel which we can't answer right now. @MalteJ maybe you have an idea about his question.
Also we currently face the task of testing this algorithm. We don't have access to the official testing vectors, so we thought we might test it by sending different Tek and RPis into the original CWA app and our algorithm, and check if the output is equal. I thought the way we could do this was to use the xposed framework to hock the function that calls the calculation of the exposure risk. However we can't wait for someone to get corona to be able to test it, so we will also have to inject fake TeK/RPi pairs into the app.
What do you think, is there a simpler solution? @MalteJ could you guys test our algorithm for us instead? Would someone want to help us with the testing framework?
use the xposed framework
No please not. Xposed is a hugh security desaster and increase the attack surface a lot.
No please not. Xposed is a hugh security desaster and increase the attack surface a lot.
For testing!? I think this is exactly why Xposed was developed in the first place...
@theScrabi testing your algorithm would also be possible with xposed, but en-i13n solves exactly this problem so it may save you some time. Basically using this project you can:
provideDiagnosisKeys - you would be able to call it practically unlimited times a day instead of just 20Basically it provides you the same capability as if you were SAP developer with access to official signing keys and whitelisted development account. If you decide to use it and need some help setting this up - let me know
@theScrabi, in case it might be useful:
We've recently released a collection of snippets to provide additional transparency in how the Exposure Notifications System works.
_Originally posted by @nic0lette in https://github.com/google/exposure-notifications-android/issues/21#issuecomment-661471479_
Also we currently face the task of testing this algorithm. We don't have access to the official testing vectors ...
First, thank you all for your marvellous efforts. As a doctor working in a GP practice at the front line of the Covid19 crisis with public health training, I very much appreciate that you are trying to make the warning app available for the google free user community, which I am part of.
I want to make a point regarding compatibility and ask professional SAP and Telekom developers to offer the following support.
As far as I understand, the different European Corona apps are not fully intercompatible yet (please correct me if I am wrong). If correct, this foreseeably contributes to the second corona wave currently under way, as holiday makers meet and don't get warnings. The same problem may occur if a free corona app is incompatible with current solutions.
Could professional developers help ensuring intercompatibility?
If this should be difficult or impossible due to Google's control overt the API, I strongly encourage European app makers to find a European solution with a European API. It is, in my opinion, prohibitive to force people that want to protect themselves against Covid19 into so-called privacy agreements of US-based Google or Apple. Even more, since the United States show increasingly worrying authoritarian tendencies and openly discuss whether a smooth transition of power will be possible after a likely defeat of Trump in the autumn elections.
@theflow85 cross-country interoperability is discussed in https://github.com/corona-warn-app/cwa-backlog/issues/14. An official comment about the current state of affairs can be found in theaforementioned issue: https://github.com/corona-warn-app/cwa-backlog/issues/14#issuecomment-649024930
Some ideas for the man/womenpower - question that came up earlier: Consider trying to team up with e.g. Fairphone or Shiftphone developers, or with developers that work on other European apps that have less google dependencies (I think the Spanish app uses another protocol?). I recently saw an overview which country works on which kind of Corona warning apps, not sure where but I can look it up if it helps.
Please let me know whether these inputs were helpful or not.
I just added initial support for Exposure Notification API to @microg, written in Kotlin, see org.microg.gms.nearby.exposurenotification. I'll try to make it a mostly independent module so that it can be used without microG.
I also plan to provide the client library so that it is theoretically possible to make a fully open source version of CWA and other apps that use EN API. This would allow publishing CWA in F-Droid, however, without official adoption of that replacement client library, signature checks are going to fail.
@mar-v-in I have a superficial understanding of this: if signature checks will fail, does that mean theoretically one could still receive exposure notifications, but not upload their own exposure list as a positive case?
@ljl-covid no, the signature check (which btw is not present or planned for microG because I don't see a reason for it) prohibits all usages of the API for apps+signature not on a list maintained by Google.
@mar-v-in does it make sense for us to continue with coralibre with our own sdk, or should we then switch over to your solution?
I don't think it's a bad idea to have multiple open-source implementations. This can make testing and verifying implementations a lot easier.
microG implementation is currently not embeddable into apps and thus the implementation is only useful for users that run a microG compatible system (which is only a very small number).
The client library isn't a priority for me right now.
So goals/focus of the two projects is not exactly the same. We surely can help each other one way or another, even if it's just by copying code snippets. For example microG still needs proper attenuation calculation/correction (using calibration data), not sure if you got that already in coralibre.
The goal of CoraLibre is not only to crate a libre version of the CWA app but also a libre sdk which can be embeded into multiple solutions. Maybe this sdk may also be useful for MicroG.
BTW this issue here overlaps with my newly created one https://github.com/corona-warn-app/cwa-app-android/issues/1483. https://github.com/corona-warn-app/cwa-app-android/issues/1483 however does not aim for a fork/complete reimplementation as CoraLibre, but tries to get this upstream app into F-Droid, as actually it can work with _microG_.
Please have a look at the other issue for more information.
I know some people may prefer no dependency on microG, but that's what we maybe have CoraLibre in the future. For now, it would already be a huge step forward to have this app in an F-Droid repository or even the official one, if this is possible. And, in any case, reproducible builds would of course be awesome.
Just as a heads-up: the fork from https://codeberg.org/corona-contact-tracing-germany/cwa-android seems to work fine, and will hit the F-Droid repo this week (hopefully) :smiley:
Most helpful comment
Then in the foreseeable future, it will not be possible for everyone to use the app. I'm using a Fairphone 2 with their Google free Fairphone Open OS, no GSM available.
So the question is: Why is it necessary to use the Google APIs for this app? Are there any technical reasons?
I think the most important feature for such an app is, to make it available to the biggest user spectrum possible. But when using Google APIs this target can't be reached. :-(