Cwa-app-android: [Discussion] Key-count

[DarkBlue-Night]

I have a question to version 1.6
I started a history with all transmitted keys for each version every day. But I did not get any keys for 3th to 6th December for Version 1.6 . It would be very helpfull if this number of keys could be posted here.
On 2th of December Key-Number for Version 1.6 was: 276750. Number of Keys on 7th was 278094.
Hope someone can help me and post the missing number of transmitted keys.

[MikeMcC399]

@DarkBlue-Night
There's a lot of detailed history on https://ctt.pfstr.de/.

[DarkBlue-Night]

@MikeMcC399
As you wrote - "lots of" .... I have no idea, how to find the numbers I've been looking for in all this data. I can't even see where to look for the Version 1.6 in special. I started to look for the number I have for 2th December - to have an option to find the equivalent number for the other days. But the number for 2th December is not in the sheet I expected it.

[heinezen]

@DarkBlue-Night Scroll down and you should see a list of dates. E.g. you can click on plaintext for any data and it should show you the number of keys in the first lines of the text file.

Here the data for 2020-12-03 for example:

Exposure Notification Diagnosis Key Parser
This script parses published Diagnosis Keys.

File 'page/keys/2020-12-03.zip' read.
- Time window: 2020-12-03 00:00:00 UTC - 2020-12-04 00:00:00 UTC
- Region: DE
- Batch: 1 of 1
- # of keys in batch: 24186
- Signature Info:
verification_key_version: "v1"
verification_key_id: "262"
signature_algorithm: "1.2.840.10045.4.3.2"

As you can see, there are 24,186 keys in this batch.

---

Corona-Warn-App Open Source Team

[MikeMcC399]

@DarkBlue-Night
I hope you know that CWA 1.6 is not the current version, and it wasn't the current version on Dec 3 - 6 either, because CWA Android 1.7.1 was rolled out on Nov 27, 2020.

If the hints from @heinezen don't help, perhaps you can explain what you intend to do with the data?

[DarkBlue-Night]

@heinezen
24186 - is the number of keys which were submitted on 4th December - the next day of course - in Version 1.3 as a new daily "key-package". I have this number in my Version-1.3-Excel-Sheet. The link says "- Region: DE" - this is Version 1.3
In Version 1.6 you only get ONE key-package for one day. And this number is the one I'm looking for.
But I'm very thankful for your help !!

[ndegendogo]

@DarkBlue-Night I can give you my numbers, but they seem to be different from yours ... so not sure if this is helpful ...

I assume you want these numbers just for statistical reasons? Because for your "productive" device that you carry with you, I strongly recommend that you follow the advice of @MikeMcC399 and upgrade to the newest available version.

• cwa 1.7 supports hourly files and is able to perform several checks per day, which results in a faster turn-around time from upload of positive keys to a warning.
• cwa 1.9 has an improved risk calculation formula based on Google's ENS version 1.5 API.

This said, here are my numbers as I see them in my log file (and I guess these are European numbers, so not only Germany).
I give you a few days overlap with your numbers so you can estimate how they align with yours.

27 Nov 20078
28 Nov 18498
29 Nov 9212
30 Nov 12667
01 Dec 28111
02 Dec 26910
03 Dec 24931
04 Dec 25309
05 Dec 19753
06 Dec 10656
07 Dec 14913
08 Dec 31940
09 Dec 35396
10 Dec 33271

[ndegendogo]

@DarkBlue-Night I just noticed that your numbers are ~ factor 10 larger than mine.
So I guess you want accumulated number of keys of the last 14 days.

Till version 1.6, a single check per day was performed, and it covered the key files of 14 days into the past.
So try to apply this sum formula to the single-day numbers at https://ctt.pfstr.de/ and see if you get a match.
If the numbers still don't match, the reason might be All-European numbers vs Germany-only. I am not sure which of these numbers is shown at the given link.

Also, for your statistic project, you can consider to retrieve the numbers directly from the endpoint (but of course ony 14 days into the past). You know then exactly what you get (daily vs hourly files, German vs European).

[DarkBlue-Night]

@ndegendogo
"statistical reasons" - exactly !
I have a 1.6-sheet and a 1.3-sheet. I startet in August to document all transferd keys and of course - on my smartphone, I actualy use for contact-tracking, I run CWA 1.9 ;).
I will check the numbers you gave me - on the first look, they seem perfect for me and match with the numbers I have calculated until 3th December by myself.
So thanks a lot !!!!!!
I was just giving up on my 1.6-Sheet and so I missed some later numbers as well :(.
May I ask you, where from do you have this numbers you wrote ??? So I do not have to ask again for missing numbers ...
They are not from the link https://ctt.pfstr.de/ - there are the 1.3-Numbers I have in my sheet, but I can't find there the numbers you wrote.
And what do you mean with "from the endpoint" ? Maybe this is a stupid question, but I'm commpletely new in this forum.

[ndegendogo]

@DarkBlue-Night

of course - on my smartphone, I actualy use for contact-tracking, I run CWA 1.9 ;).

😀👍

where from do you have this numbers

Well - I have iPhone, and so my ENF log looks different from Android, and it shows more details; even with 1.9 it shows numbers per key file, whereas Android shows only a single aggregated number
(if you perform the check between 00:00 UTC and 01:00 UTC, your 1.7 / 1.9 number should be aggregating over the past 14 days without new hourly files, compatible with your 1.6 numbers)

I have a 1.6-sheet and a 1.3-sheet

As far as I understand, the differences of these numbers are German numbers (1.3) vs European numbers (1.6). I have not checked myself, but was not aware that the difference is so small ... so this seems to tell us that only few keys are coming from the other countries? (I'll open a separate ticket with this question).

what do you mean with "from the endpoint"

As far as I know, the key files can be pulled via a REST interface. No password or special authorization required. And this is exactly what several statistics projects like Jan and Michael are doing. So all you need to know is the URI then you can pull the hourly files or the current daily files (Europe) or the "version-1.3" daily files (Germany).
Unfortunately I don't know the URI at the moment, it is documented somewhere - if I find it again I can post it here ...

so I missed some later numbers as well

no problem, I can fill in for your missing numbers - only don't wait too long before asking, because old ENF log entries are deleted after 14 days

Have fun!

[ndegendogo]

@DarkBlue-Night see this discussion

but the URI should also be documented somewhere here in the cwa project

[DarkBlue-Night]

@ndegendogo
I appreciate your help so much !!! You really made my day !
The numbers you posted are exactly the one, I was looking for !!!
They fit exactly in my caluculation.
I'm always one day "behind" your numbers.
So I have calculated:
20078 keys for the 28 Nov - because I got 281430 new keys in Version 1.6 that day - I did not get 20078 keys - but I can calculate the "new" keys as the difference between the keys I get in version 1.6 and the keys from the last 13 days (calculated key-package from the 14th day is dismissed). The difference must be the "new" number of keys for that day. And your numbers are exactly the ones I calculated until 3th December.
But I missed numbers again - sometimes I got 0 keys transmitted. I don't know why :(
The last number I calculated is 31800 for the 17th of December.
When everything is as I think - you should have this number in your log for the 16th December.
Than I missed two days. And without this two days, the whole sheet breaks apart. Because I only calculate this numbers - without actually "knowing" them or see them somewhere.
If you can post the next two or threee numbers - would be great.
And if you find the URI - would be great as well. I will look into the posts from Jan and Michael. There are really a LOT OF NUMBERS ... Maybe I can find the system in this numbers.

(if you perform the check between 00:00 UTC and 01:00 UTC, your 1.7 / 1.9 number should be aggregating over the past 14 days without new hourly files, compatible with your 1.6 numbers)

This is exactly what I though !!! That there should be a match in the daily-key-numbers, which are transfered in Version 1.6 and the higher versions - once a day. Thanks a lot for this unquestioned answer !

--

[ndegendogo]

@DarkBlue-Night See here section "Service APIs / Distribution Service" for description of the endpoint

[ndegendogo]

@DarkBlue-Night

You really made my day

😀 thanks - glad to be helpful

sometimes I got 0 keys transmitted. I don't know why :(

ooops - sounds not good. Does this happen on your 'productive' device with cwa 1.9.1?
If yes: consider to open a ticket for this.

And for your missing numbers: here are the last days, again with a bit overlap, so you can better synchronize:

13 Dec 13635
14 Dec 17302
15 Dec 31800
16 Dec 37643
17 Dec 55009
18 Dec 47796
19 Dec 42050

[DarkBlue-Night]

@ndegendogo

thanks a lot for your numbers - helps me a lot !! These numbers show me as well, that my kind of "calucate" is correct.

and yes - the "0 keys submitted" happens in 1.9 version.

today- yesterday -
started a couples of days ago - actually with the upgrade to 1.6 - where my inital problem started with the missing numbers from 3-6 December.
Since that - I have always two smartphone with me - one with the reliable 1.3 version - and one with the "today"-version, which is not so reliable for me.

[ndegendogo]

yes - the "0 keys submitted" happens in 1.9 version

@DarkBlue-Night please open a new ticket for this issue so the developers can look deeper into this. Please mention also that your other device works more reliably, and provide all technical details (device type, Android version, cwa version, ...). And if possible also provide the exported ENS log files of both devices

Do you get any error messages when it fails? Or does it just fail silently?

[heinezen]

Hey @DarkBlue-Night ,

Do you consider your original question solved? If so, we would close this issue.

But I missed numbers again - sometimes I got 0 keys transmitted. I don't know why :(

Regarding this: Would be good if you could make a separate bug report with more information for this in the issue tracker. If you have a Samsung device, it could be the same issue as https://github.com/corona-warn-app/cwa-app-android/issues/1817 .

Thank you

---

Corona-Warn-App Open Source Team

[DarkBlue-Night]

My "orginal" problem is solved - I got the numberes for the special days I was looking for.
But I still hope to get the information, where I can find 1.6-and higher Version "submitted keys" here on github.
I'm looking for something like that:
https://github.com/micb25/dka/blob/master/plots_en/plot_num_keys_submitted_last_6_weeks.png
These are 1.3-Version Numbers - and there should be an equiv for the others version somewhere here.
And is there a possibility to write a private message to someone if I have missing keys again? I cant find something like that.
But yes, you can close this issue.

[ndegendogo]

is there a possibility to write a private message to someone if I have missing keys again

@DarkBlue-Night feel free to ping me again if you need these numbers.

I don't know if github supports something like private message - but, you could misuse this ticket even after it is closed, I will keep it watching ... and if I understand correctly, the cwa maintenance team has setup their robots / notifications to skip closed tickets - so hopefully you won't disturb them too much ...

Another question: have you ever tried programming yourself? Or considered to try it?
I am currently trying if I succeed to capture the key files directly from the endpoint. If you are interested we could exchange experience and information.

[dsarkar]

@ndegendogo, @DarkBlue-Night, no problem at all if you wish to continue in this thread. We can keep it open. DS