Covid_19: field naming -- cumulative vs non-cumulative

[rneher]

Thanks a lot for scraping these togehter!

In the README, you explain columns as

| Field Name | Description | Format |
|---------------------|--------------------------------------------|------------|
| date | Date of notification | YYYY-MM-DD |
| time | Time of notification | HH:MM |
| abbreviation_canton_and_fl | Abbreviation of the reporting canton | Text |
| ncumul_tested | Tests performed (cumulative) | Number |
| ncumul_conf | Number of confirmed cases | Number |
| ncumul_hosp | Number of hospitalised patients | Number |
| ncumul_ICU | Number of hospitalised patients in ICUs | Number |
| ncumul_vent | Number of patients requiring ventilation | Number |
| ncumul_released | Number of patients released from hospitals | Number |
| ncumul_deceased | Number of deceased | Number |
| source | Source of the information | href |

most field names suggest the data are cumulative, but only one (tested) points this out explicitly. I assume fields with cases, deaths, and released are cumulative, while the remaining ones are not. please clarify

[tlorusso]

@rneher to rename the columns at this point would break a lot of stuff. However, we could be more specific in the description field and distinguish between 'current' and 'cumulative' for all. Would this help? However, a problem that might pop up at some point is different reporting policies among cantons.

[metaodi]

I think we should change the description of all fields beginning with ncumul_ to have "(cumulative)" in the description, just like we have for ncumul_tested.

WDYT @tlorusso ?

[tlorusso]

I would propose adding _(cumulative)_ to the description field of the confirmed cases, deaths, and released, while adding _(current)_ to the field of the hospitalized / ICU and ventilated columns. For the later it might have been better to not include the prefix ncumul_ in the column names, as they reflect the current numbers without those that have already left the hospitals. @rneher @metaodi

[rneher]

I support not changing the columns. but making it clear in the table would help (plus a note that headers are named sub-optimally but are kept for consistency).

[tlorusso]

@rneher ok like this?

| Field Name | Description | Format |
|---------------------|--------------------------------------------|------------|
| date | Date of notification | YYYY-MM-DD |
| time | Time of notification | HH:MM |
| abbreviation_canton_and_fl | Abbreviation of the reporting canton | Text |
| ncumul_tested | Tests performed _(cumulative)_ | Number |
| ncumul_conf | Number of confirmed cases _(cumulative)_ | Number |
| ncumul_hosp* | Number of hospitalised patients | Number |
| ncumul_ICU* | Number of hospitalised patients in ICUs | Number |
| ncumul_vent* | Number of patients requiring ventilation | Number |
| ncumul_released | Number of patients released from hospitals _(cumulative)_ | Number |
| ncumul_deceased | Number of deceased _(cumulative)_ | Number |
| source | Source of the information | href |

*These variables reflect current, not cumulative numbers, even if the prefix in the column-name might suggest otherwise. Column names will not be changed to grant stability.

[SarahNadeau]

I second the request to clarify the descriptions! I also included suggestions from our group on changing field names, but I understand this may not be feasible. If possible, I would suggest to be even more verbose with the descriptions:

Field Name | Description | Format
-- | -- | --
date | Date of notification | YYYY-MM-DD
time | Time of notification | HH:MM
abbreviation_canton_and_fl | Abbreviation of the reporting canton | Text
ncumul_tested | Reported number of tests performed as of date | Number
ncumul_conf | Reported number of confirmed cases as of date | Number
~ncumul_hosp~ ninstant_hosp | Reported number of hospitalised patients on date | Number
~ncumul_ICU~ ninstant_ICU | Reported number of hospitalised patients in ICUs on date | Number
~ncumul_vent~ ninstant_vent | Reported number of patients requiring ventilation on date | Number
~ncumul_released~ ncumul_recovered | Reported number of patients released from hospitals or reported recovered as of date | Number
ncumul_deceased | Reported number of deceased as of date | Number
source | Source of the information | href

[tlorusso]

@SarahNadeau Thanks! Looks good & makes sense, we should implement your version of the column descriptions in the Readme. I've a question regarding ncumul_vent, appearingly for some cantons, information about intubated patients has been stored in this column for now. How important is the distinction between ventilated / intubated from your point of view? If we want to separate these two things we have to discuss how to do this in a safe manner, without breaking to much stuff.

[SarahNadeau]

@tlorusso I was under the impression that we were not lumping intubation and ventilation data. So where intubation data is available, we will break this into a separate column (e.g. "ncumul_ICU_intub" for GE). However, I only joined as of this week and do not know how this was done before; @andreasamsler would have the real answer. Either way, the documentation of these new columns is missing.

We are not using the ventilated/intubated data at the moment so I can't speak to how they will be used (maybe to allocate supplies?). That said, I personally feel it is very important to separate anything that is feasibly separable. Users can lump these columns later if they wish.

[tlorusso]

@SarahNadeau as just few cantons report this numbers so far, it should not be much work to identify where this has happend. I just spotted that GE contains the same figures in the intub / vent columns, although it obviously shouldn't.

https://github.com/openZH/covid_19/blob/master/fallzahlen_kanton_total_csv/COVID19_Fallzahlen_Kanton_GE_total.csv

https://www.ge.ch/document/covid-19-situation-epidemiologique-geneve/telecharger

To ensure this is limited to GE we might have to check those Cantons that have data in the 'ncumul_vent' and check in the source whether it really reflects the number of ventilated. If not we can correct this and extend the dataset with a 'ncumul_ICU_intub' column (as GE already has) after the source column and also provide a definition in the Readme-table. I'm off this afternoon, but could contribute to this tomorrow.

[SarahNadeau]

@tlorusso good catch that we had incorrectly duplicated intub data in the vent column for GE. I'll flag this for the 2pm update person to correct. I will not add any columns myself (as discussed with Andreas) but if you add the column we will be happy to check it/ update it!

[metaodi]

@tlorusso are you creating a PR for this?

[tlorusso]

Either later this evening or tomorrow morning. @metaodi

[metaodi]

Fixed in #243