Covid_19: BS scraper adds old source url

[jb3-2]

The BS Scraper seems to add an old source url instead of using the current url, see e.g. here, were the scraper uses the url dating back from 27.3.2020 instead of the url of today (30.3.2020).

It's not such a big problem since I go and enhance the data everyday anyway, thereby replacing the old url with the correct one.

[metaodi]

The source is currently hardcoded, I'll look into getting the actual URL from the scraper.

[baryluk]

BS scraper should be extracting the latest URL and latest data correctly.

It gives me now: https://www.gd.bs.ch//nm/2020-tagesbulletin-coronavirus-621-bestaetigte-faelle-im-kanton-basel-stadt-gd.html , which is the correct URL. Not sure why CVS got other URL.

[baryluk]

Oh. I see @metaodi . I see. It is not a problem with scraper, but with the workflow.

In add_db_entry.py we should do similar to what we do in convert_parsed_to_csv.py, and not use SCRAPE_SOURCE from environment.

[metaodi]

@baryluk exactly 😉 I'm on it. I anyway touch this code currently to support custom schemas (i.e. cantons with extra fields in their CSV).

[andreasamsler]

Same issue for BL. Thanks.

[baryluk]

@metaodi Great work Stefan! Thanks.