Electricitymap-contrib: Standard api for data providers

[magol]

This is probably a crazy idea, but still want to share it with me

It seems that all regions have their own api/file format on their public data. It tells me that there are no standards for how to provide this type of data.

My idea is that we create a easy "standard" in the absence of anything else. I think that it make it easyer for data providers to publish data if they have a api specification to follow.

[systemcatch]

It's a good idea and there are already standards (insert relevant xkcd) that are easy to follow. Maybe you are talking about the information returned rather than the format it's in though?

[magol]

Ok. If there are standards already, then my suggestion was based on incorrect assumptions.

I have not implemented any parses myself, but I got the impression from discussions here that it is sometimes a challenge to get out and interpret information.

However, it may also be useful to specifikt WHAT information the data providers should provide rather then HOW.

[systemcatch]

You're correct in that some data is much harder to get than others. Generally I'd divide the parsers into 2 categories; parsers that use API's (e.g. US-MISO, ENTSOE, DK-BHM) and parsers that scrape data from html/files/images/backend services (e.g. AR, MD, NA, TW).

The scrapers are more likely to have bugs and break on changes. API's are usually provided as services which are fairly consistent and easy to work with.

[magol]

Ok, now I understans. We can probably close this issue?

[brandongalbraith]

@magol Perhaps we ask scraped data providers for an API as part of this issue?

[systemcatch]

@brandongalbraith that's not a bad idea, we could use this as a general thread for improving current data sources.

As a starter the GB-NIR parser would be a good candidate for an api.

[systemcatch]

@brandongalbraith do you want to make a request to them?

[brandongalbraith]

@systemcatch Yep! Happy to! Will make the request and come back with an update.

[strassburger]

Just stumble over an app of king island.
Somebody more competent please look if it may help us:
http://www.kingislandrenewableenergy.com.au/app/about

[corradio]

@brandongalbraith any update on your request?

[corradio]

@magol @brandongalbraith I'm closing this issue due to inactivity. As much as I love the idea, I'm not sure we can do much about it right now.

[brandongalbraith]

@corradio Good call on closing this. After having spent a few hours digging and speaking to an ISO operator, it turns out that this problem is fairly large and difficult to tackle in the scope of a Github Issue. I'd be interested in hearing what your team's idea of an ideal API for consumption would be, which would help me mount a standardization effort (which might be in vain but would be fun to attempt). If/when you've got the time, feel free to get in touch, my email address is in my GH bio.

cc @magol