Obspy: Dynamic URL in fdsn.client

[jschaeff]

In obspy 1.2.2, le FDSN clients URL of services are hardcoded, while they could be fetched from the FDSN datacenter webservice API.

Documentation: https://www.fdsn.org/datacenters/help/#web-service-api

[ThomasLecocq]

Hi, thanks for this, indeed it could be a good way to fetch the datacenters! Do you feel confortable submitting a PR ?

[jschaeff]

Yes.
One way would be to add an option to the object so that on instanciation it tries to guess the services URL. On failure, it falls back to the hardcoded one. Is this acceptable ?
Other strategy, if guessing fails, raise an exception. The user has to specify the services himself. Seems less user friendly but avoids hardcoding the services.

[ThomasLecocq]

I'm in favor of a obspy.clients.fdsn.client.discover_webservices or some helper function. Ideally, I'd suppose this func should store a dict somewhere, which would allow for not re-running this every time. @chad-iris maybe has an idea?

[chad-iris]

Fetching web services endpoints from the FDSN's Data Center Registry is a great idea.

Not sure what the helper function idea implies. I think it'd be nice if it were automatic as in a user can

1. fetch a list of data centers _and_ their advertised data sets (which is also included in the Registry); and
2. instantiate a client for a given data center name

which first uses the Registry information and then falls back to some known options if the registry lookup fails. There is always a manual config of endpoints needed of course.

I recommend an approach that caches the results, as the data do not change often (and the service is not particularly implemented for heavy usage).

[filefolder]

If I can be the lone dissenting voice, a neat idea but I wonder if the maneuvering and bloat required just for perpetually fresh shorthand links, tiny as it may be, may still not be worth the trouble. Or is there another aspect to this I am missing?

Also just eyeballing that FDSN link now and all but two niche servers are already hard coded (missing IESDMC which I think was previously listed as BATS, and KAGSR which doesn't appear to link to a valid fdsnws link), and several notable servers appear to be missing entirely?

Anyway I just put in a PR to add IESDMC for the next release (seems imminent?) which hopefully keeps this up to date for a long while.

[jschaeff]

There is a special use case here at Résif (sorry in advance for that): the webservices are not all hosted by the same institute. Thus we have:

• dataselect: ws.resif.fr/fdsnws/dataselect
• station: ws.resif.fr/fdsnws/station
• event: http://api.franceseisme.fr/fdsnws/event/1

The way fdsn.client behaves it is not possible to set this up I guess. Do you have any idea ?

[filefolder]

That's a tough one, if it were me I might just make two entries, e.g. "RESIF" and "RESIF_EVENT"

or, attempt some sort of address forwarding mechanism, so calls to ws.resif.fr/fdsnws/event are directed to the other server

[jschaeff]

I think obspy does not follow the redirections.
For the moment, the workaround is to provide the event endpoint as a parameter for the client:

from obspy.clients.fdsn.client import Client

client=Client('RESIF', service_mappings={'event': 'https://api.franceseisme.fr/fdsnws/event/1'})
event=client.get_events(eventid='fr2019clftsg')

[ThomasLecocq]

There is a special use case here at Résif (sorry in advance for that): the webservices are not all hosted by the same institute. Thus we have:

• dataselect: ws.resif.fr/fdsnws/dataselect
• station: ws.resif.fr/fdsnws/station
• event: http://api.franceseisme.fr/fdsnws/event/1

The way fdsn.client behaves it is not possible to set this up I guess. Do you have any idea ?

(entering the French-French discussions here 😆 ), but would it be imaginable to create a proxy/http redirect/other from ws.resif.fr/fdsnws/event to api.franceseisme.fr/fdsnws/event ?

[jschaeff]

About the redirection, it was the case, but a lot of clients does not follow HTTP 300 response, so we removed it.