Astroquery: Module namespace restructure?

Created on 12 Dec 2018 · 12Comments · Source: astropy/astroquery

The flat namespace of astroquery.xx is getting crowded. Do we want to restructure the namespace to be more hierarchical, either archive-centric or mission-centric?

Source

keflavich

Most helpful comment

As part of this work, perhaps also good to produce a dev guide to say something like "before you develop a new module for astroquery, please check here and there to make sure you are not re-implementing something that astroquery already provides; if you are from such and such org, please name your module like this" or something like that.

pllim on 14 Dec 2018

👍2

All 12 comments

In a backward compatible way or not? Also, care needs to be given in picking categories in case some services fall into more than one.

pllim on 12 Dec 2018

Some ideas for how the namespaces could be restructured:

(1) Breakdown by "class" of the archive service, e.g.:
astroquery.archives.esa
astroquery.archives.mast
astroquery.surveys.sdss
astroquery.surveys.ukidss
astroquery.lines.hitran,
astroquery.tools.tap,
astroquery.cutouts.skyview
(but it's not clear whether these top-level queries are unique)

(2) Breakdown by archive:
astroquery.esa.hubble
astroquery.esa.herschel
astroquery.mast.hubble
astroquery.esa.planck
astroquery.lambda.planck

(3) Keep the namespace flat:
astroquery.esa_hubble
astroquery.esa_herschel
astroquery.esa_planck
astroquery.mast_hubble
astroquery.mast_planck

Other suggestions?

keflavich on 12 Dec 2018

@pllim - excellent question. I don't like the idea of breaking backward compatibility at all, which inclines me toward using a flat namespace, but the flat namespace as I laid out above is pretty confusing; why isn't there a single ESA interface as there is a single MAST interface?

The distinction between surveys and missions seems important here. GAIA is both a survey and and a mission, but the GAIA archive provides access to the GAIA survey products, so it is more analogous to the SDSS and UKIDSS modules, while Hubble, Herschel, Spitzer, etc. are more analogous to ground-based archives like ALMA and perhaps NOAO, since they are serving up individual PI-led observations rather than reduced data from large-scale surveys.

keflavich on 12 Dec 2018

Perhaps we should ask the broader community via one of the mailing lists?

pllim on 12 Dec 2018

The intent is to ask the broader community to contribute here. We can ask on the mailing lists, but I would like the discussion to be on this issue.

keflavich on 12 Dec 2018

👍2

I would prefer option No2. That would keep the current logic but provide some cleanup. E.g. as you say it's kind of confusing to have everything mast live under the module, but esa and jpl and irsa are spreading all over the space.

TAP and other future tools/utilities can keep on live in utils, at least putting tap in there still makes sense.

One category where I think it would be useful to mix in some of the logic from option No1 is that all the linelist could be grouped under one subpackage, where also the solar system stuff (mpc/jpl) can also come from one sub module (solarsystem already exists as part of this idea).

And, well, we can probably also deprecate a few modules as they are a subset of other modules. E.g. alfaalfa points to ned, so it shouldn't be in a separate place any more https://github.com/astropy/astroquery/issues/1285).

Then there is the question of secondary data products, like all 3 exoplanet catalogue queries we have. After all, they may just moved under an exoplanet submodule?

We can do it all in a backward compatible way by making the currently existing modules accessible from the top namespace with a warning about the restructure.

bsipocz on 14 Dec 2018

This being said, I think we should do it as soon as possible, but on the other hand only if when we know we have substantial chuck of time to do it in one short burst of work to make the process as smooth as possible.

bsipocz on 14 Dec 2018

We need to reach a consensus on the approach we're going to take, then schedule a work week.

keflavich on 14 Dec 2018

👍1

pllim on 14 Dec 2018

👍2

I also prefer option number 2 for the reasons @bsipocz outlines.

Additionally I think that option 1 will lead to cases where there isn't one clear home for certain classes. I.e. would tesscut stay with mast (astroquery.archive.mast.tesscut) or go in cutouts (astroquery.cutouts.tesscut).