Astroquery: Implement archival data service interfaces, and a frontend to query all

[keflavich]

Services:

• [x] [MAST](http://archive.stsci.edu/)
• [x] [ESO](http://archive.eso.org/)
• [x] [CADC](http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/)
• [x] [NRAO](http://archive.nrao.edu)
• [ ] [NOAO](http://archive.noao.edu)
• [ ] [IRAM](https://tapas.iram.es/tapas/)
• [ ] [SMA](http://www.cfa.harvard.edu/rtdc/data/search.html)
• [ ] [CARMA](http://carma-server.ncsa.uiuc.edu:8181/)
• [x] [NRAO ALMA](https://almascience.nrao.edu/aq/) / ESO ALMA
• [ ] [ING](casu.ast.cam.ac.uk/casuadc/ingarch/query)
• [ ] [CHANDRA](http://cxc.harvard.edu/cda/footprint/) and CHANDRA WebChaser

It would be particularly helpful to be able to query a position or region given a wavelength range and find all the observations taken of it.

[vilhelmp]

...and the ALMA archive! :)

https://almascience.nrao.edu/aq/

[keflavich]

Added!

[bsipocz]

What about the ING archive?

[keflavich]

@bsipocz can you provide a link?

[bsipocz]

http://casu.ast.cam.ac.uk/casuadc/ingarch/query

It's also one of those staging ones in #274.

[bsipocz]

I guess http://archive.eso.org/eso/eso_archive_main.html is the main point of entry to check whether there are any ESO observations for someone's favourite object(s). Would be nice to have it in astroquery, too.

[keflavich]

ALMA: #411

[bsipocz]

@keflavich -See my comment above for the missing eso query.

[astrofrog]

@keflavich - do you know if anyone has worked on the interface to the SMA archive?

[keflavich]

@astrofrog no, afaik, no one has. Would be great to have it.

[pllim]

Is this information enough to have some kind of implementation for MAST? http://archive.stsci.edu/archive_news/2016/12-Dec/index.html#article2

[bsipocz]

@pllim - I think the biggest blocker is that neither of us have enough free time to implement the list above and it's quite difficult to find the right balance between making sure everything we have is still working, fixing issues with it or add new services.

But: we have a template for new modules [0], so it shouldn't be too complicated for an astroquery novice person to add a new service.

[0] http://astroquery.readthedocs.io/en/latest/template.html

[pllim]

c/c @arfon ?

[arfon]

c/c @arfon ?

👀 - yep, this is on my radar. Don't have roadmaps/timelines to share yet but this is something I'd love to have us work on very soon.

[bsipocz]

@arfon - Is there an API docs for MAST? Or should astroquery go through the webforms? I though to give it a go during hackaas, but it seems to be one of the more complicated ones that goes beyond a half hackday project.

[arfon]

@arfon - Is there an API docs for MAST?

@bsipocz - not currently as far as I know. I believe the MAST team are working on doing this ASAP.

[bsipocz]

Oh, that's great news, I'm looking forward for their PR.

[AyushYadav]

@bsipocz @pllim @eteq
I would like to suggest one more addition. Not sure if the API is upto the task ( I can edit it too, already working on some of its aspects). But I would like to recommend to add 3D-HST API. 3D-HST comes from the near-infrared spectroscopic survey with the Hubble Space Telescope. It is a very detailed survey of the five well-studied extragalactic survey fields (AEGIS, COSMOS, GOODS-S, UKIDSS-UDS) with two orbits of primary WFC3/G141 grism coverage and two to four orbits with ACS/G800L coverage Link. I am the developer of its API and web interface, and would like to incorporate this into the astroquery as a module. If possible within the GSoC'17 purview.

[bsipocz]

@AyushYadav - We can definitely talk about it.
Personally would prefer not to have tons of smaller modules for every survey of a few orbits but rather to have one big module to access all HST/WFC3 preferably via a MAST module. However I'm not familiar with this survey, so others may have better formed opinions.

[arfon]

/ cc @ivastar

[ivastar]

Ha! This went full circle in ~8 hours. @AyushYadav is working with me and was interested in exploring this so I encouraged him to do so. He would very much appreciate feedback on the idea.

[arfon]

Ha! This went full circle in ~8 hours. @AyushYadav is working with me and was interested in exploring this so I encouraged him to do so. He would very much appreciate feedback on the idea.

Lol, good stuff!

[ivastar]

@bsipocz the dataset @AyushYadav is working with contains the vast majority of space- and ground-based observations in the 5 deep extra-galactic fields (AEGIS, COSMOS, GOODS-N/S, UDS), some ~3000 orbits of Hubble data + ~150 bands of ground-based data + low-resolution grism spectra for everything. Hosting these in the MAST database is being explored, but may not quite work with the dataset since these are mainly high-level science products. That said, it seems like there are other observatories still in need of astroquery so this may not be the highest of priorities.

[keflavich]

We won't reject such a contribution. There are other "small data sets" like these that already have services. So, you decide whether it's a priority. It's not one I would push for GSoC, though.

[bsipocz]

@ivastar @AyushYadav - OK, so there is a difference between ~5 and 3000 orbits :) Yet, I agree with @keflavich, that as part of GSoC we would prefer to get a few modules from the missing big archives (either from the list above, or from the requests posted by our users.

Anyway, it would be interesting to have a discussion about how to structure access to science products like this, that are too complex to be hosted on e.g. Vizier, but are only a fraction of the produce of a given observatory.

[pllim]

MAST xref #896

[AyushYadav]

NOAO ref #923

[BisariaUtkarsh]

Hello
Is this project going to be taken up for gsoc 2020??

[bsipocz]

No, there won't be any astroquery gsoc projects this year.