Astroquery: Continuous delivery of master branch

[keflavich]

We have a hard time keeping up with releases, and the default way for people to install astroquery is with pip install astroquery. Given that we encourage people to use the latest master, we should just be posting the branch to pypi every time a PR is merged.

This issue is to remind us to do that and to address practical questions. e.g., are there limits to what we can post on pypi? How do I set up travis-ci to python setup.py sdist upload on (successful) master branch merges?

[pllim]

How does change log work in this delivery scheme? :thinking:

[bsipocz]

@pllim - That's one of the open questions I have, too.

We could keep the current x.y.z scheme in the changelog, and release the interim 0.3.9.1234 for pypi. Once we had enough we can tag the 0.3.9 just as we did (or maybe in this case 0.3.10), and then package that with conda, too. Until then collect the changelog.

I've also though about restructuring by modules, just as we have in astropy.

The alternative to keep a strictly chronological order, but I fear that would be a bit messy when we look back ~year.

[keflavich]

We can still have numbered versions for 'big changes', and we'll just file the rest under 'latest'. So, pretty much as @bsipocz said.

I don't want to restructure by modules until we start getting multiple contributions over several modules in a 3-month period. If that happens, I will concede that we have grown too much.

[pllim]

Yeah, what you both said make sense. I don't think anyone would object to keeping the current change log scheme. Although, maybe change "(unreleased)" label in the change log section to something less confusing, like maybe... "(released as dev on PyPI only)"?

As for package restructuring, given the necessity of backward compatible API, it needs to be done carefully, but I have no objection to that as well.

[pllim]

BTW is astroquery is on conda too, right? How do you reconcile the different versions that users will get when they pip install vs conda install? At the very least, this needs a clear documentation.

[bsipocz]

Yeah, there are still a few practical questions. For conda, one way could be to package only the pagged versions, but I plan to ask about current practices from the conda-forge maintainers.

[keflavich]

1130 got us closer to the problem, but we still have regularly failing cronjobs. Some of these are remote service failures, i.e., HTTP response failures.

@bsipocz, what do you think about running the cronjobs with all HTTP errors as 'allowed failures'? This might hide some legitimate failures, but I think it's the only way we'll ever get all-green jobs. If you like this, do you have suggestions on how to implement it?

[keflavich]

We can use pytest-dependency to skip tests if the basic service is down:
https://github.com/astropy/astroquery/pull/1133/commits/0afdfe2fe00f8921df7d91f6822f068d29d7c3ad

[keflavich]

First delivery:
https://pypi.org/project/astroquery/0.3.9.dev343/

[keflavich]

From now on, you can install the latest astroquery with pip install --pre astroquery.

[bsipocz]

This is now reality, so I'm closing the issue.