Astroquery: Windows 10 test failures

[MSeifert04]

On Windows some tests failed with python setup.py test: I've placed the log in a gist: https://gist.github.com/MSeifert04/b2aef84e761c532bc8b747b01d13f392

[bsipocz]

We don't have appveyor CI running for astroquery, thus it's expected that there are various windows failures.

Actually I'm not sure whether this is deliberate or not, @keflavich?

Anyway any help ironing these out is very welcome.

[MSeifert04]

@bsipocz In https://github.com/astropy/astroquery/pull/709#issuecomment-225684630 @keflavich asked if the appveyor tests should be enabled.

Most of these test failures are because \\ (windows) is used instead of / (linux) as folder-seperator. Maybe it's just a matter of using os.path.join or pathlib (python 3.4+) to create paths.

Unfortunatly I don't have much time going after them just now.

[bsipocz]

@MSeifert04 - Ahh, OK, I hasn't follow that issue. In this case I'll put this on my todo.

[bsipocz]

@MSeifert04 - Could you please check whether any of these are still there?

[MSeifert04]

I'm running the remote-data tests now. I'll upload the encountered failures :smile:

[MSeifert04]

Unfortunatly there were some failures.

https://gist.github.com/MSeifert04/7322bafb40ed58be449a33ba9acf0560

There are some failures even without remote (Simbad) but most of the remote tests seem real or due to ConnectionError.

[MSeifert04]

@bsipocz (or someone else): Do the remote data tests work on other platforms? If not could you upload an Error-log here so I can focus on the windows-specific failures?

[bsipocz]

@MSeifert04 - There are quite a few failures, the log is here: https://gist.github.com/bsipocz/f4951ae7d5060611967db312ed1745d3 and in #724

[MSeifert04]

@bsipocz Thank you.

The difference (probably windows specific failures):

• alma_remote: last test
• sdss_remote: lots of tests
• simbad: not remote, not appveyor, only local encountered failures

I'll have a look at these.

[bsipocz]

Currently there 6 alma remote failures, everything else passes for me. Could you check whether you see any windows specific issues other than those? Would be nice to close this issue.

[MSeifert04]

I will run the tests and report back.

[MSeifert04]

There are two weird failures apart from the known alma remote tests:

______________________________ test_xmatch_query ______________________________

xmatch = <astroquery.xmatch.core.XMatchClass object at 0x0000029C512D4D68>

    @remote_data
    def test_xmatch_query(xmatch):
        with open(os.path.join(DATA_DIR, 'posList.csv'), 'r') as pos_list:
            table = xmatch.query(
                cat1=pos_list, cat2='vizier:II/246/out', max_distance=5 * arcsec,
                colRA1='ra', colDec1='dec')
        assert isinstance(table, Table)
        assert table.colnames == [
            'angDist', 'ra', 'dec', 'my_id', '2MASS', 'RAJ2000', 'DEJ2000',
            'errHalfMaj', 'errHalfMin', 'errPosAng', 'Jmag', 'Hmag', 'Kmag',
            'e_Jmag', 'e_Hmag', 'e_Kmag', 'Qfl', 'Rfl', 'X', 'MeasureJD']
        assert len(table) == 11

>       http_test_table = http_test()

astroquery\xmatch\tests\test_xmatch_remote.py:54:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
astroquery\xmatch\tests\test_xmatch_remote.py:81: in http_test
    table = ascii.read(outfile, format='csv')
c:\-\astropy\astropy\io\ascii\ui.py:328: in read
    dat = fast_reader.read(table)
c:\-\astropy\astropy\io\ascii\fastbasic.py:116: in read
    return Table(data, names=list(self.engine.get_names()), meta=meta)
c:\-\astropy\astropy\table\table.py:377: in __init__
    init_func(data, names, dtype, n_cols, copy)
c:\-\astropy\astropy\table\table.py:641: in _init_from_list
    self._init_from_cols(cols)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Table length=0>
<No columns>, cols = []

    def _init_from_cols(self, cols):
        """Initialize table from a list of Column or mixin objects"""

        lengths = set(len(col) for col in cols)
        if len(lengths) != 1:
            raise ValueError('Inconsistent data column lengths: {0}'
>                            .format(lengths))
E           ValueError: Inconsistent data column lengths: set()

c:\-\astropy\astropy\table\table.py:706: ValueError
---------------------------- Captured stderr call -----------------------------
Der Befehl "curl" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.
_______________________ test_xmatch_query_astropy_table _______________________

xmatch = <astroquery.xmatch.core.XMatchClass object at 0x0000029C512BF9B0>

    @remote_data
    def test_xmatch_query_astropy_table(xmatch):
        datapath = os.path.join(DATA_DIR, 'posList.csv')
        input_table = Table.read(datapath, format='ascii.csv')
        table = xmatch.query(
            cat1=input_table, cat2='vizier:II/246/out', max_distance=5 * arcsec,
            colRA1='ra', colDec1='dec')
        assert isinstance(table, Table)
        assert table.colnames == [
            'angDist', 'ra', 'dec', 'my_id', '2MASS', 'RAJ2000', 'DEJ2000',
            'errHalfMaj', 'errHalfMin', 'errPosAng', 'Jmag', 'Hmag', 'Kmag',
            'e_Jmag', 'e_Hmag', 'e_Kmag', 'Qfl', 'Rfl', 'X', 'MeasureJD']
        assert len(table) == 11

>       http_test_table = http_test()

astroquery\xmatch\tests\test_xmatch_remote.py:71:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
astroquery\xmatch\tests\test_xmatch_remote.py:81: in http_test
    table = ascii.read(outfile, format='csv')
c:\-\astropy\astropy\io\ascii\ui.py:328: in read
    dat = fast_reader.read(table)
c:\-\astropy\astropy\io\ascii\fastbasic.py:116: in read
    return Table(data, names=list(self.engine.get_names()), meta=meta)
c:\-\astropy\astropy\table\table.py:377: in __init__
    init_func(data, names, dtype, n_cols, copy)
c:\-\astropy\astropy\table\table.py:641: in _init_from_list
    self._init_from_cols(cols)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Table length=0>
<No columns>, cols = []

    def _init_from_cols(self, cols):
        """Initialize table from a list of Column or mixin objects"""

        lengths = set(len(col) for col in cols)
        if len(lengths) != 1:
            raise ValueError('Inconsistent data column lengths: {0}'
>                            .format(lengths))
E           ValueError: Inconsistent data column lengths: set()

c:\-\astropy\astropy\table\table.py:706: ValueError
---------------------------- Captured stderr call -----------------------------
Der Befehl "curl" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.

[bsipocz]

I wonder whether @keflavich saw any similar failure when fixing xmatch recently?

[bsipocz]

@MSeifert04 - What about changing the os.path.join()s to url_helpers.join() in astroquery/xmatch/tests/test_xmatch_remote.py? That did the trick for some other modules?

[MSeifert04]

Oh, just realized that the problem could be simply that I don't have curl - that's what the stderr (german locale, sorry!) message means.

[MSeifert04]

Yeah, installing curl fixed it. 😅

[bsipocz]

Sounds great, in that case I'm closing this and declare astroquery windows compatible.

[MSeifert04]

Maybe curl could be included in the dependencies?

[bsipocz]

Is there maybe a windows equivalent that is installed by default? Otherwise, yes, it can be added to the docs.

[MSeifert04]

Not sure but at least curl is avaiable via conda.

[bsipocz]

That's OK then, PR's coming soon. See #796.