astroquery.simbad.query_bibcode crashes on bibcode '2019RAA....19..138A'

[keithini]

I have used this function successfuly many times. However it repeatedly crashes when presented with the above bibcode. It reported 👎

  File "/home/keith/anaconda3/lib/python3.7/site-packages/astroquery/simbad/core.py", line 866, in query_bibcode
    verbose=verbose)

  File "/home/keith/anaconda3/lib/python3.7/site-packages/astroquery/simbad/core.py", line 1064, in _parse_result
    "Exception: " + str(ex))

TableParseError: Failed to parse SIMBAD result! The raw response can be found in self.last_response, and the error in self.last_table_parse_error. The attempted parsed result is in self.last_parsed_result.
 Exception: Unable to insert row because of exception in column 'References':
'ascii' codec can't encode character '\u2013' in position 38: ordinal not in range(128)

x3=Simbad.query_bibcode('2019RAA....19..138A')[0]
Traceback (most recent call last):

  File "/home/keith/anaconda3/lib/python3.7/site-packages/astropy/table/table.py", line 2766, in insert_row
    newcol = col.insert(index, val, axis=0)

  File "/home/keith/anaconda3/lib/python3.7/site-packages/astropy/table/column.py", line 1066, in insert
    data = np.insert(self_for_insert, obj, values, axis=axis)

  File "<__array_function__ internals>", line 6, in insert

  File "/home/keith/anaconda3/lib/python3.7/site-packages/numpy/lib/function_base.py", line 4583, in insert
    values = array(values, copy=False, ndmin=arr.ndim, dtype=arr.dtype)

UnicodeEncodeError: 'ascii' codec can't encode character '\u2013' in position 38: ordinal not in range(128)


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/home/keith/anaconda3/lib/python3.7/site-packages/astroquery/simbad/core.py", line 1053, in _parse_result
    resulttable = self.last_parsed_result.table

  File "/home/keith/anaconda3/lib/python3.7/site-packages/astroquery/simbad/core.py", line 211, in table
    table.add_row([ref])

  File "/home/keith/anaconda3/lib/python3.7/site-packages/astropy/table/table.py", line 2658, in add_row
    self.insert_row(len(self), vals, mask)

  File "/home/keith/anaconda3/lib/python3.7/site-packages/astropy/table/table.py", line 2790, in insert_row
    .format(name, err))

ValueError: Unable to insert row because of exception in column 'References':
'ascii' codec can't encode character '\u2013' in position 38: ordinal not in range(128)


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "<ipython-input-18-2843f2a7e1fb>", line 1, in <module>
    x3=Simbad.query_bibcode('2019RAA....19..138A')[0]

  File "/home/keith/anaconda3/lib/python3.7/site-packages/astroquery/simbad/core.py", line 866, in query_bibcode
    verbose=verbose)

  File "/home/keith/anaconda3/lib/python3.7/site-packages/astroquery/simbad/core.py", line 1064, in _parse_result
    "Exception: " + str(ex))

TableParseError: Failed to parse SIMBAD result! The raw response can be found in self.last_response, and the error in self.last_table_parse_error. The attempted parsed result is in self.last_parsed_result.
 Exception: Unable to insert row because of exception in column 'References':
'ascii' codec can't encode character '\u2013' in position 38: ordinal not in range(128)

[bsipocz]

Interesting, thanks for the report @keithini! I see the same failure, while the payload looks to be a valid script (http://simbad.u-strasbg.fr/simbad/sim-script?submit=submit+script&script=query+bibcode++2019RAA...19..138A)

'\u2013' is en dash, and not yet clear how it comes into the picture.

[aoberto]

Hi, the dash was in the DOI, so I have corrected this in SIMBAD. Anyway, data in SIMBAD are in UTF-8 because we have some greek letters sometimes. For instance, the 2019PASJ...71...55K article got a "alpha" in the title, so all the query methods should be prepared to get UTF-8 data in texts.

[keflavich]

It looks like this was solved upstream, thanks @aoberto.

[bsipocz]

Keep this open. The one entry has been fixed manually, but the parsing issue @aoberto raised is still valid and is a bug in astroquery.