Incubator-superset: Cannot display Time Series Line Chart

[syzuhdi]

Make sure these boxes are checked before submitting your issue - thank you!

• [X] I have checked the superset logs for python stacktraces and included it here as text if any
• [X] I have reproduced the issue with at least the latest released version of superset
• [X] I have checked the issue tracker for the same issue and I haven't found one similar

Superset version

0.22.1

Expected results

Data shown in Time series Line Chart

Actual results

When I click Run Query button in Slices, an error occur.
Error u'CITY'

Traceback (most recent call last):
File "/usr/local/venv/lib/python2.7/site-packages/superset/viz.py", line 278, in get_payload
data = self.get_data(df)
File "/usr/local/venv/lib/python2.7/site-packages/superset/viz.py", line 1045, in get_data
df = self.process_data(df)
File "/usr/local/venv/lib/python2.7/site-packages/superset/viz.py", line 981, in process_data
values=fd.get('metrics'))
File "/usr/local/venv/lib/python2.7/site-packages/pandas/core/reshape/pivot.py", line 135, in pivot_table
grouped = data.groupby(keys)
File "/usr/local/venv/lib/python2.7/site-packages/pandas/core/generic.py", line 4416, in groupby
*kwargs)
File "/usr/local/venv/lib/python2.7/site-packages/pandas/core/groupby.py", line 1699, in groupby
return klass(obj, by, *kwds)
File "/usr/local/venv/lib/python2.7/site-packages/pandas/core/groupby.py", line 392, in __init__
mutated=self.mutated)
File "/usr/local/venv/lib/python2.7/site-packages/pandas/core/groupby.py", line 2690, in _get_grouper
raise KeyError(gpr)
KeyError: u'CITY'

And this is the QUERY

SELECT "CITY",
"__timestamp",
"count_QTY"
FROM
(SELECT "CITY" AS "CITY",
"REGN_DATE" AS "__timestamp",
count(VIN_NO) AS "count_QTY"
FROM
(SELECT *
FROM mycrm_registration) expr_qry
JOIN
(SELECT "CITY__",
mme_inner__
FROM
(SELECT "CITY" AS "CITY__",
count(VIN_NO) AS mme_inner__
FROM
(SELECT *
FROM mycrm_registration) expr_qry
WHERE "REGN_DATE" >= TO_TIMESTAMP('2018-03-30T00:00:00', 'YYYY-MM-DD"T"HH24:MI:SS.ff6')
AND "REGN_DATE" <= TO_TIMESTAMP('2018-04-06T08:20:13', 'YYYY-MM-DD"T"HH24:MI:SS.ff6')
GROUP BY "CITY"
ORDER BY count(VIN_NO) DESC)
WHERE ROWNUM <= 100) anon_1 ON "CITY" = "CITY__"
WHERE "REGN_DATE" >= TO_TIMESTAMP('2018-03-30T00:00:00', 'YYYY-MM-DD"T"HH24:MI:SS.ff6')
AND "REGN_DATE" <= TO_TIMESTAMP('2018-04-06T08:20:13', 'YYYY-MM-DD"T"HH24:MI:SS.ff6')
GROUP BY "CITY",
"REGN_DATE"
ORDER BY count(VIN_NO) DESC)
WHERE ROWNUM <= 50000

Any idea?

Steps to reproduce

[mistercrunch]

Can you try on 0.24.0?

[jhettler]

Hi, I have found the similar issue yesterday on 0.24.0. And it is not a problem only in Time Series Line Chart, I was able to reproduce it with Pie Chart and Histogram as well, I didn't try more visualizations.

In my case, we use Snowflake database and I expected it is related to DB engine and I am still not sure what's exactly wrong, everything works fine with Superset examples and as well if I add the tables from Snowflake straight from dialog Sources/Tables/Add table (plus icon in right upper corner). But if I try to create datasource/table from query it didn't work. Then I realize there are quotes around column names in query (view query menu in graph view - upper right corner).

Same as in @syzuhdi case

SELECT "CITY",
"timestamp",
"count_QTY"

I tried to lowercase all column names and whoila, everything works. Lowercased name of columns are not in doublequotes and it is correctly interpreted.

I made following steps:

1. Run a query in SQL lab
2. Visualize it - it will ask for datasource name
3. Save the visualization
4. Go to Sources/Tables and edit the table/datasource you created in step 2
5. Rename all columns to lowercase
6. Try to visualize graph

It's not nice, but it works for us as a temporary workaround.
HTH.

[syzuhdi]

Thank you @jhettler. It works for us.

[mistercrunch]

This may help these issues:
https://github.com/apache/incubator-superset/pull/4724

[villebro]

I am also facing the same issue with Snowflake. I tracked it down to process_data in viz.py
https://github.com/apache/incubator-superset/blob/caa72675a5b69e0ba133a3e652e1f807e0163d9c/superset/viz.py#L1104-L1107, where fd.get('groupby') are uppercase, but the corresponding Series in df are lowercase. I was also able to confirm that the @jhettler hack works in my case. A simple fix in this case would be to lowercase the groupby columns, but not sure if that is ok under all circumstances. Any alternative suggestions?

[villebro]

This bug is the same as #4662, where an alternative solution is outlined.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[villebro]

@syzuhdi this issue has been resolved and can be closed

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.