Incubator-superset: WorldMap not working (getting 'None')

[cristobalcl]

Superset version: 0.17.0

The WorldMap visualization is not working in my installation. For instance, the slice '% Rural' that comes as example does not work. It shows an error bar with 'None'. And the server prints this backtrace:

2017-03-07 08:26:01,438:INFO:root:SELECT country_code AS country_code, SUM("SP_RUR_TOTL_ZS") AS "sum__SP_RUR_TOTL_ZS", SUM("SP_POP_TOTL") AS "sum__SP_POP_TOTL"
FROM wb_health_population
WHERE year >= '2014-01-01 00:00:00.000000' AND year <= '2014-01-02 00:00:00.000000' GROUP BY country_code ORDER BY "sum__SP_RUR_TOTL_ZS" DESC
 LIMIT 50000 OFFSET 0
2017-03-07 08:26:01,494:ERROR:root:None
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/superset/viz.py", line 258, in get_payload
    data = self.get_data(df)
  File "/usr/lib/python3.4/site-packages/superset/viz.py", line 1320, in get_data
    self.form_data.get('country_fieldtype'), row['country'])
  File "/usr/lib/python3.4/site-packages/superset/data/countries.py", line 2497, in get
    return all_lookups[field].get(symbol.lower())
KeyError: None
2017-03-07 08:26:01,495:INFO:root:Caching for the next 86400 seconds

When I try to add a WorldMap with my data I get another different error:

/usr/lib/python3.4/site-packages/flask_appbuilder/forms.py:245: FlaskWTFDeprecationWarning: "flask_wtf.Form" has been renamed to "FlaskForm" and will be removed in 1.0.
  form = self(obj=obj)
/usr/lib/python3.4/site-packages/flask_appbuilder/forms.py:245: FlaskWTFDeprecationWarning: "flask_wtf.Form" has been renamed to "FlaskForm" and will be removed in 1.0.
  form = self(obj=obj)
2017-03-07 08:34:47,922:ERROR:root:Metric 'None' is not valid
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/superset/viz.py", line 256, in get_payload
    df = self.get_df()
  File "/usr/lib/python3.4/site-packages/superset/viz.py", line 109, in get_df
    self.results = self.datasource.query(query_obj)
  File "/usr/lib/python3.4/site-packages/superset/models.py", line 1474, in query
    sql = self.get_query_str(engine, qry_start_dttm, **query_obj)
  File "/usr/lib/python3.4/site-packages/superset/models.py", line 1316, in get_query_str
    raise Exception(_("Metric '{}' is not valid".format(m)))
Exception: Metric 'None' is not valid
2017-03-07 08:34:47,923:INFO:root:Caching for the next 86400 seconds

I tried to debug it by myself and found that metrics has an array like [my_var, None], so in the iteration, when it comes to None it failed to valid it as a metric. I tried to filter that in the code but then appeared another error:

2017-03-07 08:43:32,901:INFO:root:SELECT source_market AS source_market, SUM(kpi_val) AS sum__kpi_val 
FROM hbgdwc.ttv_gross_service_date_source_market 
WHERE service_date >= '2016-03-07 08:43:32' AND service_date <= '2017-03-07 08:43:32' GROUP BY source_market ORDER BY sum__kpi_val DESC 
 LIMIT 50000
2017-03-07 08:43:33,112:ERROR:root:'[None] not in index'
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/superset/viz.py", line 258, in get_payload
    data = self.get_data(df)
  File "/usr/lib/python3.4/site-packages/superset/viz.py", line 1312, in get_data
    ndf = df[cols]
  File "/usr/lib64/python3.4/site-packages/pandas/core/frame.py", line 1991, in __getitem__
    return self._getitem_array(key)
  File "/usr/lib64/python3.4/site-packages/pandas/core/frame.py", line 2035, in _getitem_array
    indexer = self.ix._convert_to_indexer(key, axis=1)
  File "/usr/lib64/python3.4/site-packages/pandas/core/indexing.py", line 1214, in _convert_to_indexer
    raise KeyError('%s not in index' % objarr[mask])
KeyError: '[None] not in index'
2017-03-07 08:43:33,115:INFO:root:Caching for the next 86400 seconds

I suppose that the problems with the '% Rural' example and my data are differents, but I can't tell.

Also, I have added a key for the MapBox API, but I think this is irrelevant, as that map doesn't use it.

Thank you very much.

Update

The error with my own data was my fault, and it seems it has nothing to do with the WorldMap view.

[cristobalcl]

I'm trying to debug it. It seems that the field 'country_fieldtype' is None in the server, but it has a value in the frontend. Even that fields appears correctly in the URI:

/superset/explore/table/2/?form_data=%7B%22datasource%22%3A%20%222__table%22%2C%20%22compare_suffix%22%3A%20%22o10Y%22%2C%20%22secondary_metric%22%3A%20%22sum__SP_POP_TOTL%22%2C%20%22granularity%22%3A%20%22year%22%2C%20%22slice_id%22%3A%208%2C%20%22row_limit%22%3A%2050000%2C%20%22entity%22%3A%20%22country_code%22%2C%20%22since%22%3A%20%222014-01-01%22%2C%20%22num_period_compare%22%3A%20%2210%22%2C%20%22groupby%22%3A%20%5B%5D%2C%20%22viz_type%22%3A%20%22world_map%22%2C%20%22until%22%3A%20%222014-01-02%22%2C%20%22markup_type%22%3A%20%22markdown%22%2C%20%22where%22%3A%20%22%22%2C%20%22metric%22%3A%20%22sum__SP_RUR_TOTL_ZS%22%2C%20%22country_fieldtype%22%3A%20%22cca3%22%2C%20%22show_bubbles%22%3A%20true%2C%20%22metrics%22%3A%20%5B%22sum__SP_POP_TOTL%22%5D%2C%20%22compare_lag%22%3A%20%2210%22%2C%20%22limit%22%3A%20%2225%22%7D

Does it make sense? Have I missing any package that I should install or configure anything?

[cristobalcl]

I keep debugging it. It seems that the constructor of the WorldMapViz is called twice. The first time the form_data is this:

{'metrics': ['sum__SP_POP_TOTL'], 'granularity': 'year', 'metric': 'sum__SP_RUR_TOTL_ZS', 'compare_suffix': 'o10Y', 'secondary_metric': 'sum__SP_POP_TOTL', 'until': '2014-01-02', 'entity': 'country_code', 'row_limit': 50000, 'num_period_compare': '10', 'country_fieldtype': 'cca3', 'where': '', 'since': '2014-01-01', 'compare_lag': '10', 'markup_type': 'markdown', 'show_bubbles': True, 'limit': '25', 'viz_type': 'world_map', 'groupby': []}

Which has the correct param: 'country_fieldtype': 'cca3'

The second time the constructor is called, the param is missing:

{'slice_id': 8, 'granularity_sqla': 'year', 'since': '2014-01-01', 'until': '2014-01-02', 'where': '', 'entity': 'country_code', 'metric': 'sum__SP_RUR_TOTL_ZS', 'secondary_metric': 'sum__SP_POP_TOTL', 'country_controltype': 'cca2', 'max_bubble_size': '25', 'datasource': '2__table', 'time_grain_sqla': 'Time Column', 'viz_type': 'world_map', 'filters': [], 'having': '', 'show_bubbles': True}

Am I missing something?

[mistercrunch]

Will be fixed in the next release as it's fixed in master

[cristobalcl]

Ok, I'll try to install from the master. Thanks!

[cristobalcl]

@mistercrunch could you tell me the exact commit that fixed the problem?

[cristobalcl]

I think I have it:

https://github.com/airbnb/superset/commit/9d8d4213840cc07d95ac971437632aeae4ff58b5

[mistercrunch]

Exactly. Closing.

[linwoodc3]

Hi all. I'm getting the same
Metric 'None' is not valid
error on the WorldMap visualization.

I tried using superset 0.17.4 and 0.17.1. No luck. I tried this viz on my custom dataset, and also tried it on the World Health demo data, by opening the Table and trying to create a new viz. Any ideas?

[Lomascolo]

I found that I got the same error (Metric 'None' is not valid) but was solved when I updated the Bubble Size field, even if I was not presenting bubbles (show bubbles check box unchecked).

[ryanli1994]

@Lomascolo 's advice works.

[shinabarger]

Still getting this error as of today, 5/30/2018.