Incubator-superset: Upload CSV does not work, in replacement and append mode

[pengyejun]

errmsg: Table name abc already exists. Please pick another

But I chose the append and replace mode.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.81. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[willbarrett]

Hi @pengyejun - what flavor of database are you attempting to load data into (postgres, bigquery, etc.)? Thanks for the additional information!

[axuew]

Mairadb has the same problem

[villebro]

I was able to reproduce. It's not that the csv file isn't being uploaded; rather Superset is complaining that the Superset table (=metadata) is already present. I'm thinking users expect replace and append to assume that the table metadata already exists, and to only try to create metadata if inserting in fail mode. Will put through a PR soon.

[pengyejun]

Hi @villebro -Thank you for your thoughts, but I found a problem. When replacing the upload csv, the subsequent upload file is different from the first upload file CSV header line. At this time, the table_columns table stores the structure error.

[villebro]

@pengyejun please check PR #8457 , is this in line with how you would expect CSV uploading to work?

[pengyejun]

@villebro thank you for your reply! I saw your PR, But it didn't solve my problem. When I want to replace and modify the CSV header structure, I expect that the table_columns table stores the same structure as the CSV header.

[pengyejun]

@villebro I think you can check the tables when uploading the CSV. If there is a direct deletion, regenerate the metadata.

[villebro]

Currently refreshing the metadata doesn't remove pre-existing but non-present columns from table metadata. Having support for removing non-present columns is a good feature, but requires a separate PR and more discussion on how this should be facilitated so as not to break existing charts. E.g. adding an option "remove non-existent columns when refreshing metadata" in the table metadata could be make sense.

[pengyejun]

@villebro Thanks, your suggestion is very correct.