Cartodb: feature request: Import API: add `overwrite` to `collision_strategy`

[andy-esch]

In cartoframes development, we rely on the Import API for updating a table in a user's account.

Having a collision_strategy=overwrite option would make our development cleaner and allow us to offload the complexity of data/map management to CARTO's APIs instead of having to rely on the unintended consequences of dropping a table that a visualization is attached to.

• [ ] Strategy
  
  
  
  • [ ] If the schema does not exactly match, raise an error.
  
  
• [ ] Documentation

cc @juanignaciosl

[juanignaciosl]

cc @noguerol @javisantana

[andy-esch]

Looks like this is a duplicate of this one: https://github.com/CartoDB/cartodb/issues/9776

[juanignaciosl]

Yep, it's similar (although this one doesn't address the naming).

As this is an enhancement, I think that we should ask @kevin-reilly about implementing this.

[juanignaciosl]

I'd like to explain the use case, step by step:

1. Let's say Andy uploads a dataset with Import API and he creates a map with it.
2. Some time later, he wants to change the data, but...

• He could use the SQL API, but as he has a file, it's not straightforward.
• He can't just use the Import API, because the table already exists, so he wouldn't override it but create a new one.
• He can't drop the dataset before reimporting it, because the visualization would be deleted.
• He could just sync file feature from Import API. He could push the new file and then force the sync. This has one obvious limitation: the file must be available at the Internet (an URL, Dropbox...).

@andy-esch what do you think? Am I missing something?

@noguerol what do you think? Does this worth the effort, or should we just encourage using sync tables for this use case?

[noguerol]

Makes sense to me but it seems to be more an internal than an external request, so I would love to have @CartoDB/support opinion about the user impact this could have.

[andy-esch]

@juanignaciosl, that sums it up really well. Being able to swap out an existing dataset with a new one without having the ability to rely on the sync feature of the Import API sums it up really well. This exact issue came up in a meeting we had with a city agency (@zingbot can give more details) where they could not host files on a server or through Dropbox, etc, but wanted to programmatically swap out the old dataset with a new one with proprietary data.

[javisantana]

If we want to implement this we should close the scope a lot, specifically about table schema. If the schema does not exactly match I'd raise an error.

To give some perspective. We had (during years) a feature called import append flag. Actually we had that feature in the first version of the editor, it was always disabled and we finally removed it because we didn't implement it. I also heard a lot of times the "replace table" thing

[jsanz]

+1 to raise an error if schema changes

--
Jorge Sanz
CARTO - Predict through location

Sent from my phone, excuse my brevity.

El 26 jun. 2017 19:23, "javi santana" [e-mail] escribió:

If we want to implement this we should close the scope a lot, specifically
about table schema. If the schema does not exactly match I'd raise an error.

To give some perspective. We had (during years) a feature called import
append flag. Actually we had that feature in the first version of the
editor, it was always disabled and we finally removed it because we didn't
implement it. I also heard a lot of times the "replace table" thing

—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
https://github.com/CartoDB/cartodb/issues/11990#issuecomment-311125502,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AALfaNS1Z7uFycE_b7aV9sIG5G3U_OVxks5sH-kTgaJpZM4M7mUL
.

[noguerol]

Let's do it that way.

[juanignaciosl]

Ok. Should this feature be available for FREE users? It's quite similar to sync tables, which is not available for free users.

[noguerol]

Interesting. I say no, but @kevin-reilly to decide

[javisantana]

FREE users don't have api key at the end of the day so they'd not be able to use it. (of course they can grabbing the api key from some requests but you know)

[javitonino]

In production. Documentation ticket: https://github.com/CartoDB/docs/pull/1245

[juanignaciosl]

@andy-esch give this a try at your CartoFrames workflow. If you find bugs or rough edges open a new issue and we'll discuss it.