Kong: API endpoint to export and import all the DB entities

[subnetmarco]

One of the most common requests is to move data from one cluster to another (or one environment to another) in an easy way. Currently the two ways this can be achieved are:

• At the datastore level, by migrating the keyspace or the database
• By replicating the same HTTP requests to the Admin API on both clusters

Kong could possibly facilitate this by providing /export and /import endpoints which basically just dump the entire content of the datastore in a JSON file, ready to be imported by another cluster.

[Tieske]

It should at least support switching datastores imo. So datastore level import/export will not suffice.

An initial implementation should not include upgrading. So you can only import data on the same Kong version that created it. If that is to be supported, then a CLI based import would be the better option (export could still be done through the rest api)

[subnetmarco]

I am actually more inclined to only have the CLI for this operations, and not an API. Since the dataset may be big (with thousand of consumers and credentials, for example) a CLI could provide better progress for the operation (without dealing with HTTP timeouts too).

[Tombar]

Hello, maybe something similar to dump/apply from kongfig is enough >
https://github.com/mybuilder/kongfig

We use them to import/export and version our API definitions.

Just my two cents

Regards

M

On Sun, Jul 10, 2016 at 5:05 PM, Marco Palladino [email protected]
wrote:

I am actually more inclined to only have the CLI for this operations, and
not an API. Since the dataset may be big (with thousand of consumers and
credentials, for example) a CLI could provide better progress for the
operation (without dealing with HTTP timeouts too).

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Mashape/kong/issues/1374#issuecomment-231608031, or mute
the thread
https://github.com/notifications/unsubscribe/AAIpgWVogJB_tpDjWDjfkmffk1D0Tsbbks5qUVCNgaJpZM4JIvDl
.

[kenspirit]

I am not sure if kong must provide a full-feature utility. But I think we can have a cli command to regenerate config based on database data? I think for migration, we can also dump & restore the DB ourself. But it seems that is not enough for kong to recognize the setting.

[thibaultcha]

Definitely want to add support for declarative configuration via the CLI and Admin API, allowing for importing/exporting data (with some metadata in the exporting tool like Kong version to avoid migrations between incompatible model versions).

[ror6ax]

I would be interested in this too.

[luispabon]

It's impossible to effectively devops kong without source-controlled configuration. We need to be able to code our Kong apis to be able to deploy the same ones on different environments.

[senthil-nathan]

Is source-controlled configuration available on Kong or is it a feature request ?

[luispabon]

There's kongfig and that's what we're using at the moment and works well enough, but it's third party therefore could break at any moment independently of kong.

[bungle]

kong config db_import <file.yml>
kong config db_export <file.yml>

Is now supported, so closing this.

[thejohn]

kong config db_import <file.yml>
kong config db_export <file.yml>

Is now supported, so closing this.

Thanks for the update. Which version of Kong is this supported?

[cilindrox]

@thejohn 1.2.x onwards IIRC