Teslamate: Feature request: CSV data import (from TeslaFi)

[fetzu]

I think a lot of users are trying to move out their data and API keys of closed-source/proprietary but would still like to have their "old" data available in their database.

Since some of these services (for example TeslaFi) allow users to export their data in CSV, it would be nice to have the ability to import that data into TeslaMate's database. I imagine it would mean implementing a small import (and eventually export?) tool into TeslaMate's UI to inject data into the DB.

Thank you for the awesome work !

[adriankumpf]

That would be nice to have! I understand why you and others would really like to see that feature. However, since this is an open source project, I don't earn money with it (besides a few donations). That's why I'm not particularly interested in putting effort into writing a script which helps people to move their data from a payed service to a one financed by donations.

However, if someone manages to create such a migration script / tool I would gladly accept a PR and merge it, so that other people can use it too.

[fetzu]

@adriankumpf : it makes perfect sense. I don't know the first thing about Elixir and not much about SQL, but I'll look around if I can find an existing tool that could be adapted for that purpose. If I can come up with anything substantial I'll make sure to keep this updated/make a PR.

[adriankumpf]

Cool, thanks!

[olexs]

I am planning the same thing, only for https://github.com/lephisto/tesla-apiscraper instead of Teslafi. I have used that setup before TeslaMate and have several months of data in the InfluxDB used by apiscraper. Currently working out a way to get it transferred. The data is stored in a very different manner from how it's handled in TeslaMate's PostgreSQL, so I guess I'll have to learn some Elixir and run the data through TeslaMate's own logic to discern drives, charge sessions and other data points... we'll see how well that works.

[bahree]

This is a little different, but somewhat related - I wanted to see if there is a way to create a backup / restore feature from TeslaMate - after the docker issues on Synology, I did lose the data and wanted to get some ability to do this.

Any ideas on how to connect to the db running in the docker container? Will try and see if I can create a script just to dump out the db contents - or if there is a export / backup in PostgreSQL? Thanks.

[olexs]

You could expose the PostgreSQL port in the docker-compose.yml (in the same way as Grafana and TeslaMate's ports are exposed) and then connect from "outside" using any SQL client.

[Sebbo69]

Can i import Data from Teslalogger?

[spacecosmos]

This is a little different, but somewhat related - I wanted to see if there is a way to create a backup / restore feature from TeslaMate - after the docker issues on Synology, I did lose the data and wanted to get some ability to do this.

Any ideas on how to connect to the db running in the docker container? Will try and see if I can create a script just to dump out the db contents - or if there is a export / backup in PostgreSQL? Thanks.

@bahree - Did you manage to get this to work? If so, how?

I have activated backups through my cloud provider, DigitalOcean. But they only take backups four times a month. And having something every day is definitely preferred.

[krezac]

@bahree - to get to postgres console
docker ps - in output find the container_id for postgres
docker exec -it /bin/sh - this gives you container shell
su postgres - switch user
psql
now do the operations you need. Lets assume you have created /tmp/backup.tgz
logout from psql and shell
docker copy :/tmp/backup.tgz . - copy the file from container.

[DrMichael]

I'd say a

docker exec -it pg_dump > teslamate.bck

Would also do it. Interesting question is, how to restore. I used the following:
Stopped all containers except the DB one.
Ran "docker exec -it psql -U teslamate teslamate"
Deleted all existing tables there and left
Copied the backup: "docker copy teslamate.bck:/tmp"
"docker exec -it /bin/sh" and then there:
"psql -U teslamate teslamate < /tmp/teslamate.bck"

Rather clumsy, I think. Any smoother way appreciated.

[krezac]

Another option would be to export the port and run any backup commands from the host machine.

[ngardiner]

Rather clumsy, I think. Any smoother way appreciated.

Instead of removing all the tables, you could likely just:

dropdb teslamate
createdb teslamate

I say likely because there's a bit more to it (some functions and extensions which are restored in the backup) but the worst that would happen is it would throw a few errors during restore for the functions that already exist, and then restore the tables.

[gundalow]

Out of interest, did anybody get anywhere with this?

[fetzu]

Unfortunately I haven't had much time to put into it. It's still on the radar, but I wouldn't be holding my breath.