Jellyfin: Multi-database support

[drakus72]

Possibility to add mysql/mariadb to replace or as an option to use instead of sqlite?

[brianjmurrell]

What do you see as the benefit?

[NumberedThought]

The ability to use it as a shared data repository for multiple servers. This would allow us to load balance.

[joshuaboniface]

100% on board with this.

[brianjmurrell]

@NumberedThought Fair enough.

But it would take more than just a network-database to properly support load balancing. You wouldn't want somebody deleting something on jellyfin server 1 while somebody else was viewing on jellyfin server 2 for example, so some locking would need to be put into place.

But that said, mysql/mariadb should be an option only. Sqlite should be perfectly fine (and preferable) for simpler installations that don't need the overhead of managing mysql/mariadb.

[drakus72]

I run a mariadb cluster at home and it would allow me & everyone else to keep all settings/libraries, etc in case of a failure and have to reinstall. The DB would never have to be rebuilt with all the shows/movies, etc. It really is a no brainer todo this. Emby 3.5 was slow as hell with sqlite and even with the current updates, it is still slow. Add mysql/maria and watch the speed increase.

Not sure on others., but my sqlite db is over 700MB (has been trimmed, etc), sqlite can handle this, but it gets slow, so why not use a real database.

Even sqlite recommends a RDBMS when doing client/server. https://www.sqlite.org/whentouse.html

Also as others have said, load balancing, shared resources/data by doing it this way.

[drakus72]

some locking would need to be put into place.

Exactly what is needed.

I understand some/maybe a lot won't want to do it, but I am sure that there are several others that will want it. This topic has been posted many & requested many times on the Emby forums.

[joshuaboniface]

@drakus72 Yup, the sqlite DB is slow as hell with my massive media collection. After the LDAP fiasco I never bothered asking for this in the upstream, but we can definitely start to work on it.

I think we should support all the big RDBMS's: MariaDB, MySQL, PostgreSQL, and SQLite, using a featureful connector.

[drakus72]

@joshuaboniface

I think we should support all the big RDBMS's: MariaDB, MySQL, PostgreSQL, and SQLite, using a featureful connector.

There is other software that will install the database right along with it. One such software that I use is SAM Broadcaster, it installs firebird with it as default as long as you don't select it, it will setup one of the other options like mysql/postgres/MSSQL during setup.

[joshuaboniface]

Yes I'd much rather use a 3rd party connector library, saves us the work of having to support every little bug in the various RDBMSes.

[brianjmurrell]

My only concern is requiring people who want to use a simple (i.e. single instance for a household perhaps) instance is the added complexity of having to install and configure the RDBMS when the simplicity of sqlite is sufficient.

[joshuaboniface]

@brianjmurrell agreed, we should keep sqlite as the default, and simply allow the option to use a more complex RDBMS for those who want to.

[BnMcG]

EntityFramework Core is well-supported and pretty database agnostic, it already has providers for SQLite, Postgres, MariaDB, etc.

[EraYaN]

The SQLite backend for EFCore is maintained by Microsoft it seems: https://docs.microsoft.com/en-us/ef/core/providers/sqlite/ So that is nice.

[majorcyto]

Oh good someone already submitted this, I would LOVE to be able to use MySQL/MariaDB as the back end as well. Most people already covered reasons above.

[drissslim]

What about sync solutions based on SqlLite such as LiteSync or rqlite. I have found this thread talking about this subject https://stackoverflow.com/questions/16032825/method-to-replicate-sqlite-database-across-multiple-servers
Is there has tried to explore a such solution?

[JustAMan]

What use is this? Current DB backend and schema are limiting us in all new features anyway, so we want to move to EFcore regardless of syncing. And after the move we should be able to use any DB backend including those which have built-in support for syncing and stuff.

[MasterCATZ]

Server version: 5.7.27-30-57-log Percona XtraDB Cluster (GPL), Release rel30, Revision 64987d4, WSREP version 31.39, wsrep_31.39

is what I use @ home with 2x external nodes for redundancy

is their anyway to point to our own db-servers ?

[anthonylavado]

@MasterCATZ The server only uses SQLite right now, we need to keep working on the migration and support before we can add any other database types.

All: https://github.com/jellyfin/jellyfin/pull/1431

[SerialVelocity]

Something to keep in mind is that setting up SQL databases as HA isn't that easy as most of them work in a master-slave fashion (e.g. Postgate HA using Patroni requires also deploying an etcd/consul/zookeeper cluster and still requires failover). That means that Jellyfin will need to support failover, and you may lose connectivity for a bit while that happens (e.g. RDS failover can take a few minutes).

Something that might be worth thinking about since you are switching the data model, is to use something like Cassandra/ScyllaDB, etc (a KVS which supports quorum reads and writes)

[EraYaN]

Cassandra is very much not a relational database though and the data is very much relational. Besides having a cluster of that will be even more complicated that getting say a Galera cluster running.

[davispuh]

I'm also interested in this, looking for PostgreSQL support.
Currently my /var/lib/jellyfin/data/library.db is 450MB and that's only 80% of my library (it's still scanning). I think SQLite performance isn't that great for large libraries.

[Ornias1993]

This would be a huge benefid.
Just keep it standard (SQL) and nothing fancy like ScyllaDB, Cassandra etcetc...

Most of the people wanting this feature already have those running and just need something a little more flexible/performant... Going straight Cassandra Scylla would be over doing it imho.

[cameronkollwitz]

Completely agree with this request. It would be great to be able to scale with microservices.

[MrBones757]

This sounds really nice and something that i would use heavily. My use-case for this functionality is to load balance / distribute trans-code load across many hosts.

by distributed transcode, i mean if i had more than one simultaneous stream, each stream could be transcoded on different instances improving performance.

In this case a "non ha" db would be fine (i.e single instance postgres) that both instances could consume, while providing the aforementioned distributed transcode, perhaps across two smaller (less powerful) nodes like say Rpi or in a container environment (kubernetes / swarm)

[brianjmurrell]

@MrBones757 This doesn't sound like a problem that multiple databases is needed to solve. A single database could easily be used and allow distributed transcoding. I would suggest your request is a new ticket for distributed transcoding.

[dkanada]

@MrBones757 you can also look into rffmpeg for now which will kind of meet your criteria.