Litedb: Performance comparison

[stefandevo]

Hi,
Came across this repo while looking for a good x-platform alternative to Sqlite. The 1 million dollar questions is, how well does it perform compared to Sqlite. Do you have any benchmarks?

[mbdavid]

Hi @stefandevo, it's hard to compare two diferent kind of databases using different tecnologies. When I starts LiteDB best performance ever was my goal. I saves any possible byte, any possible loop. Compares many way of do same thing. I do a performance test with MongoDB in another repo (https://github.com/mbdavid/LiteDB-Benchmark).

But, after a while, I re-think this and change my focus: today imy goal is keep a most compreensive source code, be more safe as possible, cool features and mobile support.

I beleave LiteDB is pretty fast, fast enouth do use as standard applications, websites or mobile. Indexed queries are amazing fast (with a super-simple implementation). Maybe in next major version I do a compare with others, in features vs performances :+1:

[nerai]

@stefandevo I can only speak from limited personal experience. I used several LiteDB and System.Data.Sqlite databases in the 10 - 100 GB range without table references and few indices (PK and 1-2 more). Use case was 95% lookup, 4% insert, 1% delete. Obviously, everything was done in bulk in a transaction. At most two processes (mixed reader/writer). WAL active and no lessened integrity options. Sometimes on a HDD, sometimes on an SSD.

My experience was that LiteDB was consistently faster, sometimes by a large factor. (I don't have the notes at hand, so I can't give numbers.) In addition, it was also easier to set up and use.

Of course this is an expected result, because SQLite offers a lot more than LiteDB does (it is a full relational DB after all), so I'm not trying to blame SQLite here.

@mbdavid I believe you found a good compromise there.

[UweKeim]

I did some naive tests using the example in the Getting Started wiki to see how LiteDB compares to MS Access (mdb).

My test results:

Access of 1000 WRITE iterations took 00:00:37.2324932.
LiteDB of 1000 WRITE iterations took 00:00:10.5506745.

Access of 1000 READ  iterations took 00:00:35.6917453.
LiteDB of 1000 READ  iterations took 00:00:00.0082642.

So for my simple test/use case LiteDB is considerably faster than Microsoft Access.

Maybe @mbdavid can give a quick 👍 / 👎 feedback on whether my test results are something that looks reasonable?

[MercedeX]

I had an extremely bad experience. LiteDB was only able to save 10000 records in 512 seconds with default settings. The record was only 4 5 bytes in length. Does anyone have any idea how to increase speed?

[nerai]

@MercedeX That is ridiculously poor performance. You probably want to create a ticket with a minimal working example exhibiting the bad behavior and a description of your environment. Likely there is a simple configuration mistake.

[mbdavid]

Hi @MercedeX, can you paste your code to test? I'm doing a lot of changes in LiteDB in dev branch and including a set of basics examples to test speed in insert/update/delete/ensureindex. Thats the results about first tests (for 10.000 documents). I'm using an old 2012MacBook i5, 8Gb, SSD - Win10.

Insert time: 700ms
EnsureIndex time: 570ms
Update time: 1401ms
Delete time: 352ms

https://github.com/mbdavid/LiteDB/blob/dev/LiteDB.Tests/Engine/PerformanceTest.cs

[MercedeX]

Created a new issue. It should be the latest one in LiteDB. By the way your test makes all optimizations possible.

• It uses yield. It is only possible if the records are coming from a source whore output is predictable,
• You know in advance how many records would there be so you can create fixed sized batches.
• It uses transaction because it can predict the number of records.

here is what I recommend you:

• get rid of SSD. Nobody is going to get an SSD just for using LiteDB.
• Remove the predictability, toss the yield keyword out of the window
• get rid of transaction. even if you leave it there, the batches should be highly unpredictable. i.e. a batch of 1 record, then a batch of 10 records, then a batch of 2 records, then a batch of 1 record.

Follow what I wrote and run your test again .... my my ! you are in for a BIG surprise

[vip32]

@mbdavid small note: insert test is polluted because of the creating documents to insert.

ti.Start();
db.Insert("col", GetDocs(N1));  <<< place this before ti.Start()
db.Commit();
ti.Stop();

[mbdavid]

Hi @vip32, I will change this to see if there is any change.... but I made same test in sqlite, so both examples are spend this same time (in average)

[vip32]

if it is used for comparison than it won't matter (same baseline). to get insight in raw perf doc creation should not be part of it.
maybe it makes sense to create some better perf tests some day (or accept PR), so we can also compare with other storage systems (mongo for example). maybe some generic tests exist we can implement on top of litedb. but please spend your precious time on v3 now :)

[mbdavid]

Hi @vip32, PR are welcome! More complete test must be done to test all LiteDB features... but you right, now I complete focused in release v3 beta as soon as possible

[Jacknq]

Leave transactions there, its cool feature, just make switch for those other guys..