Meilisearch: delete all documents doesn't always delete all documents

[vird]

Describe the bug
delete all documents doesn't always delete all documents\
Bug reproduced in 100% of cases with same output

To Reproduce
Steps to reproduce the behavior:

1. Get latest release wget https://github.com/meilisearch/MeiliSearch/releases/download/v0.14.1/meilisearch-linux-amd64
2. Ensure you don't have db from previous experiments rm -rf data.ms/
3. Get this script https://gist.github.com/vird/4d65689e7c6e18467f1288c8fbae1b96
4. Install dependencies npm i meilisearch
5. Start meilisearch with logging ./meilisearch-linux-amd64 2>&1 | tee meili.log
6. Run ./bug_import_reduce2.js | tee fuzz.log

Expected behavior

fuzz_cycle 0
fuzz_cycle 1
fuzz_cycle 2
fuzz_cycle 3
fuzz_cycle 4
fuzz_cycle 5
fuzz_cycle 6
fuzz_cycle 7
fuzz_cycle 8
fuzz_cycle 9
bug is not reproduced

Real behavior

fuzz_cycle 0
fuzz_cycle 1
fuzz_cycle 2
fuzz_cycle 3
fuzz_cycle 4
fuzz complete
[
  { _id: 0, text: 'test str' },
  { _id: 1, text: 'test str' },
  { _id: 2, text: 'test str' },
  { _id: 3, text: 'test str' },
  { _id: 4, text: 'test str' },
  { _id: 5, text: 'test str' },
  { _id: 6, text: 'test str' },
  { _id: 7, text: 'test str' },
  { _id: 8, text: 'test str' },
  { _id: 9, text: 'test str' },
  { _id: 10, text: 'test str' },
  { _id: 11, text: 'test str' },
  { _id: 12, text: 'test str' },
  { _id: 13, text: 'test str' },
  { _id: 14, text: 'test str' },
  { _id: 15, text: 'test str' },
  { _id: 16, text: 'test str' },
  { _id: 17, text: 'test str' },
  { _id: 18, text: 'test str' },
  { _id: 19, text: 'test str' }
]

Desktop

• OS: Ubuntu 18.04.3 LTS
• Version 0.14.1 (also reproduced with 84a3e95fa4bdfcf2b8adf458dac62127afdc6501)

Logs
see https://gist.github.com/vird/4d65689e7c6e18467f1288c8fbae1b96

• meili.log
• fuzz.log

[MarinPostma]

I don't know how meili-js sdk works exactly and @curquiza may tell me wrong, but from a core perspective, the deletion and addition requests are enqueued and processed asynchronously. Therefore, if you send a delete request, and immediately after a read request, chances are that the deletion request was not yet completed, and you're not seing it. The deleteDocument method should return an update_id, and I suggest you check that the update associated with this id is processed before reading again.

[vird]

Even after stop and relaunch this script bug is instantly reproduced (on 0 iteration). So previous deletion has enough time to be completed.
This bug is not related to timings, because it has no random in reproduction.

[MarinPostma]

You're adding a lot of documents at the same time without checking if the query has been processed.
If you add synchronization, you will see that indexing the documents actually takes time

let response = await index.deleteAllDocuments()
for (;;) {
    let update = await index.getUpdateStatus(response.updateId);
    if (update.status == "processed") {
            break
    }
    await sleep(1000)
}

checking "http://localhost:7700/indexes/bug/updates" will also demonstrate that many queries are still "enqueued" and not "processed" even after your script finishes. This is definitely a timing problem.

I'm closing this issue, feel free to re-open if you manage to observe this despite synchronization

I'm closing, but feel free to re-open if you