Dexie.js: Safari + where + modify = only one updated

[nicolashenry]

I am doing somthing like this in an async function :

console.log((await table.where('objectProperty').equals(myValue).toArray()).length);
console.log(await table.where('objectProperty').equals(myValue).modify((object) => { object.anotherProperty = 'test'; }));

I have 3 objects matching 'myValue' for 'objectProperty'.
Chrome displays:

3
3

Safari displays:

3
1

So it modifies only one entry on Safari (v11 with Dexie 2.0.1). I tried multiple changes on my code to make it work, the only thing which works was to loop on each array value and update each entry one by one.

[dfahlander]

Thanks for the finding. Some questions just to nail it down:

• Are you using binary keys (ArrayBuffer, etc)?
• Are you using CRUD hooks?

[dfahlander]

Also, is Safari 11 in beta still? Do you see the same issue in Safari 10?

[nicolashenry]

I am using a number key and I am using CRUD hooks.

[nicolashenry]

I think Safari 11 is not in beta anymore, I never installed a beta version as far as I remember.

[nicolashenry]

Note that it also happens on iOS Safari v11.

[nicolashenry]

I tried to not use CRUD hooks but it does not seems to change anything. I think I can't downgrade Safari to version 10 sadly.

[dfahlander]

Ok thanks. I'll try repro it on Safari 10 later this evening.

[dfahlander]

Confirmed there is an issue with Safari (10 and 11). If iterating an index (Dexie where()) the object being iterated must never be modified. This does not apply when iterating an entire object store (db.table.toCollection().modify()) - just when iterating an index (db.table.where(...).modify()).

We could work around this issue in Dexie by modifying the items first when the iteration has completed, but it would be be a larger effort and could possibly affect the behavior of Colletion.modify() in some rare use cases.

For now, we need to announce this and propose a workaround.

[dfahlander]

Proposed workaround:

async function modify (collection, modifyerFn) {
  const objectsToModify = await collection.toArray();
  objectsToModify.forEach(obj => {
    modifyerFn(obj);
  });
  await table.bulkPut(objectsToModify);
  return objectsToModify.length;
}


// Example use:
const numModified = await db.transaction('rw', db.friends, ()=>modify(db.friends.where('age').above(64),
    friend => friend.retired = true
));

[dfahlander]

Filed in webkit buzilla: https://bugs.webkit.org/show_bug.cgi?id=178380

[nicolashenry]

The workaround seems to work for now, thanks!

[dfahlander]

I think we will need to fix a Safari-workaround for this issue in Dexie as Collection.modify() is a quite central part of the library. I'll see what I could do about it.

[jokamax]

Hi,
Horrible bug...
It concerned Chrome under IOS 11 too (webkit) ? It seem touch some users on my side with this configuration (not sure).

Do you think to integrate a workaround soon directly in dexie or you advise to use the workaround above (many and many modify in my application - hierarchical menu offline ;) )?

Thanks a lot,

Jonathan

[dfahlander]

Yes it's sad because in all other aspects Safari has got an almost a perfect IndexedDB implementation. My goal now is to implement a workaround as soon as possible, but can't promise when it will be released. If you do a workaround similar to the one mentioned in this issue, it should be almost identical to the workaround I'm planning to do with one caveat: If your Collection targets thousands of large objects, the workaround could suffer from large memory consumption. The final workaround that will be implemented in Dexie.js will try to avoid large memory consumption in those cases (by splitting into chunks). Otherwise, it will be almost identical.

[jokamax]

Thanks for the quick reply.
I have to think about it...
(for the moment it concern around 20 users on 500, but I have to change the remote MySql copy every day by hand... Maybe I will ask them to use firefox for the moment)

[jokamax]

Hi (again),

I'm not a specialist of ES7/8 but it there a way to write a proposal workaround without async/await (Safari 9, Chome < 50 et FF < 50 compatible) ?

Thanks,

Jonathan

[nicolashenry]

You can try this:

function modify(collection, modifyerFn) {
  return collection.toArray()
  .then(function (objectsToModify) {
    objectsToModify.forEach(function(obj) {
      modifyerFn(obj);
    });
    return table.bulkPut(objectsToModify)
    .then(function() {
      return objectsToModify.length;
    });
  });
}

[jokamax]

It works like a charm !

Question :
Collection.modify(function () {delete this.value;});

Is this case of the modify is impacted (partial delete) ?
I haven't Safari, then I can't test :'(

Exemple :

  draft.where('pj_id').equals(pj_id).and(function (draftFilter) {
    return (draftFilter.bg >= targetItem.bg && draftFilter.bd <= targetItem.bd);
  })
  .modify(function(value) {              
    if(value.df_id) {
     trashed_sync.add({pj_id: pj_id, df_id: value.df_id, revsync:value.revsync});
    }
    //-- Remove elt     
    delete this.value;
  })

[dfahlander]

There's another workaround that enables the use of Collection.modify().

table.where(...).primaryKeys().then(function (keys) {
    return table.where(':id').anyOf(keys).modify (...)
} 

This version is not as fast as the other workaround but it should support using existing callbacks to modify. I've not tested it though but if someone with a Mac could verify it that would be awesome ;)

[dfahlander]

@beidson Just want to ping you again for an update of new Safari-related issue and found that two of them stands out more than the others (imho); this one (#594) but also #656. Just for your info. The webkit bug report for this one is still at state NEW. Would be awesome if you could have a look ;)

Kind regards, David

[jokamax]

@dfahlander just a question.
This bug concern "modify".
No problem with Table.Where(toto: toto).delete() ? I ask because in the documention delete can be written with modify.

Thanks by advance,

Jonathan

[dfahlander]

delete() does not use modify anymore, so no, it should not be affected.

[jokamax]

Thanks a lot

[firien]

Safari Technical Preview 69 has a fix for https://bugs.webkit.org/show_bug.cgi?id=178380

Their release notes reference https://bugs.webkit.org/show_bug.cgi?id=190917

[dfahlander]

That's good news. Dexie update: Latest alpha version of Dexie ([email protected]) works around the Safari issue already in it's implementation of Collection.modify(). Maybe late in the game then... But anyway I hope the redesign both improves performance and works around the issue for Collection.modify.

[jokamax]

Good news but @dfahlander dexie solution is the best because : how to work with user with old safari version (old... v11 not so old)

[jokamax]

@dfahlander Excuse me I just want to be sure :
Since [email protected] ==> I haven't to use "cheat" function modify by myself anymore ? A pseudo patch inside this alpha ?

[dfahlander]

@jokamax Yes, Collection.modify() works as expected on Safari 10+ when using dexie@^3.0.0-alpha.5. To be certain, I just now enabled an old remarked unit test that triggered the error. Manually verified that it's actually solved (tested with Safari 10 and Safari 12). Let's see also that the automated tests pass in a few minutes.

[jokamax]

@dfahlander Just a big big thanks !