Realm-cocoa: After Update Realm Database not persisting data on next app session
Some of my users experience problems persisting new entries after I updated my application.
I did not update Realm, which is at version 2.4.0
I changed one of my Realm Models and increased the version number of the database.
This happens on iOS 10.2.1
I use the latest stable version of Xcode and a stable version of cocoapods.
I did not change any database logic and I'm not deleting theses entries on my own.
It seems to happen only to users migrating to the new version.
I need a very quick fix. Would just upgrading realm to 2.4.4 be enough, what about increasing the database version without any changes to the models? I read, there was a crash fixed for exactly this case.
bb-git
All 22 comments
Hey @bb-git! Thanks for reaching out. I wanted to let you know that we've seen your message and that someone will follow-up with you as soon as we can.
luna-kx
on 24 Mar 2017
Hi there @bb-git! Can you please give us some more information? Sample code would be the best.
Specifically, can you show us your model objects both before and after you updated them, as well as how you've set up the migration block? Thanks a lot!
TimOliver
on 24 Mar 2017
I can attach some code soon. But the strange thing is, the data that is lost is a realm model, that was not changed at all. Another model was changed with the update. The migration block is empty, I just increased the version number.
Am 25.03.2017 um 03:38 schrieb Tim Oliver notifications@github.com:
Hi there @bb-git! Can you please give us some more information? Sample code would be the best.
Specifically, can you show us your model objects both before and after you updated them, as well as how you've set up the migration block? Thanks a lot!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
bb-git
on 25 Mar 2017
before
class Food: Object {
dynamic var foodId = 0
dynamic var name = ""
dynamic var unit = ""
dynamic var kcal = 0
dynamic var amount:Float = 0.0
dynamic var protein:Float = 0.0
dynamic var fat:Float = 0.0
dynamic var carbs:Float = 0.0
dynamic var language = ""
dynamic var verified = false
dynamic var popularity = 0
var isInPieces: Bool { // read-only properties are automatically ignored
if (amount == 100.0 && UserData.isMetricMode()) || (amount == 1.0 && !UserData.isMetricMode()){
return false
}else{
return true
}
}
var amountFactor: Float { // read-only properties are automatically ignored
if (isInPieces){
return 1.0
}else{
return 1.0 / amount
}
}
override static func primaryKey() -> String? {
return "foodId"
}
override static func indexedProperties() -> [String] {
return ["name"]
}
}
after
class Food: Object {
dynamic var foodId = 0
dynamic var name = ""
dynamic var unit = ""
dynamic var kcal = 0
dynamic var amount:Float = 0.0
dynamic var protein:Float = 0.0
dynamic var fat:Float = 0.0
dynamic var carbs:Float = 0.0
dynamic var language = ""
dynamic var verified = false
dynamic var popularity = 0
dynamic var used = 0
dynamic var lastUsed = Date(timeIntervalSince1970: 0)
var isInPieces: Bool { // read-only properties are automatically ignored
if (amount == 100.0 && UserData.isMetricMode()) || (amount == 1.0 && !UserData.isMetricMode()){
return false
}else{
return true
}
}
var amountFactor: Float { // read-only properties are automatically ignored
if (isInPieces){
return 1.0
}else{
return 1.0 / amount
}
}
override static func primaryKey() -> String? {
return "foodId"
}
override static func indexedProperties() -> [String] {
return ["name"]
}
}
this data is lost
class History: Object {
dynamic var type = ""
dynamic var name = ""
dynamic var meal = ""
dynamic var kcal = 0
dynamic var currentSubmitDate = Date()
dynamic var timestamp = 0
dynamic var activityMinutes = 0
dynamic var foodOrSportsId = 0
dynamic var steps = 0
dynamic var priKey: String = ""
// dynamic var protein = 0.0
// dynamic var fat = 0.0
// dynamic var carbs = 0.0
override static func primaryKey() -> String? {
return "priKey"
}
override static func indexedProperties() -> [String] {
return ["type","timestamp","currentSubmitDate"]
}
}
here it's written
let timestamp: CLong = CLong(Date().timeIntervalSince1970)
let timestamp_nsts:TimeInterval = Date().timeIntervalSince1970
let preciseKey = UInt64(timestamp_nsts*1000.0)
let newHistoryEntry = History()
newHistoryEntry.currentSubmitDate = CalculationsHelper.getCurrentSubmitDate()
newHistoryEntry.kcal = kcal
newHistoryEntry.name = name
newHistoryEntry.type = type
newHistoryEntry.meal = meal
newHistoryEntry.timestamp = timestamp
newHistoryEntry.foodOrSportsId = foodSportsId
newHistoryEntry.activityMinutes = min
newHistoryEntry.priKey = type + String(preciseKey)
let realm = try! Realm()
try? realm.write {
realm.add(newHistoryEntry)
}
@bb-git The model definition doesn't seem to be problematic. It is still unclear for us what is happening. Could you tell me more details? Data lost means cleared the data on the History table? It is best to send a complete project to [email protected]. Thanks.
kishikawakatsumi
on 29 Mar 2017
Yes, the history table is cleared. People complain, that data does not stay for long and then disappears. For me it is working, so I can not reproduce it.
It is not possible to send the project, it is too big anyway. Thanks
bb-git
on 29 Mar 2017
I'm compacting the database on every launch. Could this cause the problem?
I updated to realm 2.4.4 and increased the version number of the database. Customers still report this problem.
bb-git
on 30 Mar 2017
If you're replacing the Realm file after creating a Realm instance, it could explain what you're seeing. Directly manipulating the Realm file on disk while you have it open (i.e., you have any Realm instances, or any objects retrieved from a Realm instance, alive in memory) will not do what you want. It's only safe to delete or replace the file on disk while there are no live Realm instances. Typically this means immediately on launch, prior to creating any Realm instances in your app.
bdash
on 30 Mar 2017
This is the first I do in applicationDidFinishLaunching, before accessing the database. It was working without problems before.
bb-git
on 30 Mar 2017
Are you 100% sure you're doing it prior to opening any Realms? Keep in mind that a property declared as let realm = try! Realm() on a view controller or your app delegate will be initialized prior to applicationDidFinishLaunching. You can set a breakpoint on +[RLMRealm realmWithConfiguration:error] to confirm when you first open the Realm file relative to the launch of your app.
bdash
on 30 Mar 2017
definition in class function shouldn't be a problem?
I checked and I don't do this.
That's what happens in applicationdidfinishlaunching
configureAndMigrateRealm()
//compact the database to free memory (if compacting should be done before migration, the schema version must be set before starting compacting)
if userDefaults.bool(forKey: keyCompleteSetup) && userDefaults.bool(forKey: keyUpgradeToArise2Performed){
compactRealm()
}
//this will not be used in the launch version, but is an example how to migrate for later versions
func configureAndMigrateRealm(){
//migrate the db
let migrationBlock: MigrationBlock = { migration, oldSchemaVersion in
if (oldSchemaVersion < 1) {
userDefaults.set(0, forKey: keyJoyPointsAccount)
userDefaults.set(0, forKey: keyTotalJoyPointsCollected)
userDefaults.synchronize()
}
Realm.Configuration.defaultConfiguration = Realm.Configuration(schemaVersion: 4, migrationBlock: migrationBlock)
}
func compactRealm() {
let defaultURL = Realm.Configuration.defaultConfiguration.fileURL!
let defaultParentURL = defaultURL.deletingLastPathComponent
let compactedURL = defaultParentURL().appendingPathComponent("default-compact.realm")
// autoreleasepool {
let realm = try! Realm()
try! realm.writeCopy(toFile: compactedURL)
try! FileManager.default.removeItem(at: defaultURL)
try! FileManager.default.moveItem(at: compactedURL, to: defaultURL)
// }
}
// autoreleasepool {
let realm = try! Realm()
try! realm.writeCopy(toFile: compactedURL)
try! FileManager.default.removeItem(at: defaultURL)
try! FileManager.default.moveItem(at: compactedURL, to: defaultURL)
// }
In this snippet you're opening the Realm, writing a copy of it to a new location, then removing the original path, all while the Realm is still open (before realm has been deallocated). You need to ensure that the Realm file is closed before removing it and replacing it. Given the code you shared the easiest way to ensure this would be to change the scope of the autoreleasepool to end the lifetime of the Realm instance before you remove the Realm file.
autoreleasepool {
let realm = try! Realm()
try! realm.writeCopy(toFile: compactedURL)
}
try! FileManager.default.removeItem(at: defaultURL)
try! FileManager.default.moveItem(at: compactedURL, to: defaultURL)
It'd also be worth double-checking (via the breakpoint I mentioned in an earlier comment) that you're not opening the Realm anywhere else prior to hitting this codepath.
bdash
on 30 Mar 2017
Ok, great. Thanks :). I double checked all the code now, I do not open it anywhere before. Is there a safer way to compact the database on each startup?
Am 30.03.2017 um 10:55 schrieb Mark Rowe notifications@github.com:
// autoreleasepool {
let realm = try! Realm()
try! realm.writeCopy(toFile: compactedURL)
try! FileManager.default.removeItem(at: defaultURL)
try! FileManager.default.moveItem(at: compactedURL, to: defaultURL)
// }
In this snippet you're opening the Realm, writing a copy of it to a new location, then removing it, all while the Realm is still open (before realm has been deallocated). You need to ensure that the Realm file is closed before removing it and replacing it. Given the code you shared the easiest way to ensure this would be to change the scope of the autoreleasepool to end the lifetime of the Realm instance before you remove the Realm file.autoreleasepool {
let realm = try! Realm()
try! realm.writeCopy(toFile: compactedURL)
}
try! FileManager.default.removeItem(at: defaultURL)
try! FileManager.default.moveItem(at: compactedURL, to: defaultURL)
It'd also be worth double-checking (via the breakpoint I mentioned in an earlier comment) that you're not opening the Realm anywhere else prior to hitting this codepath.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
bb-git
on 30 Mar 2017
Yes, as I outlined above you need to ensure the Realm instance you create is deallocated prior to moving the compacted copy over the original file.
bdash
on 30 Mar 2017
Ok, so i guess it is the best point really is in applicationDidFinishLaunching.
Thanks again.
Am 30.03.2017 um 11:27 schrieb Mark Rowe notifications@github.com:
Yes, as I outlined above you need to ensure the Realm instance you create is deallocated prior to moving the compacted copy over the original file.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
bb-git
on 30 Mar 2017
I talked to some of my customers and they lose sometimes only some entries, sometimes some even some days. Can this be explained by my mistake compacting the database? I do only delete entries on an explicit user action and never more than one entry.
bb-git
on 30 Mar 2017
Hi @bb-git! Good to see that @bdash was able to sort out how you were compacting your Realm files. Hopefully that'll sort the majority of your issues.
Hmm, that data loss you're experiencing is still troubling though. Compacting the database, even before when you weren't deallocating realm before performing the copy shouldn't cause data loss of specific objects.
It's possible your migration might potentially be doing that. Are you able to confirm with your customers what app version they were on, and what they were doing when they noticed the data was gone?
TimOliver
on 30 Mar 2017
The pattern of removing a replacing a Realm file while you have a live Realm instance can give the appearance of objects being deleted. In that scenario new writes to the Realm after the file is replaced may be directed to the old, now-deleted Realm file. When you relaunch and open the _new_ Realm file, the writes will appear to have been lost. In reality they were just being made against the file you deleted. This is why it's important to ensure that the Realm is not open when you're replacing the file on disk.
bdash
on 30 Mar 2017
All of them was updating the application, when they started encountering the loss of entries.
Deletion of entries seems Tonne pretty random. Some people encounter loss of whole days, some of the current day. Others just one meal or even only some entries of a meal.
So the explanation of @bdash would cover quite some cases, but not all of them. I updated to 2.5.0 and prepared an update of my app and submitted it for review. Is there sth else I can do?
bb-git
on 31 Mar 2017
@bb-git actually, @bdash's explanations above can completely explain all sorts of scenarios regarding incorrect or "lost" data. It could also lead to corrupted Realms.
The solution presented is the _only_ thing you can do to safely work with Realm files. Any file system operations on the Realm files must be done while no other Realm APIs (from any process) are accessing the file.
jpsim
on 5 Apr 2017
Deployed the new version to my customers. Everything is working for them now. Thanks for your help :)
bb-git
on 10 Apr 2017
I used a version 3.0.10 of realm Studio and that way if I recognized the data in Xcode , the problem happened with version 10 and 5 of Realm Studio in OS X 11.0.1
eduardojordan
on 15 Dec 2020
Related issues
dmorrow
·
3Comments
matteodanelli
·
3Comments
xspyhack
·
3Comments
dennisgec
·
3Comments
jpsim
·
3Comments
Most helpful comment
In this snippet you're opening the Realm, writing a copy of it to a new location, then removing the original path, all while the Realm is still open (before
realmhas been deallocated). You need to ensure that the Realm file is closed before removing it and replacing it. Given the code you shared the easiest way to ensure this would be to change the scope of theautoreleasepoolto end the lifetime of theRealminstance before you remove the Realm file.It'd also be worth double-checking (via the breakpoint I mentioned in an earlier comment) that you're not opening the Realm anywhere else prior to hitting this codepath.