Signal-android: Unable to restore backups on Signal Beta

[g2bb]

[ x] I have searched open and closed issues for duplicates ( the nearest is #9133)
[x ] I am submitting a bug report for existing functionality that does not work as intended
[ x] I have read https://github.com/signalapp/Signal-Android/wiki/Submitting-useful-bug-reports
[ x] This isn't a feature request or a discussion topic

Bug description
Signal will not restore backups on BETA. I had this issue after trying to reinstall to fix another issue where MMS messages would never download (Unfortunately I can't give any info about this since I can't reproduce it now that I'm essentially locked out of signal), making it impossible to work on my group projects. I directly copied the passphrase using Google Lens and verified the correct sequence and everything. I had to re-install (non-beta) Signal for my dad earlier today and restore a backup and it worked flawlessly, so this is something new. I believe it has to do with the fact the BETA includes spaces automatically between every 5 numbers since "feature" doesn't exist on the regular version of signal

Steps to reproduce
Be on Signal Beta
Backup & save passphrase
Uninstall Signal and reinstall it
Try to restore backup with passphrase

Device info
Device: OnePlus 6t McLaren Edition A6013
Android version: Oxygen OS 9.0.17
**Signal version: 4.49.10

Link to debug log
N/A because I can't even get that far into the app

[bepaald]

Well, since you don't provide a debug log, could you then at least describe what happens? You say it doesn't work, but what happens? Does the app crash? Does it show an error message (wrong passphrase)? Does it just start with an empty database?

I just restored a backup on 4.49.10 to test if the functionality was really broken, but it went completely fine, no problems at all.

[alan-signal]

I believe it has to do with the fact the BETA includes spaces automatically between every 5 numbers since "feature" doesn't exist on the regular version of signal

The spaces aren't really there, it's just a display change.

N/A because I can't even get that far into the app

You can tap the title of any registration page 8 times to access the debug log.

I had to re-install (non-beta) Signal for my dad earlier today and restore a backup and it worked flawlessly

What version was that?

[luiscurini]

I'm having the same issue between a Pixel 3XL and a Pixel 4 XL, both on Android 10 and app version 4.49.10. The app starts to restore the messages but then it says "incorrect Backup passphrase"

EDIT: Here's the debug log

[alan-signal]

What dates are on the files in the signal backup directory?

Nevermind, I suppose there's just the one.

[luiscurini]

Today's date for Signal and Backups and the backup file is like signal-2019-10-28-time.backup

[alan-signal]

Would you try something on your new device please?:

Install: https://signal.org/android/apk/ and try to restore with that, as that is v 4.47.

You don't need to register, just see if the backup restores.

[luiscurini]

I had already tried that but I get a message saying "Cannot import backups from newer versions of Signal"

[g2bb]

I thought I described it in my issue, but as other people have said the phone will import all the messages then an error will appear saying that the passphrase is wrong.

Like luiscurini, my backup was right before I uninstalled, which in my case was yesterday. I already tried downloading the non-beta version and it gave me the same "Cannot import backups from newer versions of Signal" error. I know the spaces aren't really there but that's the only explanation I could think of for why the system didn't work.

[alan-signal]

Related #8355

I already tried downloading the non-beta version and it gave me the same "Cannot import backups from newer versions of Signal" error.

Which version was that? And what was your fathers version?

[alan-signal]

@luiscurini what version did you make the backup on?

[luiscurini]

@alan-signal I did with 4.49.10, both of the devices are on the same version. I tried redoing the backup several times but it just doesn't work.

[g2bb]

Related #8355

I already tried downloading the non-beta version and it gave me the same "Cannot import backups from newer versions of Signal" error.

Which version was that? And what was your fathers version?

The non-beta that I downloaded and the version I downloaded for my father were both the latest non-beta version on the Play Store which is 4.48.17. I made my backup on 4.49.10, but my father's backup was made on 4.48.17 immediately prior to uninstalling/reinstalling the app

[alan-signal]

I think if you have the old device with signal on it still, I would personally try to create a new backup (just by tapping "Create backup" once more). I've not seen your logs @g2bb , but I suspect it's the the same and you do have the correct passphrase.

[g2bb]

@alan-signal As I said in my original post I was reinstalling the app on my phone because of ANOTHER issue I was having so this isn't a new phone. Again, per my original post, I used Google Lens' built in OCR to identify the text from a screenshot of the passphrase, THEN I quintuple checked (no joke) that the numbers were in the correct order. The passphrase IS correct

EDIT: I misread your last reply as you telling me that I do not have the correct passphrase.

Here is the link to my log from my fresh install, though I don't know how much use it will be:
https://debuglogs.org/335569237b3f4fe33ed3477c33e1f9adf16519c8745b824f50e626535f5c3720

[alan-signal]

Thanks, that does confirm to me you have the same issue as luiscurini at least.

[luiscurini]

Hi! I just tried just in case with 4.49.11 on both phones and still the same. This is just to inform you, no pressure.

[alan-signal]

Thanks, the fix will be in 4.50 which we hope to release to Beta this week. You are both on the beta right?

[luiscurini]

Yes, I already enrolled. Thank you, I'll write here once the version is released.

[g2bb]

Yes, I am

[g2bb]

I know you said 4.50 would be the update to fit this issue, but I would just like to confirm that this update 4.49.14 does not yet resolved the issue

[bepaald]

I have a program that might be able to give some information on the frame that gives the error. I figured this functionality of the program would be obsolete by this commit in 4.49.14, but from your comment it seems it might not be the case.

If you feel comfortable using linux, you could check out my program here: https://github.com/bepaald/signalbackup-tools. No guarantees, but it might pinpoint the message causing the problem (and, in that case, it can also fix the backup file if you don't want to wait for this issue to be resolved).

[g2bb]

This issue is still happening for me on 4.49.16

[F5KUNK]

Reporting this issue on latest beta and also latest stable build. App closes upon reaching end of messages to be restored with no information.

[greyson-signal]

You can get a debug log by tapping on any of the titles in the registration flow 8 times. Please post a log here.

[F5KUNK]

You can get a debug log by tapping on any of the titles in the registration flow 8 times. Please post a log here.

Restoring the backup (3.46GB) on a fresh install on a new device (Pixel 3a). Backup was created on a Pixel 2 running the same release.

https://debuglogs.org/ca78a479e24729f33b16c0fe4c3dcbda598f22646cb4c8096491ff130c8c53dd

[greyson-signal]

Closing as a duplicate of #9154

[g2bb]

@greyson-signal This issue is NOT resolved and this issue was opened before the "duplicate" issue that you closed this for. I was told this would be fixed last week and yet it still is not.

[g2bb]

@greyson-signal FURTHERMORE the issue you said was a duplicate is, in fact, NOT a duplicate. The issue they are having is that the app as a whole is crashing. MY issue is that the back up fails to load. The app NEVER crashes

[Cerberus0]

@F5KUNK's issue is probably being tracked in https://github.com/signalapp/Signal-Android/issues/9154. Perhaps @greyson-signal was too hasty and closed this as a duplicate of https://github.com/signalapp/Signal-Android/issues/9154 just because of F5KUNK's comment and debug log? We all make mistakes. 🤷‍♂️

@alan-signal said that the fix for this issue will be included in version 4.50, which they were _hoping_ to release last week. Things don't always turn out the way one hopes. They may have noticed more pressing issues in version 4.49 after they released it into production, delaying the release of 4.50. If they get these issues sorted out, I assume they will most likely release 4.50 into beta some time next week. 🤞

[greyson-signal]

@g2bb Please post another debuglog taken after your backup restore fails.

[g2bb]

@g2bb Please post another debuglog taken after your backup restore fails.

https://debuglogs.org/ed55ce4442700dfb30bf7ae06d7ccc85d4110ec238c00319b75a6de483e549ae

[greyson-signal]

2019-11-11 21:34:38.027 CST W RestoreBackupFragment: java.io.EOFException: Stream ended early
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at org.thoughtcrime.securesms.util.Util.readFully(Util.java:251)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at org.thoughtcrime.securesms.util.Util.readFully(Util.java:243)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at org.thoughtcrime.securesms.backup.FullBackupImporter$BackupRecordInputStream.readFrame(FullBackupImporter.java:327)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at org.thoughtcrime.securesms.backup.FullBackupImporter$BackupRecordInputStream.readFrame(FullBackupImporter.java:270)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at org.thoughtcrime.securesms.backup.FullBackupImporter.importFile(FullBackupImporter.java:79)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at org.thoughtcrime.securesms.registration.fragments.RestoreBackupFragment$2.doInBackground(RestoreBackupFragment.java:198)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at org.thoughtcrime.securesms.registration.fragments.RestoreBackupFragment$2.doInBackground(RestoreBackupFragment.java:190)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at android.os.AsyncTask$2.call(AsyncTask.java:333)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:245)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
2019-11-11 21:34:38.027 CST W RestoreBackupFragment:    at java.lang.Thread.run(Thread.java:764)

It doesn't crash, but this is a manifestation of the same bug that is tracked in #9154. i.e. the stream reading goes awry. It's just that instead of incorrectly trying to read a chunk of memory so large it crashes, it instead incorrectly tries to read a smaller chunk of memory that happens to be larger than the remainder of the file. Please track that issue for updates.

[pgerber]

Is there anything I can do to help you resolve this issue? I still have a phone I'd like to move to but I can't because I can't restore the backup.

Another debug log from a more recent version of Signal. Just in case.

[pgerber]

@greyson-signal, I think this issue should be reopened. It's apparently not resolved yet.

I have a program that might be able to give some information on the frame that gives the error. I figured this functionality of the program would be obsolete by this commit in 4.49.14, but from your comment it seems it might not be the case.

If you feel comfortable using linux, you could check out my program here: https://github.com/bepaald/signalbackup-tools. No guarantees, but it might pinpoint the message causing the problem (and, in that case, it can also fix the backup file if you don't want to wait for this issue to be resolved).

As suggested by @bepaald, I tried signalbackup-tools. This is the output I got:

[user@disp6346 signalbackup-tools]$ ./signalbackup-tools ~/QubesIncoming/disp7192/signal-2019-12-20-23-22-36.backup 498785844919826683673905069781 --output signal-2019-12-20-23-22-36.backup
signalbackup-tools source version 20191219.175337
IV: (hex:) a1 19 2f e9 1e 66 4e 3e 2c ce 88 d1 d0 c3 64 e6 (size: 16)
SALT: (hex:) 98 f0 ca d9 44 b5 fb 2f 65 2d c1 ca e1 8f 89 6a f2 4b f9 7f 8c 17 05 b6 3d 76 75 25 c7 44 ac 78 (size: 32)
BACKUPKEY: (hex:) 5f 3a cd 99 88 a8 4c fd 06 0f 12 45 ec 7a 8e 4c eb bb ae bc 4b db fc ef 34 0a 87 1c 0b 3e d4 e9 (size: 32)
CIPHERKEY: (hex:) 3e da 72 e5 e1 3f 10 db d0 13 41 20 ec 89 b8 10 27 47 33 f3 65 2f fe a3 3f 42 90 d7 a0 4b b7 f6 (size: 32)
MACKEY: (hex:) b3 43 81 56 cd 5d 01 77 69 4f 12 2f ad 8a b3 06 6c ce d4 e7 4b 74 3e 14 24 64 ad 5f 34 8b f3 b1 (size: 32)
COUNTER: 2702782441
Reading backup file...
FRAME 1069 (100.0%)... Read entire backup file...

done!

Exporting backup to 'signal-2019-12-20-23-22-36.backup'
Writing HeaderFrame...
Writing DatabaseVersionFrame...
Writing SqlStatementFrame(s)...
  Dealing with table 'sms'... 221/221 entries...done
  Dealing with table 'mms'... 230/230 entries...done
  Dealing with table 'part'... 1/259 entries...Warning: attachment data not found (rowid: 1, uniqueid: 1569527957532)
  Dealing with table 'part'... 2/259 entries...Warning: attachment data not found (rowid: 2, uniqueid: 1569527960374)
  Dealing with table 'part'... 3/259 entries...Warning: attachment data not found (rowid: 3, uniqueid: 1569527960376)
  Dealing with table 'part'... 4/259 entries...Warning: attachment data not found (rowid: 4, uniqueid: 1569527960377)
  Dealing with table 'part'... 5/259 entries...Warning: attachment data not found (rowid: 5, uniqueid: 1569527969268)
  Dealing with table 'part'... 6/259 entries...Warning: attachment data not found (rowid: 6, uniqueid: 1569527972295)
  Dealing with table 'part'... 7/259 entries...Warning: attachment data not found (rowid: 7, uniqueid: 1569527981175)
  Dealing with table 'part'... 8/259 entries...Warning: attachment data not found (rowid: 8, uniqueid: 1569527982238)
  Dealing with table 'part'... 9/259 entries...Warning: attachment data not found (rowid: 9, uniqueid: 1569527985422)
  Dealing with table 'part'... 10/259 entries...Warning: attachment data not found (rowid: 10, uniqueid: 1569527986519)
  Dealing with table 'part'... 11/259 entries...Warning: attachment data not found (rowid: 11, uniqueid: 1569527996564)
  Dealing with table 'part'... 12/259 entries...Warning: attachment data not found (rowid: 12, uniqueid: 1569527999165)
  Dealing with table 'part'... 13/259 entries...Warning: attachment data not found (rowid: 13, uniqueid: 1569528005857)
  Dealing with table 'part'... 14/259 entries...Warning: attachment data not found (rowid: 14, uniqueid: 1569528007120)
  Dealing with table 'part'... 15/259 entries...Warning: attachment data not found (rowid: 15, uniqueid: 1569528008744)
  Dealing with table 'part'... 16/259 entries...Warning: attachment data not found (rowid: 16, uniqueid: 1569528010487)
  Dealing with table 'part'... 17/259 entries...Warning: attachment data not found (rowid: 17, uniqueid: 1569528025067)
  Dealing with table 'part'... 18/259 entries...Warning: attachment data not found (rowid: 18, uniqueid: 1569528028618)
  Dealing with table 'part'... 19/259 entries...Warning: attachment data not found (rowid: 19, uniqueid: 1569528030089)
  Dealing with table 'part'... 20/259 entries...Warning: attachment data not found (rowid: 20, uniqueid: 1569528035713)
  Dealing with table 'part'... 21/259 entries...Warning: attachment data not found (rowid: 21, uniqueid: 1569528037974)
  Dealing with table 'part'... 22/259 entries...Warning: attachment data not found (rowid: 22, uniqueid: 1569528039498)
  Dealing with table 'part'... 23/259 entries...Warning: attachment data not found (rowid: 23, uniqueid: 1569528042203)
  Dealing with table 'part'... 24/259 entries...Warning: attachment data not found (rowid: 24, uniqueid: 1569528042206)
  Dealing with table 'part'... 25/259 entries...Warning: attachment data not found (rowid: 25, uniqueid: 1569528044940)
  Dealing with table 'part'... 26/259 entries...Warning: attachment data not found (rowid: 26, uniqueid: 1569528045991)
  Dealing with table 'part'... 27/259 entries...Warning: attachment data not found (rowid: 27, uniqueid: 1569528047988)
  Dealing with table 'part'... 28/259 entries...Warning: attachment data not found (rowid: 28, uniqueid: 1569528053311)
  Dealing with table 'part'... 29/259 entries...Warning: attachment data not found (rowid: 29, uniqueid: 1569528054598)
  Dealing with table 'part'... 30/259 entries...Warning: attachment data not found (rowid: 30, uniqueid: 1569528054600)
  Dealing with table 'part'... 31/259 entries...Warning: attachment data not found (rowid: 31, uniqueid: 1569528058146)
  Dealing with table 'part'... 32/259 entries...Warning: attachment data not found (rowid: 32, uniqueid: 1569528063380)
  Dealing with table 'part'... 33/259 entries...Warning: attachment data not found (rowid: 33, uniqueid: 1569528064451)
  Dealing with table 'part'... 34/259 entries...Warning: attachment data not found (rowid: 34, uniqueid: 1569528064461)
  Dealing with table 'part'... 35/259 entries...Warning: attachment data not found (rowid: 35, uniqueid: 1569528068818)
  Dealing with table 'part'... 36/259 entries...Warning: attachment data not found (rowid: 36, uniqueid: 1569528070410)
  Dealing with table 'part'... 37/259 entries...Warning: attachment data not found (rowid: 37, uniqueid: 1569528083325)
  Dealing with table 'part'... 38/259 entries...Warning: attachment data not found (rowid: 38, uniqueid: 1569528083327)
  Dealing with table 'part'... 39/259 entries...Warning: attachment data not found (rowid: 39, uniqueid: 1569528083328)
  Dealing with table 'part'... 40/259 entries...Warning: attachment data not found (rowid: 40, uniqueid: 1569528091396)
  Dealing with table 'part'... 41/259 entries...Warning: attachment data not found (rowid: 41, uniqueid: 1569528100195)
  Dealing with table 'part'... 42/259 entries...Warning: attachment data not found (rowid: 42, uniqueid: 1569528105654)
  Dealing with table 'part'... 43/259 entries...Warning: attachment data not found (rowid: 43, uniqueid: 1569528119571)
  Dealing with table 'part'... 44/259 entries...Warning: attachment data not found (rowid: 44, uniqueid: 1569528121534)
  Dealing with table 'part'... 45/259 entries...Warning: attachment data not found (rowid: 45, uniqueid: 1569528123542)
  Dealing with table 'part'... 46/259 entries...Warning: attachment data not found (rowid: 46, uniqueid: 1569528135253)
  Dealing with table 'part'... 47/259 entries...Warning: attachment data not found (rowid: 47, uniqueid: 1569528135254)
  Dealing with table 'part'... 48/259 entries...Warning: attachment data not found (rowid: 48, uniqueid: 1569528136538)
  Dealing with table 'part'... 49/259 entries...Warning: attachment data not found (rowid: 49, uniqueid: 1569528140284)
  Dealing with table 'part'... 50/259 entries...Warning: attachment data not found (rowid: 50, uniqueid: 1569528140285)
  Dealing with table 'part'... 51/259 entries...Warning: attachment data not found (rowid: 51, uniqueid: 1569528140286)
  Dealing with table 'part'... 52/259 entries...Warning: attachment data not found (rowid: 52, uniqueid: 1569528149598)
  Dealing with table 'part'... 53/259 entries...Warning: attachment data not found (rowid: 53, uniqueid: 1569528157865)
  Dealing with table 'part'... 54/259 entries...Warning: attachment data not found (rowid: 54, uniqueid: 1569528158969)
  Dealing with table 'part'... 55/259 entries...Warning: attachment data not found (rowid: 55, uniqueid: 1569528166088)
  Dealing with table 'part'... 56/259 entries...Warning: attachment data not found (rowid: 56, uniqueid: 1569528175436)
  Dealing with table 'part'... 57/259 entries...Warning: attachment data not found (rowid: 57, uniqueid: 1569528175438)
  Dealing with table 'part'... 58/259 entries...Warning: attachment data not found (rowid: 58, uniqueid: 1569528176523)
  Dealing with table 'part'... 59/259 entries...Warning: attachment data not found (rowid: 59, uniqueid: 1569528181952)
  Dealing with table 'part'... 60/259 entries...Warning: attachment data not found (rowid: 60, uniqueid: 1569528181965)
  Dealing with table 'part'... 61/259 entries...Warning: attachment data not found (rowid: 61, uniqueid: 1569528181965)
  Dealing with table 'part'... 62/259 entries...Warning: attachment data not found (rowid: 62, uniqueid: 1569528183178)
  Dealing with table 'part'... 63/259 entries...Warning: attachment data not found (rowid: 63, uniqueid: 1569528184396)
  Dealing with table 'part'... 64/259 entries...Warning: attachment data not found (rowid: 64, uniqueid: 1569528184397)
  Dealing with table 'part'... 65/259 entries...Warning: attachment data not found (rowid: 65, uniqueid: 1569528192831)
  Dealing with table 'part'... 66/259 entries...Warning: attachment data not found (rowid: 66, uniqueid: 1569528199868)
  Dealing with table 'part'... 67/259 entries...Warning: attachment data not found (rowid: 67, uniqueid: 1569528199871)
  Dealing with table 'part'... 68/259 entries...Warning: attachment data not found (rowid: 68, uniqueid: 1569528199871)
  Dealing with table 'part'... 69/259 entries...Warning: attachment data not found (rowid: 69, uniqueid: 1569528199872)
  Dealing with table 'part'... 70/259 entries...Warning: attachment data not found (rowid: 70, uniqueid: 1569528202764)
  Dealing with table 'part'... 71/259 entries...Warning: attachment data not found (rowid: 71, uniqueid: 1569528206016)
  Dealing with table 'part'... 72/259 entries...Warning: attachment data not found (rowid: 72, uniqueid: 1569528206018)
  Dealing with table 'part'... 73/259 entries...Warning: attachment data not found (rowid: 73, uniqueid: 1569528214241)
  Dealing with table 'part'... 74/259 entries...Warning: attachment data not found (rowid: 74, uniqueid: 1569528223897)
  Dealing with table 'part'... 75/259 entries...Warning: attachment data not found (rowid: 75, uniqueid: 1569528230183)
  Dealing with table 'part'... 76/259 entries...Warning: attachment data not found (rowid: 76, uniqueid: 1569528239845)
  Dealing with table 'part'... 77/259 entries...Warning: attachment data not found (rowid: 77, uniqueid: 1569528241189)
  Dealing with table 'part'... 78/259 entries...Warning: attachment data not found (rowid: 78, uniqueid: 1569528249704)
  Dealing with table 'part'... 79/259 entries...Warning: attachment data not found (rowid: 79, uniqueid: 1569528254268)
  Dealing with table 'part'... 80/259 entries...Warning: attachment data not found (rowid: 80, uniqueid: 1569528256074)
  Dealing with table 'part'... 81/259 entries...Warning: attachment data not found (rowid: 81, uniqueid: 1569528256076)
  Dealing with table 'part'... 82/259 entries...Warning: attachment data not found (rowid: 82, uniqueid: 1569528266433)
  Dealing with table 'part'... 83/259 entries...Warning: attachment data not found (rowid: 83, uniqueid: 1569528270094)
  Dealing with table 'part'... 84/259 entries...Warning: attachment data not found (rowid: 84, uniqueid: 1569528272876)
  Dealing with table 'part'... 85/259 entries...Warning: attachment data not found (rowid: 85, uniqueid: 1569528288306)
  Dealing with table 'part'... 86/259 entries...Warning: attachment data not found (rowid: 86, uniqueid: 1569528292965)
  Dealing with table 'part'... 87/259 entries...Warning: attachment data not found (rowid: 87, uniqueid: 1569528292968)
  Dealing with table 'part'... 88/259 entries...Warning: attachment data not found (rowid: 88, uniqueid: 1569528301020)
  Dealing with table 'part'... 89/259 entries...Warning: attachment data not found (rowid: 89, uniqueid: 1569528304662)
  Dealing with table 'part'... 90/259 entries...Warning: attachment data not found (rowid: 90, uniqueid: 1569528305678)
  Dealing with table 'part'... 91/259 entries...Warning: attachment data not found (rowid: 91, uniqueid: 1569528307013)
  Dealing with table 'part'... 92/259 entries...Warning: attachment data not found (rowid: 92, uniqueid: 1569528307825)
  Dealing with table 'part'... 93/259 entries...Warning: attachment data not found (rowid: 93, uniqueid: 1569528309606)
  Dealing with table 'part'... 94/259 entries...Warning: attachment data not found (rowid: 94, uniqueid: 1569528309608)
  Dealing with table 'part'... 95/259 entries...Warning: attachment data not found (rowid: 95, uniqueid: 1569528309609)
  Dealing with table 'part'... 96/259 entries...Warning: attachment data not found (rowid: 96, uniqueid: 1569528309609)
  Dealing with table 'part'... 97/259 entries...Warning: attachment data not found (rowid: 97, uniqueid: 1569528309610)
  Dealing with table 'part'... 98/259 entries...Warning: attachment data not found (rowid: 98, uniqueid: 1569528309612)
  Dealing with table 'part'... 99/259 entries...Warning: attachment data not found (rowid: 99, uniqueid: 1569528310588)
  Dealing with table 'part'... 100/259 entries...Warning: attachment data not found (rowid: 100, uniqueid: 1569528310593)
  Dealing with table 'part'... 101/259 entries...Warning: attachment data not found (rowid: 101, uniqueid: 1569528310594)
  Dealing with table 'part'... 102/259 entries...Warning: attachment data not found (rowid: 102, uniqueid: 1569528310595)
  Dealing with table 'part'... 103/259 entries...Warning: attachment data not found (rowid: 103, uniqueid: 1569528310595)
  Dealing with table 'part'... 104/259 entries...Warning: attachment data not found (rowid: 104, uniqueid: 1569528310595)
  Dealing with table 'part'... 105/259 entries...Warning: attachment data not found (rowid: 105, uniqueid: 1569528330054)
  Dealing with table 'part'... 106/259 entries...Warning: attachment data not found (rowid: 106, uniqueid: 1569528334968)
  Dealing with table 'part'... 107/259 entries...Warning: attachment data not found (rowid: 107, uniqueid: 1569528334970)
  Dealing with table 'part'... 108/259 entries...Warning: attachment data not found (rowid: 108, uniqueid: 1569528334971)
  Dealing with table 'part'... 109/259 entries...Warning: attachment data not found (rowid: 109, uniqueid: 1569528334971)
  Dealing with table 'part'... 110/259 entries...Warning: attachment data not found (rowid: 110, uniqueid: 1569528334972)
  Dealing with table 'part'... 111/259 entries...Warning: attachment data not found (rowid: 111, uniqueid: 1569528334972)
  Dealing with table 'part'... 112/259 entries...Warning: attachment data not found (rowid: 112, uniqueid: 1569528335828)
  Dealing with table 'part'... 113/259 entries...Warning: attachment data not found (rowid: 113, uniqueid: 1569528335830)
  Dealing with table 'part'... 114/259 entries...Warning: attachment data not found (rowid: 114, uniqueid: 1569528337921)
  Dealing with table 'part'... 115/259 entries...Warning: attachment data not found (rowid: 115, uniqueid: 1569528339057)
  Dealing with table 'part'... 116/259 entries...Warning: attachment data not found (rowid: 116, uniqueid: 1569528340386)
  Dealing with table 'part'... 117/259 entries...Warning: attachment data not found (rowid: 117, uniqueid: 1569528341687)
  Dealing with table 'part'... 118/259 entries...Warning: attachment data not found (rowid: 118, uniqueid: 1569528343146)
  Dealing with table 'part'... 119/259 entries...Warning: attachment data not found (rowid: 119, uniqueid: 1569528348595)
  Dealing with table 'part'... 120/259 entries...Warning: attachment data not found (rowid: 120, uniqueid: 1569528348598)
  Dealing with table 'part'... 121/259 entries...Warning: attachment data not found (rowid: 121, uniqueid: 1569528348599)
  Dealing with table 'part'... 122/259 entries...Warning: attachment data not found (rowid: 122, uniqueid: 1569528348599)
  Dealing with table 'part'... 123/259 entries...Warning: attachment data not found (rowid: 123, uniqueid: 1569528348600)
  Dealing with table 'part'... 124/259 entries...Warning: attachment data not found (rowid: 124, uniqueid: 1569528348601)
  Dealing with table 'part'... 125/259 entries...Warning: attachment data not found (rowid: 125, uniqueid: 1569528362403)
  Dealing with table 'part'... 126/259 entries...Warning: attachment data not found (rowid: 126, uniqueid: 1569528363979)
  Dealing with table 'part'... 127/259 entries...Warning: attachment data not found (rowid: 127, uniqueid: 1569528363982)
  Dealing with table 'part'... 128/259 entries...Warning: attachment data not found (rowid: 128, uniqueid: 1569528367009)
  Dealing with table 'part'... 129/259 entries...Warning: attachment data not found (rowid: 129, uniqueid: 1569528369029)
  Dealing with table 'part'... 130/259 entries...Warning: attachment data not found (rowid: 130, uniqueid: 1569528373041)
  Dealing with table 'part'... 131/259 entries...Warning: attachment data not found (rowid: 131, uniqueid: 1569528375121)
  Dealing with table 'part'... 132/259 entries...Warning: attachment data not found (rowid: 132, uniqueid: 1569528376068)
  Dealing with table 'part'... 133/259 entries...Warning: attachment data not found (rowid: 133, uniqueid: 1569528380420)
  Dealing with table 'part'... 134/259 entries...Warning: attachment data not found (rowid: 134, uniqueid: 1569528380422)
  Dealing with table 'part'... 135/259 entries...Warning: attachment data not found (rowid: 135, uniqueid: 1569528381627)
  Dealing with table 'part'... 136/259 entries...Warning: attachment data not found (rowid: 136, uniqueid: 1569528385075)
  Dealing with table 'part'... 137/259 entries...Warning: attachment data not found (rowid: 137, uniqueid: 1569528390600)
  Dealing with table 'part'... 138/259 entries...Warning: attachment data not found (rowid: 138, uniqueid: 1569528394261)
  Dealing with table 'part'... 139/259 entries...Warning: attachment data not found (rowid: 139, uniqueid: 1569528395122)
  Dealing with table 'part'... 140/259 entries...Warning: attachment data not found (rowid: 140, uniqueid: 1569528398811)
  Dealing with table 'part'... 141/259 entries...Warning: attachment data not found (rowid: 141, uniqueid: 1569528402613)
  Dealing with table 'part'... 142/259 entries...Warning: attachment data not found (rowid: 142, uniqueid: 1569528403100)
  Dealing with table 'part'... 143/259 entries...Warning: attachment data not found (rowid: 143, uniqueid: 1569528421367)
  Dealing with table 'part'... 144/259 entries...Warning: attachment data not found (rowid: 144, uniqueid: 1569528423890)
  Dealing with table 'part'... 145/259 entries...Warning: attachment data not found (rowid: 145, uniqueid: 1569528424611)
  Dealing with table 'part'... 146/259 entries...Warning: attachment data not found (rowid: 146, uniqueid: 1569528425487)
  Dealing with table 'part'... 147/259 entries...Warning: attachment data not found (rowid: 147, uniqueid: 1569528426002)
  Dealing with table 'part'... 148/259 entries...Warning: attachment data not found (rowid: 148, uniqueid: 1569528427425)
  Dealing with table 'part'... 149/259 entries...Warning: attachment data not found (rowid: 149, uniqueid: 1569528430395)
  Dealing with table 'part'... 150/259 entries...Warning: attachment data not found (rowid: 150, uniqueid: 1569528439393)
  Dealing with table 'part'... 151/259 entries...Warning: attachment data not found (rowid: 151, uniqueid: 1569528440757)
  Dealing with table 'part'... 152/259 entries...Warning: attachment data not found (rowid: 152, uniqueid: 1569528444636)
  Dealing with table 'part'... 153/259 entries...Warning: attachment data not found (rowid: 153, uniqueid: 1569528458554)
  Dealing with table 'part'... 154/259 entries...Warning: attachment data not found (rowid: 154, uniqueid: 1569528461708)
  Dealing with table 'part'... 155/259 entries...Warning: attachment data not found (rowid: 155, uniqueid: 1569528465846)
  Dealing with table 'part'... 156/259 entries...Warning: attachment data not found (rowid: 156, uniqueid: 1569528465848)
  Dealing with table 'part'... 157/259 entries...Warning: attachment data not found (rowid: 157, uniqueid: 1569528470533)
  Dealing with table 'part'... 158/259 entries...Warning: attachment data not found (rowid: 158, uniqueid: 1569528473256)
  Dealing with table 'part'... 159/259 entries...Warning: attachment data not found (rowid: 159, uniqueid: 1569528483804)
  Dealing with table 'part'... 160/259 entries...Warning: attachment data not found (rowid: 160, uniqueid: 1569528490108)
  Dealing with table 'part'... 161/259 entries...Warning: attachment data not found (rowid: 161, uniqueid: 1569528490879)
  Dealing with table 'part'... 162/259 entries...Warning: attachment data not found (rowid: 162, uniqueid: 1569528491682)
  Dealing with table 'part'... 163/259 entries...Warning: attachment data not found (rowid: 163, uniqueid: 1569528492673)
  Dealing with table 'part'... 188/259 entries...Warning: attachment data not found (rowid: 188, uniqueid: 1569528544380)
  Dealing with table 'part'... 219/259 entries...Warning: attachment data not found (rowid: 219, uniqueid: 1569528575497)
  Dealing with table 'part'... 242/259 entries...Warning: attachment data not found (rowid: 242, uniqueid: 1569528621579)
  Dealing with table 'part'... 251/259 entries...Warning: attachment data not found (rowid: 251, uniqueid: 1569527497285)
  Dealing with table 'part'... 259/259 entries...Warning: attachment data not found (rowid: 259, uniqueid: 1576880380845)
  Dealing with table 'part'... 259/259 entries...done
  Dealing with table 'thread'... 4/4 entries...done
  Dealing with table 'identities'... 4/4 entries...done
  Dealing with table 'drafts'... 0/0 entries...
  Dealing with table 'push'... 0/0 entries...
  Dealing with table 'groups'... 1/1 entries...done
  Dealing with table 'group_receipts'... 87/87 entries...done
  Dealing with table 'job_spec'... 0/0 entries...
  Dealing with table 'constraint_spec'... 0/0 entries...
  Dealing with table 'dependency_spec'... 0/0 entries...
  Dealing with table 'sticker'... 51/51 entries...done
  Dealing with table 'recipient'... 7/7 entries...done
  Dealing with table 'storage_key'... 0/0 entries...
Writing SharedPrefFrame(s)...
Writing Avatars...
Writing EndFrame...
Done!

[bepaald]

@pgerber Maybe you're having a different issue? I do believe the problem reported here (and discussed more in #9154) is fixed. Note: the new versions (with the fix) still can not restore the broken backups, but they should just export new, proper backups. So make sure you install the latest version on the old phone and create a _new_ backup to move to the new phone (if you haven't already and you still have your old phone).

As for my program: I've had multiple reports of people using it successfully to fix the backups broken by this bug (see https://github.com/signalapp/Signal-Android/issues/9154#issuecomment-555731942 for an overly complicated way to fix), but something is somehow going wrong here. It should definitely run into the same problem Signal runs into: some attachment size apparently is larger than the number of bytes left in the backup. However, from the output you posted, it shows no errors when reading the backup file. Could you tell me anything about the resulting output file? Is it the same size as the input? What happens if you try to restore it?

[pgerber]

@bepaald, thank for looking into this. You're probably right about this being a separate issue. Thus, I opened #9302. Both, the export and the import were done on a recent version of Signal (4.52.4).

However, from the output you posted, it shows no errors when reading the backup file. Could you tell me anything about the resulting output file? Is it the same size as the input? What happens if you try to restore it?

The resulting file is bit-by-bit identical to the original.

[bepaald]

I opened #9302.

Yes, that seems like the right thing to do. I hope it gets some attention soon, I'm quite curious myself.

The resulting file is bit-by-bit identical to the original.

Well, obviously I could have some major bug in my code, but I'm 99% sure this (plus the fact that no bad mac or other error occurs) means the backup file is good. The only thing I actually thought of that could explain this (apart from bugs) is what @johanw666 also mentioned in the #9302: namely a truncated file. Definitely not the one fed into my program, but maybe it gets truncated when transferring to the new phone (is some FAT filesystem used anywhere?). This is just grasping at straws, but just to be sure have you checked the size (or better yet, the hash) of the backup on the new phone after copying it over? Otherwise, let's just see what the response to #9302 is. If I think of anything in the meantime, I'll let you know.