Bookmarks: Upgraded to 3.4.7 deleted all bookmarks

[bstin]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce

1. Deleted from 3.4.6 to 3.4.7 and now all bookmarks are gone.
2. Tested on laptop + desktop each with Brave / Firefox and same results.

Expected behavior
I can see the bookmarks in Nextcloud "Recent" page, but all folders are gone

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):
OS Linux

Server (please complete the following information):

• OS: Debian 9
• HTTP server: Apache 2
• Database: MySQL 10.1
• PHP version: v7.2
• Nextcloud version: 19.0.4
• Bookmarks app version: 3.4.7
• Activated Nextcloud Apps: [the output of running $ sudo -u www-data php occ app:list in your command line]
• Nextcloud configuration: [the output of running sudo -u www-data php occ config:list system]
• Nextcloud external user backend: [e.g. ldap, or none]

Additional context
Add any other context about the problem here.

[stevesanda]

This happened to me as well. I can see my bookmarks listed in "Recent" but the folder structure I had in place is completely missing.

[potassium1338]

Can confirm this. Is this just a displaying thing or are they bookmarks in fact missing? In the android app I still can see them...

OS: Ubuntu 20.04.1 LTS
PHP: 7.4.3
HTTP server: apache 2
Nextcloud: 19.0.3

[bstin]

Luckily I had another laptop that had not synced....I was able to put in airplane mode then export the bookmarks from Chrome. Then on I re-imported / sync into Nextcloud. Now all bookmarks are back...

I don't think the bookmarks are actually "deleted" per my original post...they in fact are accessible from "Recent" folder. I think somehow they loss their "Folder" or organizational links. Nonetheless, still a bit of a mess....

[pirovc]

My bookmarks are also gone. I can still see a list of all of them without any folder structure in the Untagged category. If I try to export them, I just get an empty .html.

[marcelklehr]

Sigh. I'm on it. The update broke the API. All data is still in the database.

[bstin]

Thank you Marcel!

On Nov 3, 2020, at 4:11 PM, Marcel Klehr [e-mail] wrote:


Sigh. I'm on it. The update broke the API. All data is still in the database.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

[marcelklehr]

The update broke the API. All data is still in the database.

Update: It turns out this is not true. The API works fine. What broke was a repair step that was supposed to remove orphaned folders, and instead removed all folders. :(

Update2: I'm sorry folks, it's probably been a few releases to many for me the last couple of days. Hopefully you have database backups. Release rollout is stopped. Bug is fixed, but I'm gonna do the next release tomorrow, when I'm more alert.

[bstin]

Well thanks for the quick update. I suppose restore from backup is only practical solution?

If the folders were recreated manually how to get the “Recent Items” back into the newly created folders?

Maybe need to first run a script to assign all orphans to some default folder than can drag-drop from there? Or is it possible in UI to drag-drop from Recent items?

On Nov 3, 2020, at 4:24 PM, Marcel Klehr [e-mail] wrote:


The update broke the API. All data is still in the database.

Update: It turns out this is not true. The API works fine. What broke was a repair step that was supposed to remove orphaned folders, and instead removed all folders. :(

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

[NiteCrwlr]

Same happened to me. All Bookmarks gone. Tried to uninstall and install the Nextcloud-App again, but now I get an error:
"Client error: GET https://github.com/nextcloud/bookmarks/releases/download/v4.0.4/bookmarks-4.0.4.tar.gz resulted in a 404 Not Found response: Not Found"
Then I found this thread here. Hmmm. I'm lucky that firefox creates backups of the bookmarks.

[daddydaycare]

Yup, me too bookmarks now gone from browser. Cant see them in app, but it says i have a count total just none listed. Glad to see it is not just me. misery loves company.

[daddydaycare]

So are we just hosed?

[bstin]

One thing you should check....is I know Firefox, but I think Chrome/Brave also creates a Bookmarks.bak file somewhere in your profile directory....may be possible to simply rename that to Bookmarks and relaunch and have your Bookmarks back....

[xxKeith20xx]

One thing you should check....is I know Firefox, but I think Chrome/Brave also creates a Bookmarks.bak file somewhere in your profile directory....may be possible to simply rename that to Bookmarks and relaunch and have your Bookmarks back....

Saved the day! I had to copy it over to another chromium browser to see it work for some reason, but I was then able to export html and then import into my browser with floccus.

[pirovc]

In Firefox I managed to restore my deleted bookmarks with: Ctrl+Shift+O -> Import and Backup -> Restore

[marcelklehr]

It turns out, only the root folders have been removed. I'm working on an update that will reinstate them along with their associated contents.

[marcelklehr]

If the folders were recreated manually how to get the “Recent Items” back into the newly created folders?

Yeah, you should be able to move the existing bookmarks into the right folders.

[marcelklehr]

So are we just hosed?

Short answer: Yes.

Long answer:

1. This was caused by a typo (:see_no_evil:).

2. I've recreated the damage -- there is no easy way to fix this with a script/repair step from my side.

3. v3.4.9 is out, which fixes the offending repair step (mainly to finally give access to a different fix that was screwed up by the typo. The new release doesn't fix the damage.)

4. I'm sorry.

[potassium1338]

Hi Marcel,
are you sure that it is now fixed?
I have installed 3.4.9. and restored a backup of the prefix_bookmarks_* tables. Although under the tags they are available the bookmarks in the root folder do not show up. Any ideas?

[bstin]

Hi @potassium1338 - I'm just a regular user, but I think trying to fix from the mysql side may be difficult....possibly broken / changed id's?

Its probably safer to fix from client side. Find a backup of bookmarks in Firefox / Chrome and restore that way via floccus

[marcelklehr]

Any ideas?

My guess would be that the backup is not old enough

[potassium1338]

Is there a way to see in the admin section /log when a specific app has been updated?

[bstin]

Not sure, but on my install the dates on the folders under /var/www/nextcloud/apps matches update time.

[daddydaycare]

No worries Marcelklehr,
It was easy as, to disable bookmarks in NC, Turn off sync in floccus, then restore a back up. Thanks for your work. And more importantly, your honesty and approach to this. Kudos!

[ChrisWiegman]

Seeing this still in 4.0.5. I disabled the plugin and deleted its tables. Reactivated and synced and all worked for about an hour. Now all bookmarks are hidden in "recent" again. Using Floccus for sync.

[altepizza]

I can still see a list of all of them without any folder structure in the Untagged category.

That's actually also working for me as well. Many of my tagged and untagged bookmarks are still there and even the API spits them out. Only the folders are missing and therefore they are not displayed in the nextcloud frontend... so I guess that I could write a short python script to move the orphans into a tmp folder

[altepizza]

Sorry for the double post, but I just created a script to get my orphaned bookmarks back into the GUI and wanted to share it with you: link
USE AT YOUR OWN RISK

[daddydaycare]

Was working for a bit but now getting floccus error
E020: Could not parse server response. Is the bookmarks app installed on your server? JSON.parse: unexpected character at line 1 column 1 of the JSON data ..

[reelsense]

Where is the official method to recover?

[marcelklehr]

@daddydaycare That usually indicates a server problem, try checking the nextcloud log

[marcelklehr]

@reelsense The official method is restoring a database backup.

[daddydaycare]

@daddydaycare That usually indicates a server problem, try checking the nextcloud log

I opted to uninstall this and floccus. Will stick with Firefox sync for now. unfortunately hit my time limit to troubleshoot this. Will check again in a few months as a cool idea to keep in NC. It does the same thing on NC 20 and app 4.0.5

[ChrisWiegman]

Do we have any updates here? I've had to go back to Firefox sync for now as this has been so thoroughly broken.

[marcelklehr]

I'm sorry for the long delay here, I had to take a break the last couple of weeks due to personal reasons, which I realize was unfortunate for everyone affected by this but not possible to avoid.

The fix for the bug that caused this has been released as part of v4.0.5 and 3.4.9 as far as I can tell.

I'm available to help troubleshoot any outstanding issues that resulted from this over at https://gitter.im/nextcloud-bookmarks/community or at https://cloud.nextcloud.com/call/u52jcby9

[marcelklehr]

Together with a user I worked out the following approach to getting your bookmarks app back on track, if you've been hit by this issue.

The plan is as follows:

1. get your current set of bookmarks out of the app
2. drop all bookmarks related database tables
3. recover all bookmarks related database tables from your backup
4. get the new bookmarks from the beginning back in

0.

Make a backup of your current nextcloud database.

1. get your current set of bookmarks out of the app

Run the following mysqldump command to create a backup of your current set of bookmarks (without folders).

mysqldump --no-create-info --insert-ignore --single-transaction db_name oc_bookmarks > newer_bookmarks.sql

2. drop all bookmarks related database tables

In your mysql shell run the following to remove all bookmarks related data from the datbase.

drop table oc_bookmarks; drop table oc_bookmarks_tags; drop table oc_bookmarks_folders; drop table oc_bookmarks_folders_public; drop table oc_bookmarks_root_folders; drop table oc_bookmarks_shared_to_shares; drop table oc_bookmarks_shares; drop table oc_bookmarks_shared_folders; drop table oc_bookmarks_tree;

3. recover all bookmarks related database tables from your backup

Assuming you have your nextcloud db backup in nextcloud-db-backup.sql, we are extracting the bookmarks related tables from that file into a new file:

sed -n -e '/CREATE TABLE.*`oc_bookmarks.*`/,/Table structure for table/p' nextcloud-db-backup.sql > old_bookmarks_tables.sql

Then we're copying the comment headers from the backup file over to the start of olc_bookmarks_tables.sql to make it work.

Now, we're importing those tables into the database:

mysql nextcloud_db_name < old_bookmarks_tables.sql

(nextcloud_db_name is the name of your database.)

Update: If you run into errors at this point, this may solve your problems: https://stackoverflow.com/questions/29112716/mysql-error-1231-42000variable-character-set-client-cant-be-set-to-the-val#29115459

4. get the new bookmarks from the beginning back in

This will re-insert any newer bookmarks that did not yet exist when the backup was done.

mysql nextcloud_db_name < newer_bookmarks.sql

Done.

Now you should be able to see your folder structure in the bookmarks app again. Any bookmarks created after the folders were destroyed should be available in 'Recent bookmarks'. You can now move them to a folder of your choosing.

I'm closing this for now. If you have troubles with this or anything else, feel free to comment here, open a new issue, or contact me at https://gitter.im/nextcloud-bookmarks/community or over at https://cloud.nextcloud.com/call/u52jcby9