Forem: Deleted posts still appear in the reading list

[rhymes]

Describe the bug

A deleted post appears in the reading list and I can't remove it manually.

To Reproduce
Steps to reproduce the behavior:

A DEV user creates a post, someone else adds it to their reading list. The post's author deletes the post. The owner of the reading list still sees the post listed.

Expected behavior

A deleted post should be removed from the reading list.

Desktop (please complete the following information):

• OS: macOS
• Browser: Firefox
• Version: 65

Additional context

It might also be related to https://github.com/thepracticaldev/dev.to/issues/1061

[aspittel]

Thanks for catching this @rhymes!

[benhalpern]

There must be a logical issue with our removal from Algolia upon deletion. I have removed this article in question and will look to hunt down the bug.

We'll also do an index sweep asap to remove any other posts that might still exist after being deleted.

[benhalpern]

This is the general problem area if it is indeed a logical issue:

Other possible candidate: We may have had some memory bloat issues with the environment our async jobs were running in, causing occasional failures. It's one possible issue.

PS @rhymes I removed the link/screenshot from your issue just to help that article get forgotten as the author would have intended via deletion, now that we are aware of the issue.

[rhymes]

Thanks for editing my issue @benhalpern and for removing it from my reading list

My first thought was that maybe the order of the operations gets mixed up in some corner cases.

I'm not sure I understand the logic intrigger_delayed_index because it can both index or remove from an index, like a toggle. I noticed also that there is a before_destroy action that outright removes the object.

Is there a case when the before destroy removes the article from the index, then trigger_delayed_index runs and puts the object back in the index before the object is "phisically" removed from the DB? 🧐

[lightalloy]

It seems like trigger_delayed_index is not useful in case of destroying. trigger_delayed_index runs after_destroy, so this check record.&persisted? will always be false and record.delay.remove_from_index! won't be executed. An article is actually removed from the index in before_destroy_actions (remove_algolia_index).
So I would specify an option auto_remove: false on algoliasearch and remove part of the trigger_delayed_index method to remove the confusion.

[lightalloy]

I've found a couple of problems related to this issue:

• The cache key only depends on the user, so when an article from the reading list is destroyed, the cached_ids_of_articles won't be updated.
• Same when an article goes unpublished
• The reading list doesn't check if articles are published

Ideally, these issues shouldn't affect the list of the articles, cause they should be removed from the Algolia index when they get unpublished or destroyed.
But still it would be more correct to update cache and filter by published in my opinion, moreover, in some cases, reindexing is done asynchronously and there could possibly be issues like memory bloat like Ben said.

@benhalpern Do you think these issues worth fixing?

[rhymes]

After Ben's amazing changes to the reading list in https://github.com/thepracticaldev/dev.to/pull/2695 - now I can see the three deleted posts that have been in my "reading list" until now:

I can't remove them because they are deleted from the website, but they are now appearing where until now they had disappeared (I've removed the titles and authors)

[benhalpern]

Ah! I didn't realize this would be the case with the new approach, but this is much easier to debug with the new way we're doing this. The old way we did this was really hard to track down, but I'll get this one worked out once and for all soon.

[jessleenyc]

I believe @benhalpern knows what's going on but we're waiting to fix because it'll require running a long script.

[rhymes]

@benhalpern they are still there :D

[benhalpern]

@rhymes I'll DM you and try to get to the bottom of this.

[sandordargo]

I have the same issue. I cannot remove this from my reading list:
https://dev.to/theobendixson/how-to-safeguard-your-productive-time-by-batching-bullshit-2p27

[jessleenyc]

Just a note that someone else is experiencing this issue as documented here: https://github.com/thepracticaldev/dev.to/issues/3677

Specifically, the deleted posts can't be removed by archiving either.

[rgoyal05]

Trying to understand if this has been resolved or someone is working on it?

[maestromac]

Hello @rgoyal05 this has not been resolved yet

[mstruve]

This has been resolved with the move to Elasticsearch

[sandordargo]

So how can we remove those? Here is an example: https://dev.to/theobendixson/communism-for-you-capitalism-for-me-3j2k

[mstruve]

Looks like we have some data cleanup that needs to happen. Your account should be all set though!

[mstruve]

Data all cleaned up!

[rhymes]

@mstruve this is still happening to me as well. I think I have 3 or 4 deleted posts in my reading list.

[sandordargo]

Data all cleaned up!

Sorry, but I don't see it cleaned up, still the same for multiple articles.

[mstruve]

@rhymes I did all unpublished articles, I will do another run for deleted articles

[mstruve]

@sandordargo can you check now and tell me if the count is correct?

[sandordargo]

The deleted articles are not on my list anymore and the count also decreased!
Thanks a lot!

[mstruve]

That was only a year+ in the making 😅 better late than never!

[rhymes]

Thank you a lot @mstruve!