Ombi: Duplicate Notifications

[ronso21]

Describe the bug
Ombi is sending multiple duplicate notifications for content via email and Pushover

To Reproduce
Steps to reproduce the behavior:
Ombi is sending the notifications several times a day.

Expected behavior
Send only one completed email.

Screenshots

Logs (Logs directory where Ombi is located)

Desktop (please complete the following information):

• OS: Win10

Ombi Version (please complete the following information):

• Version 3.0.4540
  
  
  
  • Media Server Plex
  
  

Additional context
Add any other context about the problem here.

[tidusjar]

logs?

[ronso21]

log-20190601.txt
log-20190602.txt
log-20190531.txt

[ronso21]

Issue continues after update to latest beta release - 3.04616
log-20190606.txt
log-20190607.txt
log-20190605.txt

[tidusjar]

Is it possible there are users with the same email address in ombi?

[ronso21]

No but even if there were it would not make sense that the same notification is sent several times a day over several days to both email and Pushover.

From: Jamie [mailto:[email protected]]
Sent: Saturday, June 8, 2019 6:31 AM
To: tidusjar/Ombi [e-mail]
Cc: ronso21 [e-mail]; Author [e-mail]
Subject: Re: [tidusjar/Ombi] Duplicate Notifications (#2998)

Is it possible there are users with the same email address in ombi?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/tidusjar/Ombi/issues/2998?email_source=notifications&email_token=ALPS7457W4HLIKDV2UWPZPDPZOKADA5CNFSM4HSXRJF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXHS44Y#issuecomment-500117107 , or mute the thread https://github.com/notifications/unsubscribe-auth/ALPS746BILRGHFLE7XBDOQTPZOKADANCNFSM4HSXRJFQ . https://github.com/notifications/beacon/ALPS7425IBKJFK7UEIUXRL3PZOKADA5CNFSM4HSXRJF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXHS44Y.gif

[tidusjar]

The problem is I cannot reproduce this issue

[tidusjar]

Can you try build 3.0.4616? Develop

[ronso21]

I am currently running that build and the issue has persisted.

From: Jamie [mailto:[email protected]]
Sent: Saturday, June 8, 2019 7:55 AM
To: tidusjar/Ombi [e-mail]
Cc: ronso21 [e-mail]; Author [e-mail]
Subject: Re: [tidusjar/Ombi] Duplicate Notifications (#2998)

Can you try build 3.0.4616? Develop

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/tidusjar/Ombi/issues/2998?email_source=notifications&email_token=ALPS742FNPSRISA7BIFVTXTPZOTZDA5CNFSM4HSXRJF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXHUDYA#issuecomment-500122080 , or mute the thread https://github.com/notifications/unsubscribe-auth/ALPS744HYQVCNWOLBZWR6MDPZOTZDANCNFSM4HSXRJFQ . https://github.com/notifications/beacon/ALPS742JEYK4KVX4OYWXG7DPZOTZDA5CNFSM4HSXRJF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXHUDYA.gif

[tidusjar]

Ok,

Can you please create me a local user (admin) on your server and send me your Ombi.db, OmbiExternal.db and OmbiSettings.db to [email protected] so I can attempt to reproduce it

[jeffkeller87]

I can confirm this issue on my installation.

[simonk83]

I'm also getting this as well

[schaef87]

I am having this issue as well. I sent my .db files to you as well.

[seriouslyharmed]

I see the same thing happening with my discord notifications.
The number of notifications are increasing too. My guess would be that it doesn't properly remove notifications from a queue after sending them.

[CaptAintHere]

I am having this issue too, but only on email notifications, not Pushover. I'm using Gmail as SMTP server. Emails are sent twice at the same time.

[thu3n]

Same here, using outlook as smtp. And this begun when i updated to 3.0.4680.

Note! This is only in e-mail notification, not app-notification

[schaef87]

Here is a small paste with info from my console. I imported about a dozen items and was getting a lot of notifications.

My issue does include duplicate app notifications as well.

https://pastebin.com/GtMLntSr

[eagleclawls]

Same issue .4680 I get emails and app notifications every 5th minute of the hour from and old request that just filled overnight, woke up to 50+notifications for the same movie request.

[tidusjar]

Can people people take a look through their log files and include them.

On Sat, 10 Aug 2019, 12:35 pm eagleclawls, [e-mail] wrote:

Same issue .4680 I get emails and app notifications every 5th minute of
the hour from and old request that just filled overnight, woke up to
50+notifications for the same movie request.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tidusjar/Ombi/issues/2998?email_source=notifications&email_token=ABSVULGZ6T36VVOWMMQRYPLQD2RWVA5CNFSM4HSXRJF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4ALY2I#issuecomment-520141929,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABSVULGHRJMGY7TOKDJEU4TQD2RWVANCNFSM4HSXRJFQ
.

[eagleclawls]

Would love to attach logs and been digging through internet on how to get a copy of my logs file as im using the .4680 repack for qnap from qnapclub store (i know its not official, my only option). And am having a hell of a time seeing any files. Was able to access /share/CACHEDEV1_DATA/.qpkg/Ombi3/OmbiV3 but there is absolutely nothing in it. Dont know where to go from here. Not super familiar with digging around qnap file system as its been pretty plug and play since i got it, and the file system seems pretty locked down.

[crobibero]

Yesterday Ombi sent me 150+ emails, all from the same 10 movies it found.
I don't see anything interesting in the logfile but I've attached it anyway
log-20190818.txt

[tidusjar]

I am still unable to reproduce this and it's really bugging me. Can you please send me your ombi db and ombi external dB? To [email protected]

[tidusjar]

Can someone try latest develop?

[ronso21]

Just installed latest develop version – 3.0.4729 and added several requests.

Will see if the issue persists.

From: Jamie [mailto:[email protected]]
Sent: Tuesday, August 20, 2019 3:11 AM
To: tidusjar/Ombi [e-mail]
Cc: ronso21 [e-mail]; Author [e-mail]
Subject: Re: [tidusjar/Ombi] Duplicate Notifications (#2998)

Can someone try latest develop?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/tidusjar/Ombi/issues/2998?email_source=notifications&email_token=ALPS742EA6Y5UFDOMTAYFOLQFORJ5A5CNFSM4HSXRJF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4VONZY#issuecomment-522905319 , or mute the thread https://github.com/notifications/unsubscribe-auth/ALPS74YWFDBUC7GZFW4VMVDQFORJ5ANCNFSM4HSXRJFQ . https://github.com/notifications/beacon/ALPS743Y5FVY6AK6KEWEVLDQFORJ5A5CNFSM4HSXRJF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4VONZY.gif

[ronso21]

The latest develop version seems to have fixed the issue.

On Tue, Aug 20, 2019 at 6:43 AM Robert [e-mail] wrote:

Just installed latest develop version – 3.0.4729 and added several
requests.

Will see if the issue persists.

From: Jamie [mailto:[email protected]]
Sent: Tuesday, August 20, 2019 3:11 AM
To: tidusjar/Ombi [e-mail]
Cc: ronso21 [e-mail]; Author [e-mail]
Subject: Re: [tidusjar/Ombi] Duplicate Notifications (#2998)

Can someone try latest develop?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tidusjar/Ombi/issues/2998?email_source=notifications&email_token=ALPS742EA6Y5UFDOMTAYFOLQFORJ5A5CNFSM4HSXRJF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4VONZY#issuecomment-522905319,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALPS74YWFDBUC7GZFW4VMVDQFORJ5ANCNFSM4HSXRJFQ
.

[jeffkeller87]

My hypothesis for this is that the affect requests were ones that were made during the time period where the database locking issue was prevalent a few months back. Because since then, I've had newer requests be filled and only a single notification would be sent, as desired.

Then when multiple notifications were sent, I noticed that they all seemed to have been requests that were initially made during that database locking period.

So for me, the frequency of duplicate notifications has been dropping over time. I know this doesn't include any hard data, but it was something I observed.

I am still on 3.0.4727.

[nonissue]

@jeffkeller87 That's a decent hypothesis. I'm getting inundated with notifications, but it's not all newly available requests, there seems to be a 'group' of requests that just get repeated over and over. I've been getting hundreds of emails over the last week or two.

I'm running 3.0.4680 and can't run develop since I'm using a docker image. Disabled notifications for now, but once the fix hits the master branch, I can reenable them / test if that would help @tidusjar.

[tidusjar]

There has been a report on discord that this is still an issue in develop so needs further investigation. Need to add some more logging around the Availability Checker

[northtowerdesigns]

@nonissue, Same setup and version here. Only one title though, recent, and was requested on this version. Thanks, @tidusjar for your work on this.

[ZuluWhiskey]

Same issue for me too, getting this with email and Ombi app notifications. Again, not for every request I make but for random requests which then randomly stop notifying me after a while. The issue persists with Ombi restarts etc.

[Fiala06]

Ah so I’m not the only one :) I’m just getting dupe app notifications on iOS. Sometimes more then 4 of the same movie

[dawnerd]

Same here, thinking it might be when a new version (of a movie) is downloaded.

[Lownin]

I have this issue too. Any updates?

[tidusjar]

Hi sorry but there has been no further progress on this yet.

Please remember that I have a full time (quite demanding) job, a toddler, a wife and a puppy.
All of my limited free time is usually spent either developing Ombi, fixing bugs on Ombi or providing support with Ombi, keeping in mind that no one is paying me to do this.

Ombi is currently a 1 person development team.

So if I have not yet got around to this you now know why 😄

Also there is the fact that I am still unable to reproduce this issue. If someone could help with more investigation that would be great. e.g.

What notification agents is this affecting?
When this happens does it correlate with a specific background job?
Does the media item actually get marked as available in Ombi? e.g. Has Plex/Emby URL

[Lownin]

No worries here @tidusjar Your efforts are appreciated and even with this issue, Ombi makes my life a lot easier.

What notification agents is this affecting?

I use both email and the Discord notifier. Both are affected, and both send out duplicates at the same time, though I've had email disabled for a couple updates as to not spam people so I don't know if that's changed. Let me know if you want to re-enable.

I will check on your other two questions soon and respond here. I'm also happy to provide db and access if that helps.

[tidusjar]

No that's fine you can leave email off.

Yeah the key part is what job correlates to it being sent and if it's actually marked as available.

My suspicion is that somehow one of the jobs is being triggered more than once at the same time so they all think it's unmonitored, send the notification then mark as monitored. But I cannot repro.

[Lownin]

It seems to correlate with a job that occurs on the half-hour.
Example:
A title was requested on Sunday at 12:34 PM
A notification went out that it was fulfilled Monday at 1:30PM, again at 11PM, Tuesday at 12:10AM, again (twice) at 2:01AM, Wednesday at 10AM and then stopped.

I do not know 100% if the titles are showing as available in ombi at the time they are getting duplicate notifications. I will try to figure that out. I do know that Tautulli shows the titles "recently downloaded" near the beginning of when ombi sends the first notification for a title.

I'm not sure how to correlate the job. I have what I think is the default job schedule.

[jeffkeller87]

FWIW, I haven't seen this issue repeat since my last comment back in August, and none of my users have reported it since either. I am currently on 3.0.4809.

When it was occurring, it was for email notifications, and would occur on the hour or a minute past. I did not have any other notification agents set up. I am pretty sure I had the default cron settings as well back then.

I did add a job to restart the ombi service every hour (for other reasons), so this may have cleared things up. Sorry I can't be more helpful.

[Lownin]

Confirmed: a movie was acquired, Ombi reported in the GUI as "available" right away but notifications went out at 8:30pm, 9pm, 10:30pm, and 2am. The movie was not re-acquired/updated in that time.

I am running the docker container linuxserver/ombi and my version currently displays as 3.0.4680

[tidusjar]

Did the movie have the Plex URL against it in the UI?

[Lownin]

The "View On Plex" button with a correct URL?

I did not check. I will check that next time I see duplicates.

It does currently have a correct URL, but it stopped sending duplicates.

[Lownin]

Scratch that; I just got another notification for the same movie again, and that is after confirming a valid Plex URL was there in Ombi.

[tidusjar]

That is very interesting, basically what that means is that it's marked as available in the database, so in-fact the query that finds everything that is not yet available cannot be working correctly. Thanks, this is starting to narrow down the search

[Lownin]

You're welcome. Let me know if you want any of my logs or databases or other files.

[tidusjar]

Yeah can you please send me your Ombi.db, OmbiExternal.db to [email protected] with the name of the movie/tv show title that you are getting notifications for?

[Lownin]

Sent

[tidusjar]

Think I've finally cracked it with @Lownin 's help, somehow I didn't notice this before.

What seems to be happening is once the Plex jobs run, it will kick off two jobs (Depending on what has happened).

1. Availability checker
2. Metadata refresh

The availability checker grabs the unavailable entities, the metadata refresh grabs data that has missing metadata (imdb id etc), keep in mind these jobs are running in parallel.

So the availability checker is marking things as available, whilst the metadata refresh is updating the imdb etc and things, the metadata job to keep it as fast as possible updates everything in memory and when the job has finished then commits it to the DB. So the state of the data is still unavailable according to the metadata job.

Then it enters the cycle, the availability job kicks in later on and marks as available and sends a notifications, whilst the metadata job is running.

Now I'm confident I know what this is I can fix it!

[tidusjar]

So I have pushed what should be a fix to the develop branch, If someone could run that and let me know the outcome that would be much appreciated!

[Lownin]

Is that version 3.0.4825? If so I've got it installed and will report back when the issue occurs again or in a week if no issues.

[Lutaksur]

Hello,

I'm on develop 3.0.4840 and still have duplicate notification on discord and mail. The 3 duplicate notification of 3 different movie appear at the same time: 02:01am.

Edit: The time seem to be the Plex Sync cron job on Ombi.

[CaptAintHere]

Just tried a fresh install today with latest version 3.0.4817 and the problem still occurs for me. Emails (Gmail) are sent twice and at the same time...

[soulvoid86]

Just another confirmation that this is an issue. It's slightly different for me than most. I use Discord and Email for notifications. Discord works fine. For email notifications, every single one sends two at once. Whether it's to admin or to user. Otherwise, it works as expected (for me, it DOES NOT send them later in the day as well as others have stated.) Test emails doesn't do this, only notifications.

Below is a snippet from my log. This was a notification sent at 9AM (4AM server time) on the dot. Though the server is in a different time zone, so the email sent is showing a different time. Not sure if that might be causing something.

log snippet.txt

Full Logs:
log-20191127.txt

[mpearon]

I can confirm that this issue persists on 3.0.4892. Emails and Pushbullet notifications are going out at 5 past the hour for several hours.
I'll get some logs attached shortly.

[lastb0isct]

Confirmed -- Same issue. Multiple emails get sent out constantly. Seems a restart of the docker container fixes this but it is annoying.

[JoshDi]

This is still occurring in the latest dev version

[SmallwoodDR82]

Same issue here. I know it's a single dev and totally appreciate the product. Any update on this? Might need to turn off notifications until it's fixed. Woke up to over 100 emails :(

[nonissue]

I haven't had this issue since migrating to MYSQL for the database. In fact, Ombi has been a lot more stable overall.

[goatlube]

Also experiencing this issue - duplicate emails and duplicate app notifications.
log-20200505.txt
Duplicate notification at 2020-05-05 18:00:00.857 +10:00 in the log file.

[1nf1n17y]

I have been having this issue the last week or so as well.

[tidusjar]

So regarding this, I have yet been unable to actually reproduce the issue.

My assumption is that for some reason either

1. When checking for available content we are using stale data somehow
2. The update to the request to mark it as available is not working even know we are sending out a notification.

If someone is still having this issue please contact me on Discord so I can try and debug using your data?

[crobibero]

So regarding this, I have yet been unable to actually reproduce the issue.

My assumption is that for some reason either

1. When checking for available content we are using stale data somehow

2. The update to the request to mark it as available is not working even know we are sending out a notification.

If someone is still having this issue please contact me on Discord so I can try and debug using your data?

I was able to get the duplicate notifications to go away after switching to MySQL, so I believe it has something to do with SQLite locking

[mnkhouri]

I was able to get the duplicate notifications to go away after switching to MySQL

This was also the case for me

[MrTechGadget]

Just recently started experiencing this issue as well, with Pushover and Ombi app notifications. Is there a guide for moving to MySQL on docker container? Happy to try this to confirm resolution.

[lastb0isct]

@MrTechGadget -- I'd love a guide or some guidance at least as well. I think it will solve most of my problems, but don't have as much time as i used to to figure this stuff out.

[MrTechGadget]

Looks like there is some documentation on this, I’ll see if I can put together a dockerfile to get the base database done.

https://github.com/tidusjar/Ombi/wiki/Using-a-different-database

[somerandom48]

Looks like there is some documentation on this, I’ll see if I can put together a dockerfile to get the base database done.

https://github.com/tidusjar/Ombi/wiki/Using-a-different-database

Did you manage to make a docker-compose with MariaDB to change the DB over?

[MrTechGadget]

No, I haven’t gotten back around to this yet.

On Tue, Jun 30, 2020 at 8:16 PM somerandom48 [e-mail]
wrote:

Looks like there is some documentation on this, I’ll see if I can put
together a dockerfile to get the base database done.

https://github.com/tidusjar/Ombi/wiki/Using-a-different-database

Did you manage to make a docker-compose with MariaDB to change the DB over?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tidusjar/Ombi/issues/2998#issuecomment-652113337, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AECUW5PE2NO5QU62XIXFR4DRZJ547ANCNFSM4HSXRJFQ
.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[soulvoid86]

I’m experiencing this issue on Linux. It was happening in v3 for months and is also happening after updating to v4. I’m willing to provide access to my Ombi instance for debugging. 2 emails are sent from my system every single time.

[Shadow229]

Same issue on latest v4 (4.0.533.0). 2 emails sent every time from Gmail address. Also happy to provide anything that will help.

Latest Log:
log-20201021.txt