Plugin.video.netflix: Automatic Library Update does not work

[pipsen]

I'm submitting a ...

• [x] bug report
• [ ] feature request
• [ ] support request

General information

Automatic Library Update does not work

Addon version used

0.14.1 release

Description

Automatic Library Update does not work

Steps to Reproduce

• Installed netflix plugin from scratch
• logged in with username / password
• activated "Synchronize library with my list"
• Once "Manual full synchronizsation"
  => Current state of Netflix list is successfully imported to library
• Add a new movie to "my list" on netflix webinterface in browser
• activated automatic synchronisation (time: e.g. 5 p.m.)
• check interval every five minutes
• wait for at least 24 hours

Expected behavior: [What you expected to happen]

• the new movie is automatically added to library

Actual behavior: [What actually happened]

• Nothing happend. No new stream files in "movies" folder
• No new movie inside library

Context (Environment)

Installation

• [ x] I'm using this Netflix Repo
• [ ] I'm using other Netflix Repo (Please specify which)
• [ ] I'm using a different source (Please tell which)

Operating System

• [ x] Linux (x86/x64)
• [ x] OSX (x86/x64)
• [ ] Windows (x86/x64)
• [ ] Linux (ARM)
• [ ] Android

Additional informatin on the environment

Tested on iMac and Intel NUC with LibreELEC 9.0.1

[Descripe your environment a bit more detailed, are you using LibreElec f.e.]
Tested on iMac and Intel NUC with LibreELEC 9.0.1

Debug log

2019-05-02 10:52:02.165 T:139932718855936 WARNING: Attempt to use invalid handle -1
2019-05-02 10:57:01.006 T:139934983771904 WARNING: Previous line repeats 4 times.
2019-05-02 10:57:01.006 T:139934983771904 WARNING: [plugin.video.netflix (0)] Canceling previous library update: duration >6 hours
2019-05-02 10:57:01.274 T:139932718855936 WARNING: Attempt to use invalid handle -1
2019-05-02 11:02:01.086 T:139934983771904 WARNING: Previous line repeats 4 times.
2019-05-02 11:02:01.086 T:139934983771904 WARNING: [plugin.video.netflix (0)] Canceling previous library update: duration >6 hours
2019-05-02 11:02:01.432 T:139932718855936 WARNING: Attempt to use invalid handle -1
2019-05-02 11:07:01.169 T:139934983771904 WARNING: Previous line repeats 4 times.
2019-05-02 11:07:01.169 T:139934983771904 WARNING: [plugin.video.netflix (0)] Canceling previous library update: duration >6 hours

SIMPLE RULES:

• Before performing any operation enable Kodi debug logging [Settings->System->Logging->Enable debug logging] only after perform the operations to reproduce the issue
• Copy here a link to your debug log, use something like https://paste.kodi.tv/ or similar
• Do NOT paste the content of the log here
• Do NOT cut the log
• If the log file is really huge (more than 1Mb) then in Kodi settings disable the Component-specific logging

Other information

[e.g. detailed explanation, related issues, suggestions how to fix, links for us to have context, etc.]

Screenshots

[Please add a screenshot if that helps understanding your problem]

[You can erase any parts of this template not applicable to your Issue.]

[Smeulf]

There's a related issue with exported tv shows : no new episodes are added automatically. Sorry no logs, it's a bit hard to catch.

@pipsen : you should be aware that this function when working may result in some ban from Netflix because of the number of request it can do in a few minutes...

@CastagnaIT : I start to feel the same as you about library exports, maybe it's not such a good idea to keep that function, because of the bans, and because of the resume points issue (https://github.com/CastagnaIT/plugin.video.netflix/issues/7)...

[CastagnaIT]

I've been thinking about doing something for a long time
in order not to harm the many users who use it you could think of limiting the functionality,
I thought I'd do it this way:

no more searching for automatic updates for new episodes
no more synchronization between kodi library and my list
keep only the manual updates via context menu

I do not understand why users do not use my list of netflix...

[Smeulf]

I do not understand why users do not use my list of netflix...

It's much more easier to have the all movies/tv shows (local + Netflix + any other providers) in the same view, no matter the source location ^^

[pipsen]

Exactly :-)

[CastagnaIT]

hehe I use kodi only for netflix that's why I did not realize immediately :-D

[pipsen]

What is your plan now? A fully automatic solution without any user interaction would be awesome

[CastagnaIT]

this function forces a large amount of data to synchronize everything, it's not welcome by netflix servers

more people were banned for 24 hours by netflix for using this function, it's not the first time, the ideal solution as also said smeulf is to remove this function

I'll just leave the possibility to update manually via context menu

the first thing to do is to respect the netflix rules

[Smeulf]

IMHO, if the automatic sync is maintained, it should :

• run once or twice a day max

• be 'respectful' towards Netflix servers. I mean perform no more than X request in a minute (I don't know if there's some Netflix best practices regarding API usage...)

• If my list is still sync, then the addon should merge 'my list' and the manually exported items into one single in-memory list, and then perform the sync

About the export itself, I still can't understand why it first remove the all STRM files and add them again when refreshing... Maybe to ensure the files are really correct and was not modified ?

Also, an option to avoid scanning the library automatically after every manual export would be good. I mean, when I export some TV shows, it's usually 5 or 10... So no need to scan at every show, I just run a full scan after I exported everything.

Remains one subject : what about multiple profiles and multiple 'My List' ?

Wide subject ^^

Cheers.
Smeulf.

[CastagnaIT]

you can do a lot in words, but implementing them is quite different

To point 2, I had thought about it, but we don't have the specifications of netflix api and we can't know exactly the limit, if you think that there are users with more than 15/20 exported series become heavy the thing...

if theorizing we found a way to slow down the operation, it would become very long,
there are many points to consider:
-first the scheduled synchronization occurs only if the player kodi is idle not to interfere with the playback, slowing down the operation there is a possibility that it will take a long time to complete
-we have to consider how to implement how to stop/pause the process in case of exit from kodi or start playback, there are many slow devices based on linux that use this app so it is imperative to implement
-the operation is started by the thread of the service not the main one, so things get complicated enough you do not have direct access to variables and states used in main thread

the current implementation has many gaps and bugs many already you know them, another one for example is the removal of an exported tv series does not always work properly, sometimes causes wrong behavior by deleting or not the exported files

some small fixes i could do them, but it's getting out of hand...
so the real changes to be implemented and fixed are not simple,
it takes several days of work, not hours, to be able to study it and reimplement it, i don't have that kind of time and nobody pays me to do it,
if someone has experience, ability and free time, can propose its changes well implemented and in code documented where necessary

now i think you understand my intention to limit or remove this option, but if someone wants to take charge then welcome it, except for a few little helpings in the last months i am the only one to keep the project alive...
unless there are other forks hahaha :-D

[dm2912]

I do not understand why users do not use my list of netflix...

simple. trakt synchronisation.

I have kodi on multiple devices and there are some netflix programs ive watched 50+ episodes of and I dont have the time to go through marking them all watched.

With library kodi automatically syncs my watched history with trakt, which means also i can watch something on one device, its marked watched on another device.

For example, say I add Orange is the new Black to "my list". Ive already watched 30+ episodes, but the add-on doesnt know that, so it doesnt mark them watched. I have to remember where I got to (and manually mark them watched episode by episode). I then watch 2-3 episodes on my main device. Later when i go to watch on a secondary device, it doesnt have the watched history from first device, so again, i have to remember where I got to.

With Library, the minute I add Orange is the New Black to my library, it syncs with trakt, and marks every episode ive watched as marked. And if i watch an episode on one device, trakt will mark it as watched on the other so all my devices remain in sync.

the library integration is actually a crucial part of the add on for me, without it, watched history across devices is useless.

the solution would be if the add-on itself synced watched history with trakt. Then there would be no need to use the library. For trakt users it would pick up the watched history, and that would also mean it would sync between devices

[Smeulf]

@dm2912 I'm currently working on the library, and the first steps are to make it working well (https://github.com/CastagnaIT/plugin.video.netflix/issues/38) and faster. First on a single device, and then on multiple devices, if the addon's internal database permits it.

I'm not using trakt, and I don't want to use it. So I won't use it as a main synchronization point for this addon.

I have the same problem you have with multiple devices, and I'm doing my best to make it work without extra addon! But it takes time ;)

[dm2912]

@dm2912
I'm not using trakt, and I don't want to use it. So I won't use it as a main synchronization point for this addon.

I have the same problem you have with multiple devices, and I'm doing my best to make it work without extra addon! But it takes time ;)

oh completely understand, was just explaining why a user may prefer to use library than the mylist option. :D

[Smeulf]

Some news: I have something almost finished in a branch.

Remains 2 questions:

• Should episodes deleted from Netflix be deleted from the library?

• About NFO, I have two options.

The first is to add a flag in our internal library database, so when the NFO flag is 'true' for a show or a season, the NFO will be exported for new episodes. But this implies a refresh of the shows exported with NFO.

The second option is to determine if a NFO file exists in the export folder and export new episodes with or without NFO according the test. But if someone saved the database as NFO files, it may return a false positive.

I prefer the first option, but your opinion is very welcome.

[dm2912]

• Should episodes deleted from Netflix be deleted from the library?

One thing to be aware of should you choose to. A lot of people sync their Library to Trakt.

I do, and that includes my entire library so local files as well as Netflix items etc.

In this instance a lot of people have the two following trakt options set:

1. Add new movies/episodes to Trakt collection (this automatically adds anything we put in our library to our online trakt collection)

2. Remove deleted movies/episodes from our collection.

The second setting would mean people who used netflix library integration alongside trakt may find things disappearing from their online collection.

What I would suggest is that it is disabled by default, with a checkbox and an instruction "If you are using Trakt to sync to your collection disable - remove deleted episodes - in your trakt settings"

[Smeulf]

I think I gonna start with only new episodes, and add removing features in the future if it's requested.

The feature name is still 'export new episodes' ^^