Peertube: Video Views from Following Instances are not Counted

[prannonpendragas]

Describe the current behavior
I administer greatview.video.

When I watch a video directly on a hosting peertube instance (where the video originates), the video view is recorded in redis as redis-greatview.video-views-${uuid}-${my ip address}. This record is later processed in a peertube batch job and the view is counted against the video that I watched.

When I watch the same video on a _following_ peertube instance (utilizing federation), the video view is recorded in redis as redis-greatview.video-video-view-###-h#. This record is _never_ processed in a peertube batch job and the view is never counted against the video that I watched.

Steps to reproduce:

1. On the peertube server where redis is hosted, watch -n1 'echo "keys *video-view*" | redis-cli -h ${the ip address of the redis server} -p 6379'
2. Access the originating peertube instance and watch a video for 30 seconds, and observe the redis-${instance name}-views-${uuid}-${my ip address} key entered into redis for later processing.
3. Access a following peertube instance and watch the same video for 30 seconds, and observe the redis-${instance name}-video-view-###-h# key entered into redis for later processing.
4. After the top of the hour, observe that the redis-greatview.video-views-${uuid}-${my ip address} entries in redis have been processed and removed, whereas the redis-greatview.video-video-view-###-h# entries remain and the views are not counted against the video.

Describe the expected behavior
All views for a given video across the federation should be counted against the video on the originating instance.

• PeerTube instance:
  
  
  
  • URL: https://greatview.video
  
  
  • version: 2.4.0
  
  
  • NodeJS version: 10.23.0-1nodesource1
  
  
  • Ffmpeg version: 4.3.1-0york0~16.04
  
  

[Chocobozzz]

Hello,

redis-${instance name}-views-${uuid}-${my ip address} is a key to prevent you to add multiple views with your IP. But even with a local view, peertube will use the redis-${instance name}-video-view-###-h# key. This is in fact a redis set. But it's weird that after 1 hour, this entry is not processed and deleted from redis.

Could you enable debug logs and check this log in particular?

[prannonpendragas]

@Chocobozzz I've had the debug log enabled for a day or two, and I haven't spotted any obvious errors. I've seen a few entries for adding views to videos.

2020-12-27 17:01:00.011 #033[34mdebug#033[39m: Adding 1 views to video 3 in hour 16.
2020-12-27 17:01:00.150 #033[34mdebug#033[39m: Adding 1 views to video 34 in hour 16.
2020-12-27 17:01:00.272 #033[34mdebug#033[39m: Adding 1 views to video 62 in hour 16.
2020-12-27 17:01:00.348 #033[34mdebug#033[39m: Adding 1 views to video 103 in hour 16.
2020-12-27 17:01:00.473 #033[34mdebug#033[39m: Adding 1 views to video 591 in hour 16.

What's interesting is that I've been running a script on the side to print out the view keys in redis, and there are some entries in there that have been there for many hours.

cat watch_views.sh 
#!/bin/bash

while true; do
        echo "keys *video-view*" | redis-cli -h ${ip address} -p 6379 > buffered_views_$(date +%s).txt
        sleep 3600
done

ls -l buffered_views_16090*
-rw-r--r-- 1 root root 198 Dec 26 20:13 buffered_views_1609013634.txt
-rw-r--r-- 1 root root 198 Dec 26 21:13 buffered_views_1609017183.txt
-rw-r--r-- 1 root root 198 Dec 26 21:13 buffered_views_1609017234.txt
-rw-r--r-- 1 root root 198 Dec 26 22:13 buffered_views_1609020783.txt
-rw-r--r-- 1 root root 198 Dec 26 22:13 buffered_views_1609020834.txt
-rw-r--r-- 1 root root 198 Dec 26 23:13 buffered_views_1609024383.txt
-rw-r--r-- 1 root root 198 Dec 26 23:13 buffered_views_1609024434.txt
-rw-r--r-- 1 root root 198 Dec 27 00:13 buffered_views_1609027983.txt
-rw-r--r-- 1 root root 198 Dec 27 00:13 buffered_views_1609028034.txt
-rw-r--r-- 1 root root 198 Dec 27 01:13 buffered_views_1609031583.txt
-rw-r--r-- 1 root root 198 Dec 27 01:13 buffered_views_1609031634.txt
-rw-r--r-- 1 root root 198 Dec 27 02:13 buffered_views_1609035183.txt
-rw-r--r-- 1 root root 198 Dec 27 02:13 buffered_views_1609035234.txt
-rw-r--r-- 1 root root 198 Dec 27 03:13 buffered_views_1609038783.txt
-rw-r--r-- 1 root root 198 Dec 27 03:13 buffered_views_1609038834.txt
-rw-r--r-- 1 root root 198 Dec 27 04:13 buffered_views_1609042383.txt
-rw-r--r-- 1 root root 198 Dec 27 04:13 buffered_views_1609042434.txt
-rw-r--r-- 1 root root 198 Dec 27 05:13 buffered_views_1609045983.txt
-rw-r--r-- 1 root root 198 Dec 27 05:13 buffered_views_1609046034.txt
-rw-r--r-- 1 root root 198 Dec 27 06:13 buffered_views_1609049583.txt
-rw-r--r-- 1 root root 198 Dec 27 06:13 buffered_views_1609049634.txt
-rw-r--r-- 1 root root 198 Dec 27 07:13 buffered_views_1609053183.txt
-rw-r--r-- 1 root root 198 Dec 27 07:13 buffered_views_1609053234.txt
-rw-r--r-- 1 root root 198 Dec 27 08:13 buffered_views_1609056783.txt
-rw-r--r-- 1 root root 198 Dec 27 08:13 buffered_views_1609056834.txt
-rw-r--r-- 1 root root 198 Dec 27 09:13 buffered_views_1609060383.txt
-rw-r--r-- 1 root root 198 Dec 27 09:13 buffered_views_1609060434.txt
-rw-r--r-- 1 root root 198 Dec 27 10:13 buffered_views_1609063983.txt
-rw-r--r-- 1 root root 198 Dec 27 10:13 buffered_views_1609064034.txt
-rw-r--r-- 1 root root 198 Dec 27 11:13 buffered_views_1609067583.txt
-rw-r--r-- 1 root root 198 Dec 27 11:13 buffered_views_1609067634.txt
-rw-r--r-- 1 root root 198 Dec 27 12:13 buffered_views_1609071183.txt
-rw-r--r-- 1 root root 198 Dec 27 12:13 buffered_views_1609071234.txt
-rw-r--r-- 1 root root 198 Dec 27 13:13 buffered_views_1609074783.txt
-rw-r--r-- 1 root root 198 Dec 27 13:13 buffered_views_1609074834.txt
-rw-r--r-- 1 root root 198 Dec 27 14:13 buffered_views_1609078383.txt
-rw-r--r-- 1 root root 198 Dec 27 14:13 buffered_views_1609078434.txt
-rw-r--r-- 1 root root 198 Dec 27 15:13 buffered_views_1609081983.txt
-rw-r--r-- 1 root root 198 Dec 27 15:13 buffered_views_1609082034.txt
-rw-r--r-- 1 root root 198 Dec 27 16:13 buffered_views_1609085583.txt
-rw-r--r-- 1 root root 198 Dec 27 16:13 buffered_views_1609085634.txt
-rw-r--r-- 1 root root 712 Dec 27 17:13 buffered_views_1609089183.txt
-rw-r--r-- 1 root root 712 Dec 27 17:13 buffered_views_1609089234.txt

Note how the filesize never changes from Dec 26 20:13 until Dec 27 17:13. Incidentally, that's the hour when the log entries show up.

cat buffered_views_1609085583.txt
redis-greatview.video-video-view-255-h0
redis-greatview.video-video-view-35-h0
redis-greatview.video-video-view-253-h0
redis-greatview.video-video-view-622-h0
redis-greatview.video-video-view-70-h0

[Chocobozzz]

Thank you very much @prannonpendragas, I found the bug and it should be fixed by https://github.com/Chocobozzz/PeerTube/commit/cc49be3e3547703d00ef7363d664221a39334ca3