Peertube: Peertube is not using redundant storage space

Created on 1 Oct 2018 · 16Comments · Source: Chocobozzz/PeerTube

What do you see?
I have redundancy enabled since beta 14 was released, and configured it for 4 large instances. However, Peertube is only using a small fraction of the space I configured for redundancy, according to https://peertube.social/api/v1/server/stats:

{
  "totalLocalVideos": 1364,
  "totalLocalVideoViews": 16269,
  "totalVideos": 18048,
  "totalLocalVideoComments": 149,
  "totalVideoComments": 1977,
  "totalUsers": 888,
  "totalInstanceFollowers": 82,
  "totalInstanceFollowing": 280,
  "videosRedundancy": [
    {
      "totalVideos": 2,
      "totalVideoFiles": 6,
      "strategy": "most-views",
      "totalUsed":  2329184245,
      "totalSize": 10737418240
    },
    {
      "totalVideos": 2,
      "totalVideoFiles": 8,
      "strategy": "trending",
      "totalUsed":  103911849,
      "totalSize": 10737418240
    },
    {
      "totalVideos": 24,
      "totalVideoFiles": 60,
      "strategy": "recently-added",
      "totalUsed":  4439486834,
      "totalSize": 10737418240
    }
  ]
}

What is the expected behaviour?
As you can see, Peertube is only using between 10% and 50% of the available storage for redundancy.

I don't see anything related to redundancy in the logs.

PeerTube version or URL: https://peertube.social/api/v1/server/stats

Redundancy Type

Source

Nutomic

👍1

Most helpful comment

I finally found the bug: https://github.com/Chocobozzz/PeerTube/commit/f8278b96053d42ac46aea17f949325caa03e8a00

Chocobozzz on 4 Sep 2019

🎉2

All 16 comments

Thanks for the issue @Nutomic. I'm pretty sure this is related to https://github.com/Chocobozzz/PeerTube/issues/1139

Chocobozzz on 1 Oct 2018

FYI it looks even worse in the latest version (running for about 1 day now):

{
  "totalLocalVideos": 1393,
  "totalLocalVideoViews": 16801,
  "totalVideos": 18065,
  "totalLocalVideoComments": 158,
  "totalVideoComments": 2011,
  "totalUsers": 899,
  "totalInstanceFollowers": 81,
  "totalInstanceFollowing": 280,
  "videosRedundancy": [
    {
      "totalVideos": 0,
      "totalVideoFiles": 0,
      "strategy": "most-views",
      "totalUsed": 0,
      "totalSize": 10737418240
    },
    {
      "totalVideos": 0,
      "totalVideoFiles": 0,
      "strategy": "trending",
      "totalUsed": 0,
      "totalSize": 10737418240
    },
    {
      "totalVideos": 4,
      "totalVideoFiles": 8,
      "strategy": "recently-added",
      "totalUsed":   335934142,
      "totalSize": 10737418240
    }
  ]
}

Nutomic on 2 Oct 2018

What about now with RC.2?

Chocobozzz on 4 Oct 2018

https://peertube.social/api/v1/server/stats

{
  "totalLocalVideos": 1417,
  "totalLocalVideoViews": 17226,
  "totalVideos": 18346,
  "totalLocalVideoComments": 163,
  "totalVideoComments": 2054,
  "totalUsers": 921,
  "totalInstanceFollowers": 83,
  "totalInstanceFollowing": 280,
  "videosRedundancy": [
    {
      "totalVideos": 21,
      "totalVideoFiles": 74,
      "strategy": "most-views",
      "totalUsed":  5354024649,
      "totalSize": 10737418240
    },
    {
      "totalVideos": 15,
      "totalVideoFiles": 54,
      "strategy": "trending",
      "totalUsed":  2149938229,
      "totalSize": 10737418240
    },
    {
      "totalVideos": 18,
      "totalVideoFiles": 47,
      "strategy": "recently-added",
      "totalUsed":  2421595898,
      "totalSize": 10737418240
    }
  ]
}

Better than before, but still not great. 20-50% usage (one day after upgrading).

Nutomic on 4 Oct 2018

I checked and now it's

{
  "totalLocalVideos": 1421,
  "totalLocalVideoViews": 17293,
  "totalVideos": 18377,
  "totalLocalVideoComments": 163,
  "totalVideoComments": 2065,
  "totalUsers": 926,
  "totalInstanceFollowers": 83,
  "totalInstanceFollowing": 280,
  "videosRedundancy": [
    {
      "totalUsed": 5746413590,
      "totalVideos": 25,
      "totalVideoFiles": 86,
      "strategy": "most-views",
      "totalSize": 10737418240
    },
    {
      "totalUsed": 5912138713,
      "totalVideos": 20,
      "totalVideoFiles": 70,
      "strategy": "trending",
      "totalSize": 10737418240
    },
    {
      "totalUsed": 4172786422,
      "totalVideos": 22,
      "totalVideoFiles": 55,
      "strategy": "recently-added",
      "totalSize": 10737418240
    }
  ]
}

So it seems to work :) If you want to speed up the redundancy, you can use a shorter interval for check_interval in the configuration.

Chocobozzz on 5 Oct 2018

🎉2

Awesome!

Nutomic on 5 Oct 2018

@Chocobozzz Looks like this isnt fixed at all. I am giving 150 GB for redundancy, but the redundancy folder only has 26 GB. And I had this configuration for at least a few weeks.

Nutomic on 9 Mar 2019

@Nutomic Please paste your redundancy section of your configuration file

Chocobozzz on 11 Mar 2019

Here it is. I also have redundancy enabled for 14 different instances.

redundancy:
  videos:
    check_interval: '1 hour' # How often you want to check new videos to cache
    strategies:
      -
        size: '50GB'
        # Minimum time the video must remain in the cache. Only accept values > 10 hours (to not overload remote instances)
        min_lifetime: '24 hours'
        strategy: 'most-views' # Cache videos that have the most views
      -
        size: '50GB'
        # Minimum time the video must remain in the cache. Only accept values > 10 hours (to not overload remote instances)
        min_lifetime: '24 hours'
        strategy: 'trending' # Cache trending videos
      -
        size: '50GB'
        # Minimum time the video must remain in the cache. Only accept values > 10 hours (to not overload remote instances)
        min_lifetime: '24 hours'
        strategy: 'recently-added' # Cache recently added videos
        min_views: 10 # Having at least x views

Nutomic on 11 Mar 2019

Do you still have the issue @Nutomic with 1.4? Because I can't reproduce: https://peertube3.cpy.re/api/v1/server/stats

If yes, please send me by email your logs and the list of other instances on which you enabled redundancy

Chocobozzz on 15 Aug 2019

I'm still waiting for the Docker image to be available. My offer from #1820 still stands.

Nutomic on 15 Aug 2019

So I updated the instance 3 days ago, and it looks exactly like before. On your test instance I also see the problem, it is only using ~30% of the available space according to the api. I would expect that it always uses close to 100%.

I have redundancy enabled for a couple of big instances, but I dont see where I can get a full list (its not possible to sort by the redundancy column). And which logs do you need, the ones from docker-compose logs peertube, or from the logs folder?

Nutomic on 19 Aug 2019

I tried something: https://github.com/Chocobozzz/PeerTube/commit/217ffacfdde0972a3ee4e2db152e1710139b4ec4, let's see if it fixes the bug.

I have redundancy enabled for a couple of big instances, but I dont see where I can get a full list (its not possible to sort by the redundancy column)

You can't for now.