Nixpkgs: youtube-dl is broken due to DMCA takedown

[chris-martin]

The source URL for the youtube-dl package redirects to GitHub, which due to https://github.com/github/dmca/blob/master/2020/10/2020-10-23-RIAA.md now returns: ERROR 451: Unavailable for Legal Reasons.

[chris-martin]

Maintainers: @bluescreen303 @phreedom @AndersonTorres @fpletz @enzime @ma27

[worldofpeace]

smh

[AndersonTorres]

What we can do? Just mark it as broken or throw some message?

[berbiche]

Would it be viable to use an unofficial fork?
Otherwise, is it possible to instruct users to do a simple override?

youtube-dl.overrideAttrs (_: {
  src = pkgs.fetchgit {
    ...
  };
})

[worldofpeace]

Yeah, our only choice currently is to throw a message like github has

[worldofpeace]

Would it be viable to use an unofficial fork?
Otherwise, is it possible to instruct users to do a simple override?

youtube-dl.overrideAttrs (_: {
  src = pkgs.fetchgit {
    ...
  };
})

Generally, it's not a good idea to put an unofficial fork of something that, github has already taken down the official, and to use something unofficial in a package collection like nixpkgs. If GitHub has to comply to these hostilities every fork is going to get flagged as well

[Infinisil]

I'd say let's wait a couple days for this to calm down. There is a chance that youtube-dl can be brought back up.

[mohe2015]

Would it be viable to use an unofficial fork?
Otherwise, is it possible to instruct users to do a simple override?

youtube-dl.overrideAttrs (_: {
  src = pkgs.fetchgit {
    ...
  };
})

I think this could possibly lead to a takedown of nixpkgs? (I'm not a lawyer and didn't even read the DMCA laws, Germany doesn't have such a thing (yet))

[berbiche]

I retract my suggestion. Aiding users with precise instructions on how to circumvent the ban is too risky.

[AndersonTorres]

I retract my suggestion. Aiding users with precise instructions on how to circumvent the ban is too risky.

Even because the DMCA note includes a bunch of forks, and explicitly says "but not limited".

[worldofpeace]

If we'd go for a mirror outside github there's pypi https://pypi.org/project/youtube_dl/#files and gitee https://gitee.com/mirrors/youtube-downloader

[rnhmjoj]

There is a high chance that this will come back with a counter-notice, so me might as well do nothing.
In the meantime, there is an official archive on archive.org...

[AndersonTorres]

how much we have to wait, then? Three days?

[Ma27]

shouldn't the current source tarball be available in tarballs.nixos.org anyways?

[L-as]

@AndersonTorres AFAIK they have to put in a counter-notice as was done with Popcorn Time, and then if the other party does not respond within some timeframe (1 month?), GitHub should put it back up. In any case, I think 3 days is way too optimistic.

IMO nixpkgs also ought to be moved off of GitHub, since it would be catastrophic if some copyright troll brought it down for such a long period.

[worldofpeace]

IMO nixpkgs also ought to be moved off of GitHub, since it would be catastrophic if some copyright troll brought it down for such a long period.

This is very off topic to this thread.

[juliosueiras]

Question, since I always wonder, what are the procedure for situation like this if a particular package is removed due to legal reason , does the package get removed , or does it get removed retroactively (all history) ?

[chris-martin]

shouldn't the current source tarball be available in tarballs.nixos.org anyways?

It seems to be, because I can still install youtube-dl via Nix. Is there any limit to how long stuff stays in that cache?

[samueldr]

All source files are derivations. They are special in that they are fixed-output, so kind-of content-addressed.

All derivations that go through the org's hydra instance are cached in the cache.

So as long as the hash requested is the same, we have the intrinsic cache from it being content-addressed. Since we don't expected to see official updates on that repo until this all gets resolved, we still have a working source for users using the official cache.

So I'm doubting that youtube-dl is "broken", as asserted. Especially since we can see users installing it. There is nothing to do until we have news from upstream. Things will continue to work with Nixpkgs.

Don't believe me?

Make an inconsequential change and see it still building fine!

diff --git a/pkgs/tools/misc/youtube-dl/default.nix b/pkgs/tools/misc/youtube-dl/default.nix
index e3b64939a0a..dea6512b2be 100644
--- a/pkgs/tools/misc/youtube-dl/default.nix
+++ b/pkgs/tools/misc/youtube-dl/default.nix
@@ -47,6 +47,7 @@ buildPythonPackage rec {

   postInstall = ''
     installShellCompletion youtube-dl.zsh
+
   '';

   # Requires network

If we had had open PRs updating the revision, then they would be broken.

The binary cache gives us quite a strong resiliency from upstreams being flaky or even going missing.

---

Note: This is all about the practical side of Nixpkgs for end-users. Obviously it's best if we have an upstream to follow. But let's NOT act in a reactionary manner, and wait until upstream gives news about all this.

[samueldr]

I would go even as far as to say that any PRs about changing youtube-dl's sources are to be outright closed as invalid until there is news from upstream, or a consensus among distros for a new upstream.

[arianvp]

It sounds like we can just close this given we have the source in cache?. It's only a problem if someone wants to bump a version. Let's hope GitHub puts it back online before a new version is released

[samueldr]

It's only a problem if someone wants to bump a version

And not even a problem, considering we have no upstream to give us a new version to bump to!

[chris-martin]

Okay, it seems I was wrong to assume that this was a problem. Good to know the Hydra cache is robust enough to rely on.

[fgaz]

Fyi downloads from the official website are working now (they no longer redirect to github)

[mohe2015]

I would keep this open as long as the underlying issue is not resolved. Maybe the title can be changed.

[samueldr]

When there will be a problem that we have to handle, we can open an issue (or a PR).

For the time being, nothing differs for us: we are still waiting for upstream for updates, whether they come from GitHub or elsewhere.

We're already fetching from their source tarball download rather than directly from GitHub, and they are now serving downloads from elsewhere. So no need for an issue that would be open "just in case".

[Infinisil]

It's back up! https://github.blog/2020-11-16-standing-up-for-developers-youtube-dl-is-back/