Zstd: Consider re-licensing to Apache License v2

[umbrant]

Hello,

The Apache Software Foundation recently changed its policy regarding the "Facebook BSD+patents" license that applies to zstd and many other FB open-source projects, and now considers it unsuitable for inclusion in ASF projects. There is a discussion of this in the context of RocksDB on LEGAL-303, which was resolved when RocksDB was relicensed as dual ALv2 and GPLv2.

Is the zstd community also open to relicensing with ALv2? This change would be helpful for Apache Hadoop (of which I'm a PMC member) since it would let us bundle zstd as part of our release artifacts. @omalley also expressed interest in this relicensing as an Apache ORC PMC member.

Thanks in advance!

[Conan-Kudo]

My main concern with relicensing to purely ASL 2.0 is that it means that zstd would be unsuitable for projects that have an explicit goal of being (L)GPLv2 compatible or are GPLv2. It makes things very ugly for GPLv2-only projects or people using GPLv2+ projects in a manner in which they do not wish to abide by the terms of the GPLv3 in order to be compatible with ASL 2.0.

My preference here would be to reduce the scope of the PATENTS license to something similar to what the WebM Project does, which limits it to patents related to the codebase itself and limits the domain to software patents.

[umbrant]

Hi @Conan-Kudo,

RocksDB went with a dual-license of ALv2 and GPLv2, which would work for GPLv2 projects. I think this would be workable for zstd too, but this is ultimately up to the zstd maintainers.

[Conan-Kudo]

If zstd were being dual-licensed, I'd probably go with ASL 2.0 + LGPLv2+ as the combo, since it is a library and the goal is to encourage adoption.

[boxerab]

Good idea! I for one am not comfortable with the PATENT grant in zstd, so ASLv2 is way better.
+1

[nemequ]

Does Facebook actually claim to have any patents which cover zstd? AFAICT (IANAL) the patent license only revokes the license to use patents which are necessary for zstd if you're involved in patent litigation against FB; the copyright license (BSD-3-Clause) remains.

If there are no such patents it's a moot point; they're just terminating a license to use nothing. OTOH, if FB does own relevant patents then simply removing the PATENTS file would eliminate the license to use them, which puts everyone in the same position they would be in if they had sued Facebook.

This issue is obviously impeding zstd adoption, which seems like a pretty high cost for an empty (assuming FB doesn't have any relevant patents) threat, so I'd like to see it resolved somehow. Switching to Apache 2.0 would be fine, but it seems like overkill… I'd be comfortable if FB just publicly stated that they don't have any relevant patents (like Google did for Brotli) and dropped the threatening PATENTS file.

[boxerab]

Even if they have no patents at the moment (and there is no way of really telling whether this is true)
they may file them in the future. ASLv2 gives a patent grant in the license itself, with no strings attached like the PATENTS grant (but there is a retaliatory clause )

http://en.swpat.org/wiki/Patent_clauses_in_software_licences

[boxerab]

More discussion of this issue can be found here

[boxerab]

The ASLv2 retaliatory clause is limited to patents related to the Work being licensed, while the zstd grant
applies to any patent litigation, not necessarily related to zstd.

[nemequ]

I agree that ASLv2 would be stronger than what I suggested, but there are drawbacks… for one, there is something to be said for short, easy to understand licenses like BSD/MIT/ISC/zlib etc. which most people can grok without consulting a lawyer. Another good solution would be BSD + a patent grant which doesn't terminate on legal action against Facebook.

I don't think future patents are much of a concern; zstd has been public for a while now, so AFAIK it shouldn't be possible to file new patents which cover it.

FB acquiring existing patents which cover zstd is more of an issue, but if those exist they're already out there and already a threat to anyone thinking about using zstd; Facebook acquiring them wouldn't make that problem any worse. Frankly I'd rather have Facebook holding something like that defensively than a patent troll.

Going back to Brotli, part of that IPR disclosure is a promise that any patents is a commitment to make any patents they do/will have available for licensing under FRAND terms. FB could easily do the same.

Like I said, I would be comfortable with a statement that FB doesn't believe it owns any patents which cover zstd, which should at least get you clear of willful infringement territory. Others may want something more, but there are solutions other than ASLv2.

[JarekDuda]

nemequ, I have checked it looks Facebook has no published related patent application in the neighboring issue, however, later someone has commented it that beside 1 year since making idea public, there are 18 months for publication of patent application - so we need to wait ~1.5 more years to be certain...
However, as the Google ANS patent application took 6 months to be published, I have found this article saying that these 18 months is a myth, in practice publication can be 3-36+ months from filling ... making the use of ZSTD a real paranoia not worth the risk for large companies ... another comment I have found here: "I work at a fairly big company that has a good relationship with Facebook (they're a customer and partner) and we aren't allowed to use React or zstd."

[adamkennedy]

If zstd is not relicensed, it would seem that it will have to be rolled back and removed from Hadoop, HBase and Spark by August 31st. Am I reading that correctly?

All the issues for adding support to Hadoop seems to be here https://issues.apache.org/jira/browse/SPARK-19112?jql=text%20~%20%22ZStandard%22

It looks like HBase is already in the processing of identifying what needs to be removed in HBASE-18391.

[adamkennedy]

It would be really good if we could get guidance from the maintainers on their intentions for this...

[umbrant]

From the Apache Hadoop and HBase perspective, our current integration with zstandard is fine since we do not include zstandard directly in our projects. It's available as an off-by-default optional dependency. Users enable it by installing zstandard themselves and recompiling Hadoop to link against it.

However, this is difficult for many users and complicates setup. Particularly since it doesn't look like there are pre-build zstd packages for the major platforms (RHEL, CentOS, Ubuntu, Suse). It'd be a lot easier if we could enable zstd by default and bundle it with the rest of our code.

[Conan-Kudo]

@umbrant zstandard is available in RHEL/CentOS through EPEL (as zstd).

[boxerab]

I was planning on using this codec for a project of mine, also contributing code, but will not be able to do so until license is changed.

[adamsch1]

Hi Cyan, any update on this issue? Would love to see ZSTD relicensed like other Facebook software wrt the patents clause.

[Cyan4973]

working on it...

(sorry, I can't be more precise)

[boxerab]

In a nutshell, either there are patents or planned patents for zstd, or there are not.
If there aren't any patents, please remove the PATENTS file, as it is not relevant to the project.
If there are patents, or planned patents, at the very least please disclose this so users can make
informed decisions.

Otherwise, there is a chilling effect on use of this fantastic library.

[umbrant]

Thanks for the update Cyan, appreciate your efforts. I understand that these internal discussions can take time.

[aras-p]

Hey look, #801 just merged! This is awesome.

[Conan-Kudo]

@Cyan4973 Thanks for the fix! This keeps everyone sane and happy. :)

[Cyan4973]

Wanted to let the community know that we’ve released zstd v1.3.1 earlier today, which adjusts it to be dual licensed as BSD and GPLv2.
We expect this new license to also be compliant with ASF projects.

[Conan-Kudo]

@Cyan4973 Is it dual-licensed or double-licensed? That is, can I pick either BSD or GPLv2 (dual-licensed) or do the terms of both apply simultaneously (double licensed)?

[sergey-dryabzhinsky]

You can agree only with one of them.
They are mutually exclusive.

[Cyan4973]

Indeed, dual license always implies an "or", you can select the one which better fits your needs.

[peterh]

Since you mean "or", could you please change the text to use the word "or"? Right now, all the source files read "and", which doesn't mean the same thing.

The other dual-licensed projects I could think of off the top of my head (Perl5, jQuery circa 2012) use "either … or" language. I've also seen "or (at your option)" for dual/multi licensed projects (eg. in the default GPL boiler plate).

[Cyan4973]

The license text at the top of each file will be changed to this content :

/**
* Copyright (c) .....
* All rights reserved.
*
* This source code is licensed under both the BSD-style license (found in the
* LICENSE file in the root directory of this source tree) and the GPLv2 (found
* in the COPYING file in the root directory of this source tree).
* You may select, at your option, one of the above-listed licenses.
*/

The last line is added to make it explicit that it's one license or the other.

[nemequ]

Facebook's legal signed off on that? IANAL, but that sounds like it's licensed under both or you can choose one of the two.

I would have expected something more like

This source code is licensed under either a BSD-style license (found in the LICENSE file in the root directory of this source tree) or, at your option, the GPLv2 (found in the COPYING file in the root directory of this source tree).

I guess if the lawyers are good with the first version it's not a problem…

[Cyan4973]

All license texts come straight from legal dept

[peterh]

Much better, thanks. Still a little unusual as @nemequ points out, but at least it's clearer now what is intended.

[Zeeman206]

@Cyan4973
Zstd in the performance to bring us a lot of surprises, thank you for your generous contribution.
But because of the limitations of the memory we use, we can not use the GPL license software (create multiple processes will lead to more memory consumption, and performance degradation).

We can see you have modified the zstd.h in here:
https://github.com/facebook/zstd/blob/dev/lib/zstd.h
/* You may select, at your option, one of the above-listed licenses.
*/

But here is still the old version in the archive, we can only use both BSD and GPL V2 license.
https://github.com/facebook/zstd/archive/v1.3.1.zip

**1. What is your plan to modify in the 13.1 version, or will be modified in 1.3.2?

1. For the GPL license, do we need to delete any GPL files or source code?**

Thank you！

[Conan-Kudo]

@Zeeman206 Are you trying to say that you couldn't use it as a process because you're trying to make proprietary software with it? Just say that instead.

Anyway, yes @Cyan4973, a release with the new terms would be helpful.

[Cyan4973]

The added sentence will be present in next release.
It is merely a clarifying sentence, it doesn't change the term of the license. Existing text in current release means the same : you can select whichever license suits your needs, like any other dual-license. So you can use it safely in your software.