Deeplearning4j: Eclipse: Need more information concerning intellectual property

Created on 20 Dec 2018 · 35Comments · Source: eclipse/deeplearning4j

As an Eclipse project, we need to have a clear and thorough understanding of the IP that is found in this repository. Most the files are under Skymind's copyright, but some are not. I am providing below a list of files and directories that contain third-party content, along with some of my guesses about it. Some have been included as is, some have been modified. The list is not necessarily complete, see issue #5878, for example. Please add any suspicious content to the bottom of the list. For each file or group of files, we need to be in a position to provide this information:

The name and the exact version of the project the files were copied from
A short description of that project
Cryptography? (probably "no")
The source of the content (the home page and/or the main source code repository, etc)
The short name(s) of the license(s)
Was it modified from the original? (If "yes", review by the IP team will take more time)

As a first step, I would like to have people who placed any given third-party content in this repository also write the required information below in comments. In time, any content that can't be accounted for will need to be either investigated or removed before we can move this repository to the Eclipse Foundation. Further, any third-party test resources must also be moved out into dl4j-test-resources.

Note: We will be able to keep the current content as part of archived branches.

Apart from those, there are still over 100 files missing copyright headers. We need them in all files that can accommodate comments. @sshepel?

For more information about the guidelines we need to follow as an Eclipse project:

[ ] modified nd4j/nd4j-serde/nd4j-aeron/src/main/java/org/nd4j/aeron/ipc/AeronUtil.java (Copyright 2014 - 2016 Real Logic Ltd.)
[x] modified nd4j/nd4j-common/src/main/java/org/nd4j/util/StringUtils.java (Copyright Apache Hadoop)
[x] modified nd4j/nd4j-common/src/main/java/org/nd4j/util/FingerPrintKeyer.java (Copyright 2010, Google Inc.)
[ ] 655 files probably unmodified in nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/protobuf/tf/ (Copyright 2008 Google Inc. and Copyright 2015-2017 The TensorFlow Authors.)
[ ] 3 file probably unmodified in nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/protobuf/onnx/ (Copyright (c) Facebook Inc.)
[x] modified libnd4j/include/semaphore.h (Copyright (c) 2011-2012, Archaea Software, LLC.)
[ ] 3 modified files in libnd4j/include/ops/declarable/generic/parity_ops/ (Copyright 2015 The TensorFlow Authors.)
[x] 1 file probably unmodified libnd4j/include/lapacke.h (Copyright (c) 2010, Intel Corp.)
[x] 7 modified files in libnd4j/include/ (Copyright 1993-2015 NVIDIA Corporation)
[x] 14 unmodified files in libnd4j/include/flatbuffers/ (Copyright 2014-2017 Google Inc.)
[ ] 2 modified files in libnd4j/include/cnpy/ (Copyright (C) 2011 Carl Rogers)
[x] 93 unmodified files in libnd4j/include/b40c/ (Copyright (c) 2010-2011, Duane Merrill, Copyright (c) 2011-2013, NVIDIA CORPORATION.)
[x] 9 modified and unmodified files in libnd4j/cmake/ (various copyrights)
[x] modified datavec/datavec-api/src/main/java/org/datavec/api/records/reader/impl/csv/SerializableCSVParser.java (Copyright 2005 Bytecode Pty Ltd.)
[x] 117 files probably unmodified in deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-resources/src/main/resources/ (Copyright 2013-2014, Facebook, Inc., Copyright 2011-2015 Twitter, Inc. and various others)
[x] 269 files probably unmodified in deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-play/src/main/resources/deeplearning4jUiAssets/ (Copyright 2013-2014, Facebook, Inc. Copyright 2011-2015 Twitter, Inc. and various others)
[x] 6 files probably unmodified in arbiter/arbiter-ui/src/main/resources/deeplearning4jUiAssets/ (Copyright 2011-2015 Twitter, Inc., Copyright jQuery Foundation and other contributors)
[ ] 53 files probably unmodified in deeplearning4j/deeplearning4j-nlp-parent/deeplearning4j-nlp-japanese/src/main/java/com/atilika/kuromoji/ (Copyright 2010-2015 Atilika Inc. and contributors)
[ ] 17 files probably unmodified in deeplearning4j/deeplearning4j-nlp-parent/deeplearning4j-nlp-chinese/src/main/resources/ (Copyright 2011-2016 ansj_seg)
[ ] 73 files probably unmodified in deeplearning4j/deeplearning4j-nlp-parent/deeplearning4j-nlp-chinese/src/main/java/org/ansj/ (Copyright 2011-2016 ansj_seg)
[x] probably unmodified ./deeplearning4j/deeplearning4j-nlp-parent/deeplearning4j-nlp/src/main/java/org/deeplearning4j/models/embeddings/learning/impl/elements/RandomUtils.java (Copyright ???)

Aha! Link: https://skymindai.aha.io/features/DL4J-33

DL4J Documentation Eclipse IP

Source

saudet

All 35 comments

b40c is removed in my axis pr. we're not using it anymore due to data types.

raver119 on 20 Dec 2018

👍1

Our autogen docs generator here is loosely based on Keras' autogen. MIT licensed project, available at keras.io.

Based on commit hash: df03bb5b1cc9fd297b0f19e08d916a4faedea267
License: MIT
Modiied: yes - large parts have been rewritten

maxpumperla on 20 Dec 2018

datavec/datavec-api/src/main/java/org/datavec/api/records/reader/impl/csv/SerializableCSVParser.java

Project + Version: OpenCSV 2.3
Description: CSV parsing library
Cryptography: no
Homepage: http://opencsv.sourceforge.net/ - source (note: current version, not 2.3) - https://sourceforge.net/p/opencsv/source/ci/master/tree/src/main/java/com/opencsv/CSVParser.java
Licence: Apache 2.0
Modified: yes - only a few tiny modifications, see https://gist.github.com/AlexDBlack/a0fbdd2085f3fcd0e12da6e4143549b6

AlexDBlack on 20 Dec 2018

👍1

@maxpumperla We'll need to know the exact version you took (like a git hash even) and also please let me know the license if you know it since they have not specified it in the file! Thanks

saudet on 20 Dec 2018

@saudet updated the comment, thanks

maxpumperla on 20 Dec 2018

👍1

@sshepel The files that are missing copyright headers are these ones, at least, but there's probably a bit more lying around:

**/README.md
arbiter/arbiter-core/src/main/java/org/deeplearning4j/arbiter/optimize/parameter/BooleanSpace.java
deeplearning4j/deeplearning4j-nn/src/main/java/org/deeplearning4j/util/OutputLayerUtil.java
jumpy/jumpy/keras_model.py
jumpy/pytest.ini
jumpy/release.sh
jumpy/requirements.txt
jumpy/setup.cfg
libnd4j/blas/CMakeLists.txt
libnd4j/blas/cpu/NDArray.macro
libnd4j/include/graph/generated/** 
libnd4j/include/graph/scheme/array.fbs
libnd4j/include/graph/scheme/config.fbs
libnd4j/include/graph/scheme/graph.fbs
libnd4j/include/graph/scheme/node.fbs
libnd4j/include/graph/scheme/properties.fbs
libnd4j/include/graph/scheme/request.fbs
libnd4j/include/graph/scheme/result.fbs
libnd4j/include/graph/scheme/utils.fbs
libnd4j/include/graph/scheme/variable.fbs
libnd4j/include/loops/cuda/scalar.kernels
libnd4j/include/ops/declarable/generic/broadcastable/template.tpl
libnd4j/include/types/u64.h
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/loss/LossReduce.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/samediff/serde/FlatBuffersMapper.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/samediff/serde/LegacyOpMapper.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/samediff/TrainingConfig.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/graph/**
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/impl/loss/AbsoluteDifferenceLoss.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/impl/loss/BaseLoss.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/impl/loss/CosineDistanceLoss.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/impl/loss/HingeLoss.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/impl/loss/HuberLoss.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/impl/loss/LogLoss.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/impl/loss/MeanPairwiseSquaredErrorLoss.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/impl/loss/MeanSquaredErrorLoss.java
nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/util/NDArrayPreconditionsFormat.java
nd4j/nd4j-common/src/main/java/org/nd4j/api/loader/FileBatch.java
nd4j/nd4j-common/src/main/java/org/nd4j/base/PreconditionsFormat.java
nd4j/nd4j-common/src/main/java/org/nd4j/jackson/objectmapper/holder/ObjectMapperHolder.java
nd4j/nd4j-serde/nd4j-grpc/pom.xml
nd4j/nd4j-serde/nd4j-grpc/src/main/java/org/nd4j/graph/grpc/GraphInferenceServerGrpc.java
nd4j/nd4j-tensorflow/src/main/java/org/nd4j/tensorflow/conversion/graphrunner/SavedModelConfig.java

@AlexDBlack @raver119 Can we get FlatBuffers to produce copyright headers for the files we generate with it? It's not critical, but the IP team would be happy if we could easily do that.

saudet on 20 Dec 2018

Re: DL4J and Arbiter JS/CSS resources, as noted on the call today, we might be able to use webjars (which, Eclipse paperwork aside, may be a cleaner solution anyway)
https://github.com/deeplearning4j/deeplearning4j/issues/6913

AlexDBlack on 21 Dec 2018

Hey @saudet,

I have created a PR (https://github.com/deeplearning4j/deeplearning4j/pull/6918) for required changes, also please look at comments below:

Not sure that copyright header should be added to following files:

[x] **/README.md
[x] jumpy/requirements.txt
[x] jumpy/setup.cfg

Auto-generated files (copyright headers were not updated):

[ ] libnd4j/include/graph/generated/**
[ ] nd4j/nd4j-serde/nd4j-grpc/src/main/java/org/nd4j/graph/grpc/GraphInferenceServerGrpc.java

Other files:

[x] libnd4j/include/loops/cuda/scalar.kernels - missing in git
[x] libnd4j/include/types/u64.h - has been already fixed

sshepel on 26 Dec 2018

@sshepel We can add copyright headers in README.md files using the HTML syntax, try it out!
As for generated files, let's keep them out until we figure out a way to coerce the generators.

@maxpumperla @farizrahman4u Does requirements.txt and setup.cfg support some sort of syntax for comments?

saudet on 27 Dec 2018

@saudet Lines starting with # are ignored for both requirements.txt and setup.cfg.

farizrahman4u on 27 Dec 2018

👍1

@farizrahman4u have you even seen any copyright headers in requirements.txt or setup.cfg?
Do we really need them there?

sshepel on 27 Dec 2018

Eclipse isn't requiring copyright headers for the README.md files of its other popular projects:
https://raw.githubusercontent.com/eclipse/jetty.project/jetty-9.4.x/README.md
https://raw.githubusercontent.com/eclipse/che/master/README.md
So let's not worry about that anymore. As for requirements.txt, setup.cfg, sure, let's leave them out too. The important issue that needs to be resolved first is to obtain the above information about all the third-party content that remains in this repository, unless it is removed first.

saudet on 28 Dec 2018

@sshepel Its not conventional to have copyright headers there, since there is no IP in it.

farizrahman4u on 29 Dec 2018

@maxpumperla We'll need to add the license information from Keras somewhere in this repository and have a way to connect it to that file. I guess we could start adding stuff to our main LICENSE file as per Keras itself: https://github.com/keras-team/keras/blob/master/LICENSE Sounds ok?

saudet on 9 Jan 2019

@AlexDBlack Can you also do that for CSVParser.java? In its case, since the license is the same as ours we wouldn't need to modify the LICENSE file much, just adding a copyright line should be enough:

 Copyright {yyyy} {name of copyright owner}

And Skymind should be there as well, so let's do it while we're at it :)

saudet on 9 Jan 2019

@saudet yeah, sounds good.

maxpumperla on 9 Jan 2019

👍1

@maxpumperla Please send a pull request! We all need to practice doing this :)

saudet on 10 Jan 2019

@saudet Done: https://github.com/deeplearning4j/deeplearning4j/pull/6977

AlexDBlack on 11 Jan 2019

👍1

Migration of most DL4J/Arbiter UI resources to webjars was completed here: https://github.com/deeplearning4j/deeplearning4j/pull/7046

Some 3rd party resources remain, however. See the aforementioned PR comments for some details.

AlexDBlack on 22 Jan 2019

🎉1

@AlexDBlack @raver119 These files don't seem to be used anywhere, can we remove them?

nd4j/nd4j-serde/nd4j-aeron/src/main/java/org/nd4j/aeron/util/AeronStat.java
nd4j/nd4j-serde/nd4j-aeron/src/main/java/org/nd4j/aeron/util/LogInspector.java
nd4j/nd4j-common/src/main/java/org/nd4j/util/FingerPrintKeyer.java
libnd4j/include/lapacke.h
libnd4j/include/semaphore.h
libnd4j/cmake/Check*
libnd4j/cmake/CMake*
libnd4j/cmake/Find*
libnd4j/cmake/OpenBLAS*

saudet on 8 Apr 2019

👍1

libnd4j/cmake/Check*
libnd4j/cmake/CMake*
libnd4j/cmake/Find*
libnd4j/cmake/OpenBLAS*

These are used in build procedures. Files are coming from cmake distro.

libnd4j/include/semaphore.h
libnd4j/include/lapacke.h

These are not used, and not going to be used.

nd4j/nd4j-serde/nd4j-aeron/src/main/java/org/nd4j/aeron/util/AeronStat.java
nd4j/nd4j-serde/nd4j-aeron/src/main/java/org/nd4j/aeron/util/LogInspector.java

These shouldn't be used.

nd4j/nd4j-common/src/main/java/org/nd4j/util/FingerPrintKeyer.java

Not aware of any uses, but if we need that - we could just bring in dependency i guess?

raver119 on 8 Apr 2019

These are used in build procedures. Files are coming from cmake distro.

Right, they come with CMake so we don't need to have a copy, so let's remove them.

saudet on 8 Apr 2019

👍1

@raver119 I'm pretty sure all code from NVIDIA found in libnd4j isn't going to pass muster with Eclipse:

grep -ri NVIDIA *
cuda_launch_config.h: *  Copyright 2008-2012 NVIDIA Corporation
fp16_conversion.hpp:// Paulius Micikevicius ([email protected])
fp16_emu.h: * Copyright 1993-2014 NVIDIA Corporation.  All rights reserved.
fp16_emu.h: * subject to NVIDIA intellectual property rights under U.S. and
fp16_emu.h: * CONFIDENTIAL to NVIDIA and is being provided under the terms and
fp16_emu.h: * conditions of a form of NVIDIA software license agreement by and
fp16_emu.h: * between NVIDIA and Licensee ("License Agreement") or electronically
fp16_emu.h: * written consent of NVIDIA is prohibited.
fp16_emu.h: * LICENSE AGREEMENT, NVIDIA MAKES NO REPRESENTATION ABOUT THE
fp16_emu.h: * NVIDIA DISCLAIMS ALL WARRANTIES WITH REGARD TO THESE LICENSED
fp16_emu.h: * LICENSE AGREEMENT, IN NO EVENT SHALL NVIDIA BE LIABLE FOR ANY
helper_cuda.h: * Copyright 1993-2012 NVIDIA Corporation.  All rights reserved.
helper_cuda.h: * Please refer to the NVIDIA end user license agreement (EULA) associated
helpers/sharedmem.h: * Copyright 1993-2009 NVIDIA Corporation.  All rights reserved.
helpers/sharedmem.h: * NVIDIA Corporation and its licensors retain all intellectual property and
helpers/sharedmem.h: * agreement from NVIDIA Corporation is strictly prohibited.
helper_string.h: * Copyright 1993-2012 NVIDIA Corporation.  All rights reserved.
helper_string.h: * Please refer to the NVIDIA end user license agreement (EULA) associated
types/float16.h: Half-precision data type, based on NVIDIA code: https://github.com/NVIDIA/caffe/tree/experimental/fp16

What can we do about that?

saudet on 8 Apr 2019

One option is to make these items "optional" and have a user download it separately. You're not allowed to redistribute it, but AFAIK, you are allowed to ship code than can connect to it.

A quick glace show's they are all CPP items, so not part of the Java code per-se. It should already be partially decoupled.

TheMatthew on 8 Apr 2019

@TheMatthew The issue is that the files themselves are in this repository, something Eclipse won't allow unless they are under an acceptable license, which is not something that NVIDIA typically provides. Anyway, @raver119 said we don't need those files. They will eventually disappear.

saudet on 9 Apr 2019

@AlexDBlack The only third-party content left from the deeplearning4j-ui-resources and deeplearning4jUiAssets directories seems to be uploadify.css and style.css, is this correct?

saudet on 10 Apr 2019

Ah, no, there's still a bunch of files with no attributions. Do you have an idea of where they might have been copied from?

saudet on 10 Apr 2019

Most of these came from the DL4J UI theme we purchased some time back ('metro dashboard' theme I think it was called)

AlexDBlack on 10 Apr 2019

Yeah, looks like this going to be a hassle. Unless we can find the exact terms we received those files under, they will need to go, and even if we do find those terms, they are obviously not open source so we would need Board Approval from the Eclipse Foundation to keep them here. Sounds like a lot of trouble for one theme and a couple of icons... Let's find some random open source one we can swap in?

saudet on 11 Apr 2019

Right, that's pretty much the way I expected this would go.
I don't think it'll be too hard to do that, but there's still probably at least a day of work in it...

https://github.com/deeplearning4j/deeplearning4j/issues/7537

AlexDBlack on 12 Apr 2019

I've updated the LICENSE file in commit https://github.com/deeplearning4j/deeplearning4j/commit/87653fff9a5f035c00d6d5178765dabd27d6f5f5 to include the content that we'll probably keep. We still need to figure out from which exact versions of their codes the content was taken from, but maybe the IP team will give us a pass on that. We can have them take a look as soon as we get the rest done, that is:

[x] Replace the proprietary UI theme with CoreUI (#7537)
[x] Remove last remaining content from NVIDIA in libnd4j/include/ with something else
[ ] @sshepel Could you do a last pass at the repository for any new files missing copyright headers?

Thank you all! We're almost there, let's get this done.

saudet on 7 May 2019

👍1

fp16/nv stuff https://github.com/deeplearning4j/deeplearning4j/pull/7682

raver119 on 7 May 2019

👍1

DL4J UI is basically done - will be merged to dev branch soon.
https://github.com/deeplearning4j/deeplearning4j/pull/7683

AlexDBlack on 7 May 2019

👍1

@AlexDBlack Can we have that in the master branch though?

saudet on 7 May 2019

Open a PR for latest header changes... https://github.com/deeplearning4j/deeplearning4j/pull/7690