Bigchaindb: Merge bigchaindb-common repo into the bigchaindb repo

[r-marques]

TODO:

• [x] merge the code inside a subfolder of bigchaindb bigchaindb/common -- see PR https://github.com/bigchaindb/bigchaindb/pull/731 -- see PR https://github.com/bigchaindb/bigchaindb/pull/732
• [x] merge the tests inside a subfolder of tests tests/common
• [x] fix all the tests and problems with the merge
• [x] move all issues from bigchaindb-common into bigchaindb
• [x] move the docs to ?

[r-marques]

Assigned the label common to all imported issues so its easier for us to track what was imported from bigchaindb-common

[ttmc]

I don't think we need to bring the docs from the bigchaindb-common repo. AFAIK they were just auto-generated by cookiecutter. I did add a ← Back to All BigchainDB Docs link, but I think that's the only change I made. You can see the rendered bigchaindb-common docs at:

https://bigchaindb-common.readthedocs.io/en/latest/

The intro page (imported from the README) has a short list of features that Tim probably wrote but we don't need that either.

[sohkai]

@sbellem I believe this is finished now?

[sbellem]

@sohkai indeed

[sbellem]

Some notes on how this merge was done, especially regarding:

• merge the code inside a subfolder of bigchaindb bigchaindb/common
• merge the tests inside a subfolder of tests tests/common

We'll assume the following directory structure:

$ ls
bigchaindb           common-implementation     common-tests

Cloning the repos to yield the above:

$ git clone [email protected]:bigchaindb/bigchaindb
$ git clone [email protected]:bigchaindb/bigchaindb-common common-implementation
$ git clone [email protected]:bigchaindb/bigchaindb-common common-tests

Have three shells, one for each repo:

shell 1: bigchaindb

$ cd bigchaindb/

shell 2: common-implementation (bigchaindb-common)

$ cd common-implementation/

shell 3: common-tests (bigchaindb-common)

$ cd common-tests/

---

shell 2: common-implementation (bigchaindb-common)

$ git filter-branch -f --subdirectory-filter bigchaindb_common/ -- --all
$ ls
crypto.py  exceptions.py  __init__.py  transaction.py  util.py
$ mkdir -p bigchaindb/common
$ git mv *.py bigchaindb/common/
$ git commit -am 'Extract common lib implementation'

shell 1: bigchaindb

$ git remote add common-implementation ../common-implementation/
$ git fetch common-implementation 
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 6 (delta 1), reused 1 (delta 1)
Unpacking objects: 100% (6/6), done.
From ../common-implementation
 * [new branch]      master     -> common-implementation/master
$ git checkout -b extract-common-implementation master
$ git pull common-implementation master
$ git rebase master

shell 3: common-tests (bigchaindb-common)

$ git filter-branch -f --subdirectory-filter tests -- --all
$ git filter-branch -f --index-filter 'git rm -f --cached --ignore-unmatch conftest.py bigchaindb/transaction.py __init__.py'
$ mkdir -p tests/common
$ git mv *.py tests/common/
$ git commit -am 'Extract common lib tests'

shell 1: bigchaindb

$ git remote add common-tests ../common-tests/
$ git fetch common-tests
$ git pull common-tests master
$ git rebase -i master
# fixup duplicate Planning release commit

A few pointers

Using filter-branch has serious consequences, and the above steps resulted in a messy merge, with no ticket back home. See git-filter-branch documentation:

WARNING! The rewritten history will have different object names for all the objects and will not converge with the original branch. You will not be able to easily push and distribute the rewritten branch on top of the original branch. Please do not use this command if you do not know the full implications, and avoid using it anyway, if a simple single commit would suffice to fix your problem. (See the "RECOVERING FROM UPSTREAM REBASE" section in git-rebase[1] for further information about rewriting published history.)

In other words, don't use git filter-branch unless you have no other option. In our case it seemed we had no choice.

subtree merging may be a good alternative when it fits what one needs.

Thanks @sohkai for the help!

[ttmc]

Wow, that was non-trivial. Kudos