Pipenv: Dockerfile with pipenv takes a long time
Hello pipenv community,
In a larger project I have a dockerfile heavily based on pipenv and it takes forever to build. There are other dockerfiles during the make process, but this specific one really takes so much time, and it makes no sense.
I would appreciate your suggestions of how to improve it.
ARG REGISTRY=presidio.azurecr.io
ARG PRESIDIO_DEPS_LABEL=latest
FROM ${REGISTRY}/presidio-python-deps:${PRESIDIO_DEPS_LABEL}
ARG NAME=presidio-analyzer
WORKDIR /usr/bin/${NAME}
ADD ./${NAME} /usr/bin/${NAME}
RUN pipenv install --dev --sequential && \
pipenv run pylint analyzer && \
pipenv run flake8 analyzer --exclude "*pb2*.py" && \
pipenv run pytest --log-cli-level=0
#----------------------------
FROM ${REGISTRY}/presidio-python-deps:${PRESIDIO_DEPS_LABEL}
ARG NAME=presidio-analyzer
ADD ./${NAME}/analyzer /usr/bin/${NAME}/analyzer
WORKDIR /usr/bin/${NAME}/analyzer
CMD pipenv run python __main__.py serve --env-grpc-port
The dockerfile is taken from Presidio project and it's part of the make process.
Here is the --support data (sorry it's so long)
$ pipenv --support
Pipenv version: '2018.11.26'
Pipenv location: '/usr/local/Cellar/pipenv/2018.11.26_2/libexec/lib/python3.7/site-packages/pipenv'
Python location: '/usr/local/Cellar/pipenv/2018.11.26_2/libexec/bin/python3.7'
Python installations found:
3.7.3:/Users/ityer/.local/share/virtualenvs/presidio-analyzer-F1SRmeOI/bin/python33.7.3:/usr/local/bin/python33.7.3:/usr/local/bin/python3.7m2.7.10:/usr/bin/python2.7.10:/usr/bin/pythonw2.7.10:/usr/bin/python2.7
PEP 508 Information:
{'implementation_name': 'cpython',
'implementation_version': '3.7.3',
'os_name': 'posix',
'platform_machine': 'x86_64',
'platform_python_implementation': 'CPython',
'platform_release': '18.6.0',
'platform_system': 'Darwin',
'platform_version': 'Darwin Kernel Version 18.6.0: Thu Apr 25 23:16:27 PDT '
'2019; root:xnu-4903.261.4~2/RELEASE_X86_64',
'python_full_version': '3.7.3',
'python_version': '3.7',
'sys_platform': 'darwin'}
System environment variables:
PATHTERM_PROGRAMPRESIDIO_LABELPIP_PYTHON_PATHSHELLTERMTMPDIRApple_PubSub_Socket_RenderTERM_PROGRAM_VERSIONTERM_SESSION_IDZSHUSERCOMMAND_MODESSH_AUTH_SOCK__CF_USER_TEXT_ENCODINGVIRTUAL_ENVPAGERLSCOLORSPIPENV_ACTIVE_PWDITERM_PROFILEXPC_FLAGSPYTHONDONTWRITEBYTECODEXPC_SERVICE_NAMECOLORFGBGHOMESHLVLITERM_SESSION_IDLOGNAMELESSPIP_DISABLE_PIP_VERSION_CHECKLC_CTYPEDOCKER_REGISTRYGOPATHSECURITYSESSIONIDCOLORTERMPIP_SHIMS_BASE_MODULEPYTHONFINDER_IGNORE_UNSUPPORTED
Pipenv鈥搒pecific environment variables:
PIPENV_ACTIVE:1
Debug鈥搒pecific environment variables:
PATH:/usr/local/Cellar/pipenv/2018.11.26_2/libexec/tools:/Users/ityer/.local/share/virtualenvs/presidio-analyzer-F1SRmeOI/bin:/Users/ityer/go/bin:/bin:/usr/local/Cellar/pipenv/2018.11.26_2/libexec/tools:/Users/ityer/go/bin:/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/binSHELL:/bin/zshPWD:/Users/ityer/go/src/github.com/Microsoft/presidio/presidio-analyzerVIRTUAL_ENV:/Users/ityer/.local/share/virtualenvs/presidio-analyzer-F1SRmeOI
Contents of Pipfile ('/Users/ityer/go/src/github.com/Microsoft/presidio/presidio-analyzer/Pipfile'):
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"
[packages]
cython = "*"
spacy = "*"
en_core_web_lg = {file = "https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-2.1.0/en_core_web_lg-2.1.0.tar.gz"}
regex = "*"
pyre2 = {file = "https://github.com/torosent/pyre2/archive/release/0.2.23.zip"}
grpcio = "*"
protobuf = "*"
tldextract = "*"
knack = "*"
[dev-packages]
pytest = "*"
flake8= "*"
pylint = {version = "==2.3.1"}
Contents of Pipfile.lock ('/Users/ityer/go/src/github.com/Microsoft/presidio/presidio-analyzer/Pipfile.lock'):
{
"_meta": {
"hash": {
"sha256": "be7df2b6a129090a66e0049544cdeae3425b032a6333d2d2991aa8e0e26725d2"
},
"pipfile-spec": 6,
"requires": {},
"sources": [
{
"name": "pypi",
"url": "https://pypi.python.org/simple",
"verify_ssl": true
}
]
},
"default": {
"argcomplete": {
"hashes": [
"sha256:59909d0ce5be1a46e2fb4e4fa5b714f6d605151ce88c468afb42d800879a6e6d",
"sha256:94423d1a56cdec2ef47699e02c9a48cf8827b9c4465b836c0cefb30afe85e59a"
],
"version": "==1.9.5"
},
"blis": {
"hashes": [
"sha256:039129410a338be8db8cf48c54334bd7c30da7e72bad2741e59313b1d242814b",
"sha256:058f9109aaea9d4f88cb623a44994d96c8cf36448de3e1bd30210628d6b52e9e",
"sha256:278d7b95e56cf82a6bef91cd8283eadc9401f2d3bdbbf2cdfdb605cf9081c36e",
"sha256:2d4ca1508fd6229c7994fc17ba324083a5b83f66612c8ea62623a41a1768b030",
"sha256:51a54bad6175e9b154beeb628a879ed492ee2247c9e40c77bdf6fc772145130c",
"sha256:886b313f96d4e268a0587e98c1637d963c73defa8de51e2e6b0d0bd00f16afbb",
"sha256:9f12e6f1e4b10dbb1e0e34e98f60e8435058a60d544a009cb761351fe1d12cad",
"sha256:a54d4fa1908d586f8bce9851a453cb89d1542e9aca65b8b88e9bb9432d626f80",
"sha256:b9d6cef13d95e3752320cd942df25e09160a6f9dfc3d7b41af7cdc772ab18270",
"sha256:d571464d195a950e60bf1547c8914d4da50952e06a0f38cea7b0829d0a4b985a",
"sha256:d616d64c85e6be92d69a1410dc58146cb9603fd1eb148f9ee512b8fddfd789f6",
"sha256:e477c7eaacf7dcccbb190a29559579efb287ecf5c2a9a7a6f9acb0452899f033",
"sha256:e6ae1986625af86f90f111f9d2d284b9e45fddfe56cf40524cdd9417a6a33b87"
],
"version": "==0.2.4"
},
"certifi": {
"hashes": [
"sha256:59b7658e26ca9c7339e00f8f4636cdfe59d34fa37b9b04f6f9e9926b3cece1a5",
"sha256:b26104d6835d1f5e49452a26eb2ff87fe7090b89dfcaee5ea2212697e1e1d7ae"
],
"version": "==2019.3.9"
},
"chardet": {
"hashes": [
"sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
"sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
],
"version": "==3.0.4"
},
"colorama": {
"hashes": [
"sha256:05eed71e2e327246ad6b38c540c4a3117230b19679b875190486ddd2d721422d",
"sha256:f8ac84de7840f5b9c4e3347b3c1eaa50f7e49c2b07596221daec5edaabbd7c48"
],
"version": "==0.4.1"
},
"cymem": {
"hashes": [
"sha256:081c652ae1aff4759813e93a2fc4df4ba410ce214a0e542988e24c62110d4cd0",
"sha256:0e447fa4cb6dccd0b96257a798370a17bef3ec254a527230058e41816a777c04",
"sha256:2c8267dcb15cc6ab318f01ceaf16b8440c0386ae44014d5b22fefe5b0398d05c",
"sha256:46141111eedbb5b0d8c9386b00226a15f5727a1202b9095f4363d425f259267e",
"sha256:4994c1f3e948bd58a6e38c905221680563b851983a15f1f01e5ff415d560d153",
"sha256:584872fd3df176e50c90e37aaca6cb731ac0abcdea4f5b8ad77c30674cfaaa99",
"sha256:6e3194135b21bb268030f3473beb8b674b356c330a9fa185dced2f5006cbd5ba",
"sha256:71710ee0e946a6bd33c86dd9e71f95ad584c65e8bb02615f00ceb0d8348fb303",
"sha256:741957f541fb8322de5a8c711d5d58f80d684225d2aec32fec92484cac931a52",
"sha256:7f01ba6153427811cd7d35630081c69b32c188a1d330599a826ef3bf17edbd7c",
"sha256:8d96e95902e781950d7c255b19364a1ed50a204843d63dd386b0abc5e6df5e44",
"sha256:8dd169ece1629ec4db1a592321e3ae0a9bb62fda2052a351fc36871f314c3569",
"sha256:8e6ad29636edd559b0dfe0a19c5cb5e6257461a5df90839e8c7710ddb005f4b4",
"sha256:9935b233882732f03fd0fadbeb9e9aa672edcdd126e6d52c36d60adf1def8ea5",
"sha256:a38b3229782411e4b23240f5f90000c4e7a834af88ed8763c66f8e4603db6b51",
"sha256:a5966b3171bad9c84a2b19dccda5ab37ae8437c0709a6b72cb42b64ea76a4bd3",
"sha256:ab88b1534f06df07262d9bc5efb3ba07948cdbe9a363eb9eaa4ad42fae6c7b5e",
"sha256:b08b0dd7adafbff9f0fd7dc8dcad5f3ce6f23c126c81ad8d1666880cc94e6974",
"sha256:ba47b571d480c0b76d282ff1634372070031d4998a46ae5d8305d49563b74ca6",
"sha256:bf049dc9cf0d3aa4a48ba514b7f1699fb6f35b18ad8c6f018bd13e0bccd9d30c",
"sha256:c46a122c524a3270ac5249f590ac2f75f1a83692a3d3a03479cea49de72a0a89",
"sha256:c63337aa7e1ad4ec182cc7847c6d85390589fbbf1f9f67d1fde8133a9acb7fa8",
"sha256:ec51273ea08a2c6389bc4dd6b5183354826d916b149a041f2f274431166191bc"
],
"version": "==2.0.2"
},
"cython": {
"hashes": [
"sha256:0ce8f6c789c907472c9084a44b625eba76a85d0189513de1497ab102a9d39ef8",
"sha256:0d67964b747ac09758ba31fe25da2f66f575437df5f121ff481889a7a4485f56",
"sha256:1630823619a87a814e5c1fa9f96544272ce4f94a037a34093fbec74989342328",
"sha256:1a4c634bb049c8482b7a4f3121330de1f1c1f66eac3570e1e885b0c392b6a451",
"sha256:1ec91cc09e9f9a2c3173606232adccc68f3d14be1a15a8c5dc6ab97b47b31528",
"sha256:237a8fdd8333f7248718875d930d1e963ffa519fefeb0756d01d91cbfadab0bc",
"sha256:28a308cbfdf9b7bb44def918ad4a26b2d25a0095fa2f123addda33a32f308d00",
"sha256:2fe3dde34fa125abf29996580d0182c18b8a240d7fa46d10984cc28d27808731",
"sha256:30bda294346afa78c49a343e26f3ab2ad701e09f6a6373f579593f0cfcb1235a",
"sha256:33d27ea23e12bf0d420e40c20308c03ef192d312e187c1f72f385edd9bd6d570",
"sha256:34d24d9370a6089cdd5afe56aa3c4af456e6400f8b4abb030491710ee765bafc",
"sha256:4e4877c2b96fae90f26ee528a87b9347872472b71c6913715ca15c8fe86a68c9",
"sha256:50d6f1f26702e5f2a19890c7bc3de00f9b8a0ec131b52edccd56a60d02519649",
"sha256:55d081162191b7c11c7bfcb7c68e913827dfd5de6ecdbab1b99dab190586c1e8",
"sha256:59d339c7f99920ff7e1d9d162ea309b35775172e4bab9553f1b968cd43b21d6d",
"sha256:6cf4d10df9edc040c955fca708bbd65234920e44c30fccd057ecf3128efb31ad",
"sha256:6ec362539e2a6cf2329cd9820dec64868d8f0babe0d8dc5deff6c87a84d13f68",
"sha256:7edc61a17c14b6e54d5317b0300d2da23d94a719c466f93cafa3b666b058c43b",
"sha256:8e37fc4db3f2c4e7e1ed98fe4fe313f1b7202df985de4ee1451d2e331332afae",
"sha256:b8c996bde5852545507bff45af44328fa48a7b22b5bec2f43083f0b8d1024fd9",
"sha256:bf9c16f3d46af82f89fdefc0d64b2fb02f899c20da64548a8ea336beefcf8d23",
"sha256:c1038aba898bed34ab1b5ddb0d3f9c9ae33b0649387ab9ffe6d0af677f66bfc1",
"sha256:d405649c1bfc42e20d86178257658a859a3217b6e6d950ee8cb76353fcea9c39",
"sha256:db6eeb20a3bd60e1cdcf6ce9a784bc82aec6ab891c800dc5d7824d5cfbfe77f2",
"sha256:e382f8cb40dca45c3b439359028a4b60e74e22d391dc2deb360c0b8239d6ddc0",
"sha256:f3f6c09e2c76f2537d61f907702dd921b04d1c3972f01d5530ef1f748f22bd89",
"sha256:f749287087f67957c020e1de26906e88b8b0c4ea588facb7349c115a63346f67",
"sha256:f86b96e014732c0d1ded2c1f51444c80176a98c21856d0da533db4e4aef54070"
],
"index": "pypi",
"version": "==0.29.7"
},
"en-core-web-lg": {
"file": "https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-2.1.0/en_core_web_lg-2.1.0.tar.gz"
},
"grpcio": {
"hashes": [
"sha256:0442f7d0c527ceab6a76159937ae8109941eace90ec00cb1bd08fc4f3179e52e",
"sha256:051957d0f61f4dec90868a54ee969228409926a0a19fd8ed7b4a0e50388effee",
"sha256:0d262794b2339770d5378a5717f8ddbfb68e409974582f0503272b90b7cc79bd",
"sha256:142693dc8bd427c595d030f75bf8d01c843d9ccb659499e8507ad22da832e9cf",
"sha256:18d44515a3fd3a71442abb5a1c65fc1909d859c13cda50c974cbc69742a80cea",
"sha256:1d50674bdffa18ea6143e0df9a1b97cdeab583ce5dd1cabda3502ee75215065c",
"sha256:3945335a5b8332995415c5f03da1a5f6e36da6ede819a611e2cbb093cf752bdd",
"sha256:3a9603ff14070524f4c69634afad6b280b07ad9f8c2c346c4b2290306e1928ac",
"sha256:52861aac5c1dcf4c841eb555b257cfb56d0c840a286495078382f538d0a34d6a",
"sha256:53c512c7c8af9cb9e3e1cc5ce5e4a5fb2f2e7695e69219f90016bc602abe2f3b",
"sha256:57ea92c9b81015e5f2cc355e53f08a4e661b78a207857311c7b8c55137a43b29",
"sha256:5f8574c9e42d1917e41cdedc6312682a96e4547114c7bb0f3de125199a58b3d6",
"sha256:638ff1a45dd7a226b2b9390296a111142363fe2b5503499f3987d599bce0683c",
"sha256:64fe0dc897f1f19a6500948862857cb3b97247be997bc47b4dbade42f8af5f97",
"sha256:67920ec7d2de89845e5232aed41271ef53e1a362c8ffb84f6a6c6e644a75ce3a",
"sha256:714cddc170efeedf6312d8534ef7f52dcf20dd8f5fb7c5e425c2b6819ac1b9ec",
"sha256:7edf33e929b1666ff68bfc280b9021a862ab423d0e6306889cc2bc7c907dfc27",
"sha256:84eb47b1a47e206e78f453fb92a155ed0d18d2ca8747f5c67e4b50b9c37180a7",
"sha256:8a6289e5c38318cba75115f0bf88be166ead40c83c10dd81ace52f1ab5dc1eab",
"sha256:8bd5b8c3c8872da748dc8810b664699a5f1d49f2c9ab2b205b96ec9fe06741ad",
"sha256:93e7672348d4c68ac570c499a794ff4453a1928c39cbe708472a0e1b77176411",
"sha256:9d37fb214674f0f194a80df5ad0b9c9b9f2fa5c5408ceaf0fc796e57588404d9",
"sha256:9de6746a749634004499bac773ad9877d84d826aca2dc14ba4ebd3cd9f64ed74",
"sha256:9e530c69d6e566ca985193a63363af36a7560a23f4979df6e392bb1bdf05caed",
"sha256:b37f36da8f4d0bf07d53eb34395b68f5e0dc0bcee207affde9ba29bbf6bd6ced",
"sha256:cf9b57d139e44eab294ab31eb0181150d877440a8a321bb4422e2c09f6c7a7d9",
"sha256:dd716aab42be3d1fde74577e42b6319b6399b07d418e49b653e0e1bcd88399bc",
"sha256:dea43aa864edc3b3d8de1f6e40144119fbccdf04525b3ece4fef9392b6eed436",
"sha256:e6cbd27559ff91c98991b8ec4ef19f394bf9056d6897aabb9af79568307181d3",
"sha256:f58e3377da8e8e453068dffc00d17691a97ffd1c3a5a7460b890cf83a9ca6edf",
"sha256:f938fdfb780a0658d04e1d727b4fb470490087c56cb31ba75cb54fb4bea515bd",
"sha256:fee4accad7a113004aef226b851f0494c01fc8d281fdebd74468f19cc45354a0"
],
"index": "pypi",
"version": "==1.20.1"
},
"idna": {
"hashes": [
"sha256:c357b3f628cf53ae2c4c05627ecc484553142ca23264e593d327bcde5e9c3407",
"sha256:ea8b7f6188e6fa117537c3df7da9fc686d485087abf6ac197f9c46432f7e4a3c"
],
"version": "==2.8"
},
"jmespath": {
"hashes": [
"sha256:3720a4b1bd659dd2eecad0666459b9788813e032b83e7ba58578e48254e0a0e6",
"sha256:bde2aef6f44302dfb30320115b17d030798de8c4110e28d5cf6cf91a7a31074c"
],
"version": "==0.9.4"
},
"jsonschema": {
"hashes": [
"sha256:000e68abd33c972a5248544925a0cae7d1125f9bf6c58280d37546b946769a08",
"sha256:6ff5f3180870836cae40f06fa10419f557208175f13ad7bc26caa77beb1f6e02"
],
"version": "==2.6.0"
},
"knack": {
"hashes": [
"sha256:2a4b4d86c4700dd6714e5b4ca6bbca6baf2c827d9de28ca2b66640988c1b6ff4",
"sha256:7f17d4a1b34ea76821d3504f5f0f8c1b75bd9f08497db6a5864677214ac76adc"
],
"index": "pypi",
"version": "==0.6.1"
},
"murmurhash": {
"hashes": [
"sha256:27b908fe4bdb426f4e4e4a8821acbe0302915b2945e035ec9d8ca513e2a74b1f",
"sha256:33405103fa8cde15d72ee525a03d5cfe2c7e4901133819754810986e29627d68",
"sha256:386a9eed3cb27cb2cd4394b6521275ba04552642c2d9cab5c9fb42aa5a3325c0",
"sha256:3af36a0dc9f13f6892d9b8b39a6a3ccf216cae5bce38adc7c2d145677987772f",
"sha256:717196a04cdc80cc3103a3da17b2415a8a5e1d0d578b7079259386bf153b3258",
"sha256:8a4ed95cd3456b43ea301679c7c39ade43fc18b844b37d0ba0ac0d6acbff8e0c",
"sha256:a6c071b4b498bcea16a8dc8590cad81fa8d43821f34c74bc00f96499e2527073",
"sha256:b0afe329701b59d02e56bc6cee7325af83e3fee9c299c615fc1df3202b4f886f",
"sha256:ba766343bdbcb928039b8fff609e80ae7a5fd5ed7a4fc5af822224b63e0cbaff",
"sha256:bf33490514d308bcc27ed240cb3eb114f1ec31af031535cd8f27659a7049bd52",
"sha256:c7a646f6b07b033642b4f52ae2e45efd8b80780b3b90e8092a0cec935fbf81e2",
"sha256:d696c394ebd164ca80b5871e2e9ad2f9fdbb81bd3c552c1d5f1e8ee694e6204a",
"sha256:fe344face8d30a5a6aa26e5acf288aa2a8f0f32e05efdda3d314b4bf289ec2af"
],
"version": "==1.0.2"
},
"numpy": {
"hashes": [
"sha256:0e2eed77804b2a6a88741f8fcac02c5499bba3953ec9c71e8b217fad4912c56c",
"sha256:1c666f04553ef70fda54adf097dbae7080645435fc273e2397f26bbf1d127bbb",
"sha256:1f46532afa7b2903bfb1b79becca2954c0a04389d19e03dc73f06b039048ac40",
"sha256:315fa1b1dfc16ae0f03f8fd1c55f23fd15368710f641d570236f3d78af55e340",
"sha256:3d5fcea4f5ed40c3280791d54da3ad2ecf896f4c87c877b113576b8280c59441",
"sha256:48241759b99d60aba63b0e590332c600fc4b46ad597c9b0a53f350b871ef0634",
"sha256:4b4f2924b36d857cf302aec369caac61e43500c17eeef0d7baacad1084c0ee84",
"sha256:54fe3b7ed9e7eb928bbc4318f954d133851865f062fa4bbb02ef8940bc67b5d2",
"sha256:5a8f021c70e6206c317974c93eaaf9bc2b56295b6b1cacccf88846e44a1f33fc",
"sha256:754a6be26d938e6ca91942804eb209307b73f806a1721176278a6038869a1686",
"sha256:771147e654e8b95eea1293174a94f34e2e77d5729ad44aefb62fbf8a79747a15",
"sha256:78a6f89da87eeb48014ec652a65c4ffde370c036d780a995edaeb121d3625621",
"sha256:7fde5c2a3a682a9e101e61d97696687ebdba47637611378b4127fe7e47fdf2bf",
"sha256:80d99399c97f646e873dd8ce87c38cfdbb668956bbc39bc1e6cac4b515bba2a0",
"sha256:88a72c1e45a0ae24d1f249a529d9f71fe82e6fa6a3fd61414b829396ec585900",
"sha256:a4f4460877a16ac73302a9c077ca545498d9fe64e6a81398d8e1a67e4695e3df",
"sha256:a61255a765b3ac73ee4b110b28fccfbf758c985677f526c2b4b39c48cc4b509d",
"sha256:ab4896a8c910b9a04c0142871d8800c76c8a2e5ff44763513e1dd9d9631ce897",
"sha256:abbd6b1c2ef6199f4b7ca9f818eb6b31f17b73a6110aadc4e4298c3f00fab24e",
"sha256:b16d88da290334e33ea992c56492326ea3b06233a00a1855414360b77ca72f26",
"sha256:b78a1defedb0e8f6ae1eb55fa6ac74ab42acc4569c3a2eacc2a407ee5d42ebcb",
"sha256:cfef82c43b8b29ca436560d51b2251d5117818a8d1fb74a8384a83c096745dad",
"sha256:d160e57731fcdec2beda807ebcabf39823c47e9409485b5a3a1db3a8c6ce763e"
],
"version": "==1.16.3"
},
"plac": {
"hashes": [
"sha256:854693ad90367e8267112ffbb8955f57d6fdeac3191791dc9ffce80f87fd2370",
"sha256:ba3f719a018175f0a15a6b04e6cc79c25fd563d348aacd320c3644d2a9baf89b"
],
"version": "==0.9.6"
},
"preshed": {
"hashes": [
"sha256:0c9af79c7b825793f987d477627efb81afd23384ac791bebbc88a257342a77ab",
"sha256:0ebc79431154bc5d12f97b3c93bc350af941702a44f0761dfcd395e970d693f8",
"sha256:102e71dc841c979b2ece44ab05b2b0aa39c8039493ddac40dd22cf23e2484063",
"sha256:15145b24eded01426544be829a6395d6c99e2d62f5f3b88a6e19087ebeef7237",
"sha256:195674dfb4bcf18b26e448feaabdf61adcf028ae69ecaa075c0bdfaf62a19671",
"sha256:38f7fbef59f89d3b2c8c3b102f9a7360cd73a33c829fdeb101c615b18ecc4686",
"sha256:3aa411233dc230247ea4c4558062e5b2d59d41c697107a45fddbfe03e63f3e77",
"sha256:3b8c7b607e6dce0843544cfe4f05355db0516fce8eca0c37d6b5f4f3680493bf",
"sha256:4bda4153d46a603bc6ea65380dfa091d46700f664cb906c7f26a469be6c2a503",
"sha256:541d7ed765d67512d6f9fa24fd01cc1d7a51c7ff2646362924f4db46813b485a",
"sha256:593d23b9f851ae7a4d519ca4489dd2b352d833e08f5d35795d42a591b8badb54",
"sha256:7f6fb8f4108abe958af892847ed50abe6f45aaf45a87853cc8154a7203e75d84",
"sha256:7ff7f18af1f19ea666ac4fbf48842e6acd900fbfdc26bb9aad02f353ff932386",
"sha256:9c0d503d8693bf1e08e0fa1cecbcd3253146abaa9a7501d7d583a72edd29fdd1",
"sha256:9cefe818a97134c0ddf22ef76fced1c841ebd137c2895251c5d1310276c234b5",
"sha256:9e603916a95dc524081d54c0a135611e6f68d787185d5df2b5ab3f076c3d1bd4",
"sha256:a2acacceac79aa6d4b65125e20c7de78fbca1340a251854c87967acef1795490",
"sha256:a3d592e7b265b4faf08c9b4d7493b9e8604e0ba8858cc9bd8c9aee41d3df2a3a",
"sha256:b2030e68c6f539e6dd7bfcea032940042739ef05d50a2eb1d7af24e038971b0f",
"sha256:bc894dc14d8567a5d6a1cded0a701da7fbb360b2124237fe8acde85333825aef",
"sha256:c21d4d10cc0248ba3facbbbfbe63211ce921478a3d5db6de34de39ee1b3484e1",
"sha256:dae01c74313965c487e0ec839e5f28d0c7df9bfd1d978aa5bada3f72ff20a9e5",
"sha256:ee8068035684a4b382bebb3a3f270799360545baff9742b85e627a0a889e6850"
],
"version": "==2.0.1"
},
"protobuf": {
"hashes": [
"sha256:21e395d7959551e759d604940a115c51c6347d90a475c9baf471a1a86b5604a9",
"sha256:57e05e16955aee9e6a0389fcbd58d8289dd2420e47df1a1096b3a232c26eb2dd",
"sha256:67819e8e48a74c68d87f25cad9f40edfe2faf278cdba5ca73173211b9213b8c9",
"sha256:75da7d43a2c8a13b0bc7238ab3c8ae217cbfd5979d33b01e98e1f78defb2d060",
"sha256:78e08371e236f193ce947712c072542ff19d0043ab5318c2ea46bbc2aaebdca6",
"sha256:7ee5b595db5abb0096e8c4755e69c20dfad38b2d0bcc9bc7bafc652d2496b471",
"sha256:86260ecfe7a66c0e9d82d2c61f86a14aa974d340d159b829b26f35f710f615db",
"sha256:92c77db4bd33ea4ee5f15152a835273f2338a5246b2cbb84bab5d0d7f6e9ba94",
"sha256:9c7b90943e0e188394b4f068926a759e3b4f63738190d1ab3d500d53b9ce7614",
"sha256:a77f217ea50b2542bae5b318f7acee50d9fc8c95dd6d3656eaeff646f7cab5ee",
"sha256:ad589ed1d1f83db22df867b10e01fe445516a5a4d7cfa37fe3590a5f6cfc508b",
"sha256:b06a794901bf573f4b2af87e6139e5cd36ac7c91ac85d7ae3fe5b5f6fc317513",
"sha256:bd8592cc5f8b4371d0bad92543370d4658dc41a5ccaaf105597eb5524c616291",
"sha256:be48e5a6248a928ec43adf2bea037073e5da692c0b3c10b34f9904793bd63138",
"sha256:cc5eb13f5ccc4b1b642cc147c2cdd121a34278b341c7a4d79e91182fff425836",
"sha256:cd3b0e0ad69b74ee55e7c321f52a98effed2b4f4cc9a10f3683d869de00590d5",
"sha256:d6e88c4920660aa75c0c2c4b53407aef5efd9a6e0ca7d2fc84d79aba2ccbda3a",
"sha256:ec3c49b6d247152e19110c3a53d9bb4cf917747882017f70796460728b02722e"
],
"index": "pypi",
"version": "==3.7.1"
},
"pygments": {
"hashes": [
"sha256:5ffada19f6203563680669ee7f53b64dabbeb100eb51b61996085e99c03b284a",
"sha256:e8218dd399a61674745138520d0d4cf2621d7e032439341bc3f647bff125818d"
],
"version": "==2.3.1"
},
"pyre2": {
"file": "https://github.com/torosent/pyre2/archive/release/0.2.23.zip"
},
"pyyaml": {
"hashes": [
"sha256:1adecc22f88d38052fb787d959f003811ca858b799590a5eaa70e63dca50308c",
"sha256:436bc774ecf7c103814098159fbb84c2715d25980175292c648f2da143909f95",
"sha256:460a5a4248763f6f37ea225d19d5c205677d8d525f6a83357ca622ed541830c2",
"sha256:5a22a9c84653debfbf198d02fe592c176ea548cccce47553f35f466e15cf2fd4",
"sha256:7a5d3f26b89d688db27822343dfa25c599627bc92093e788956372285c6298ad",
"sha256:9372b04a02080752d9e6f990179a4ab840227c6e2ce15b95e1278456664cf2ba",
"sha256:a5dcbebee834eaddf3fa7366316b880ff4062e4bcc9787b78c7fbb4a26ff2dd1",
"sha256:aee5bab92a176e7cd034e57f46e9df9a9862a71f8f37cad167c6fc74c65f5b4e",
"sha256:c51f642898c0bacd335fc119da60baae0824f2cde95b0330b56c0553439f0673",
"sha256:c68ea4d3ba1705da1e0d85da6684ac657912679a649e8868bd850d2c299cce13",
"sha256:e23d0cc5299223dcc37885dae624f382297717e459ea24053709675a976a3e19"
],
"version": "==5.1"
},
"regex": {
"hashes": [
"sha256:020429dcf9b76cc7648a99c81b3a70154e45afebc81e0b85364457fe83b525e4",
"sha256:0552802b1c3f3c7e4fee8c85e904a13c48226020aa1a0593246888a1ac55aaaf",
"sha256:308965a80b92e1fec263ac1e4f1094317809a72bc4d26be2ec8a5fd026301175",
"sha256:4d627feef04eb626397aa7bdec772774f53d63a1dc7cc5ee4d1bd2786a769d19",
"sha256:93d1f9fcb1d25e0b4bd622eeba95b080262e7f8f55e5b43c76b8a5677e67334c",
"sha256:c3859bbf29b1345d694f069ddfe53d6907b0393fda5e3794c800ad02902d78e9",
"sha256:d56ce4c7b1a189094b9bee3b81c4aeb3f1ba3e375e91627ec8561b6ab483d0a8",
"sha256:ebc5ef4e10fa3312fa1967dc0a894e6bd985a046768171f042ac3974fadc9680",
"sha256:f9cd39066048066a4abe4c18fb213bc541339728005e72263f023742fb912585"
],
"index": "pypi",
"version": "==2019.4.14"
},
"requests": {
"hashes": [
"sha256:502a824f31acdacb3a35b6690b5fbf0bc41d63a24a45c4004352b0242707598e",
"sha256:7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b"
],
"version": "==2.21.0"
},
"requests-file": {
"hashes": [
"sha256:75c175eed739270aec3c5279ffd74e6527dada275c5c0d76b5817e9c86bb7dea",
"sha256:8f04aa6201bacda0567e7ac7f677f1499b0fc76b22140c54bc06edf1ba92e2fa"
],
"version": "==1.4.3"
},
"six": {
"hashes": [
"sha256:3350809f0555b11f552448330d0b52d5f24c91a322ea4a15ef22629740f3761c",
"sha256:d16a0141ec1a18405cd4ce8b4613101da75da0e9a7aec5bdd4fa804d0e0eba73"
],
"version": "==1.12.0"
},
"spacy": {
"hashes": [
"sha256:0fe2e5905f2f5b41be3ebea40626f70bea567a7a2cda9c244109fffe8d964429",
"sha256:30f0f09074bf115a0384691e8ba3d64aab431192b3095a13312a93d0e8a71c07",
"sha256:6a82612f0e75c11d541002f49375d80b4800c967e5d2b402d5a8dd40b6c57ae6",
"sha256:74066ac969a587d16d00d65318c1baa3c3e9215e6858d0c81ce2823320fe09dc",
"sha256:b1b86ddf6142fa2782b2e0269d040430ae5696eb0224f3e99408897cac7bb506",
"sha256:be8a7c89461ac22d261e19e1d3eb35752d8ff3e52452af076b303561bb166408",
"sha256:e6522e1242a5a5f12ef7e55f74df020b5deea59f7d1e7b6e69298301e3c0badd",
"sha256:eb699f54bf6d131df701e6dbbef9e91b74a065a42c9d2850964282b3c14560bb",
"sha256:f385942c5b2c8cf07e4a56871f88a49d4c8a9145fcd731c455e39fb5af9b12ba"
],
"index": "pypi",
"version": "==2.1.3"
},
"srsly": {
"hashes": [
"sha256:02ea974c4b80f9ffdea4f953ffece5a8715e4e4b37d09192ab65cf4edfbf74d1",
"sha256:061ade35556e51b2e1da6f8552be7a6327d2d02b69edf0aacc9f5c4319d495f1",
"sha256:1bf6af7a86f34969a3997da09fc8c2f72ee02cd74ff40035e37c2f968776fa23",
"sha256:1e4ef85bf133e384f465865ba4e0a14a52c4f2e4b46c763faf100339a06f09c4",
"sha256:850399e43f4cefdcac7a913363b120ea084cb02fcfdbbde1bd37444804d7def4",
"sha256:977aa6e5fd3f7e9d1c8fe7aeed841dfe3ede75dfce04255d4c670e663faaef2a",
"sha256:abdc5b46866648b123517550582dc4c4b767b816ae54c44e5973bbebc3f0dab4",
"sha256:ac0dbe6e715e1fe3536397a9e65ec8f3c624c99f45b6f30e87d220071ef84721",
"sha256:b8646f0f7cf6fd1de4919ab456d9c030e09e74f741a0cecc941363414109ccdc",
"sha256:b9dc81339c1ab969057e790d7b2a56fd4da87336785bd671c86520e8272e3663",
"sha256:d7c91f59edc2ceeca70adf1b0a46d337234ff4fb7ca2b579ca41885f011b329f",
"sha256:d906a2a3df1cac2cb4bf382b8aaf14e22df2ca3758eba0d3049723c851c8ebf0",
"sha256:ecec49c9cdaae4594011666dd654e1e044e552f63bb3a62a1849c65a92ee302e",
"sha256:ef7897050c04a313f2db99c9bcaf2f0c3c75609677683ca5a6e1e7a515325d72"
],
"version": "==0.0.5"
},
"tabulate": {
"hashes": [
"sha256:8af07a39377cee1103a5c8b3330a421c2d99b9141e9cc5ddd2e3263fea416943"
],
"version": "==0.8.3"
},
"thinc": {
"hashes": [
"sha256:12c003b804fb93c64261a5010df0129f942234adb8f45d489a355a5315e06acf",
"sha256:17f9ada01f1f77a5560bc16ec5a650dca08356b50727ded0df19f0dfb4a32a25",
"sha256:26c9d54ffd90753feebbc462ae59939a9e3d2485ef24ed3dc1861c9b486fdbbe",
"sha256:3258161fc2cefa4082f099dec3748f1dcef5e920df5e9d82258ea6ffec280b9a",
"sha256:38a83b928cdc49c994852538f639b2a889681a0589c44b1a6fc3c899e5f36893",
"sha256:3e76101a733bbb0b97d44bdbcb407678b9e2b487047acb6f4c19b72909a6b12f",
"sha256:412f107c458d2951711b4d3ec53587244cd3acc032944e855f49cf94a1adc36e",
"sha256:4948c10c61e627950900cdccf506eb7398d2b28f33cf72bb4b5d9c5c572925e7",
"sha256:a8b2d7713a7dfc0b18b5c16db58ab6e015df14e4fbed0249ed49e630b2d6a86f",
"sha256:ec99c2c65962157c7ee7b947d29f2775291860b81cba62c5bd9f92fdeca2d137",
"sha256:f2386e66042218f19e511692926cef00a9646a3104d2efddfb5bec7b0388a83b",
"sha256:fc0b37733591315afddee45823d4f6740f9b0567c1ba57a3a3c319669d1fcbad"
],
"version": "==7.0.4"
},
"tldextract": {
"hashes": [
"sha256:2c1c5d9d454f79734b4f3da0d603856dd9f820753410a3e9abf0a0c9fde33e97",
"sha256:b72bef6013de67c7fa181250bc2c2e089a994d259c09ca95a9771f2f97e29ed1"
],
"index": "pypi",
"version": "==2.2.1"
},
"tqdm": {
"hashes": [
"sha256:d385c95361699e5cf7622485d9b9eae2d4864b21cd5a2374a9c381ffed701021",
"sha256:e22977e3ebe961f72362f6ddfb9197cc531c9737aaf5f607ef09740c849ecd05"
],
"version": "==4.31.1"
},
"urllib3": {
"hashes": [
"sha256:2393a695cd12afedd0dcb26fe5d50d0cf248e5a66f75dbd89a3d4eb333a61af4",
"sha256:a637e5fae88995b256e3409dc4d52c2e2e0ba32c42a6365fee8bbd2238de3cfb"
],
"version": "==1.24.3"
},
"wasabi": {
"hashes": [
"sha256:b4fbee9dd0c8f5cff6554c0463c565e2d52b7c844d7eccb477d29a6ff8567750",
"sha256:f92c83e728bf1db6dc859ffc861afa328d2da8ef0c7a19300e5fb1bd5762b277"
],
"version": "==0.2.2"
}
},
"develop": {
"astroid": {
"hashes": [
"sha256:6560e1e1749f68c64a4b5dee4e091fce798d2f0d84ebe638cf0e0585a343acf4",
"sha256:b65db1bbaac9f9f4d190199bb8680af6f6f84fd3769a5ea883df8a91fe68b4c4"
],
"version": "==2.2.5"
},
"atomicwrites": {
"hashes": [
"sha256:03472c30eb2c5d1ba9227e4c2ca66ab8287fbfbbda3888aa93dc2e28fc6811b4",
"sha256:75a9445bac02d8d058d5e1fe689654ba5a6556a1dfd8ce6ec55a0ed79866cfa6"
],
"version": "==1.3.0"
},
"attrs": {
"hashes": [
"sha256:69c0dbf2ed392de1cb5ec704444b08a5ef81680a61cb899dc08127123af36a79",
"sha256:f0b870f674851ecbfbbbd364d6b5cbdff9dcedbc7f3f5e18a6891057f21fe399"
],
"version": "==19.1.0"
},
"entrypoints": {
"hashes": [
"sha256:589f874b313739ad35be6e0cd7efde2a4e9b6fea91edcc34e58ecbb8dbe56d19",
"sha256:c70dd71abe5a8c85e55e12c19bd91ccfeec11a6e99044204511f9ed547d48451"
],
"version": "==0.3"
},
"flake8": {
"hashes": [
"sha256:859996073f341f2670741b51ec1e67a01da142831aa1fdc6242dbf88dffbe661",
"sha256:a796a115208f5c03b18f332f7c11729812c8c3ded6c46319c59b53efd3819da8"
],
"index": "pypi",
"version": "==3.7.7"
},
"isort": {
"hashes": [
"sha256:1349c6f7c2a0f7539f5f2ace51a9a8e4a37086ce4de6f78f5f53fb041d0a3cd5",
"sha256:f09911f6eb114e5592abe635aded8bf3d2c3144ebcfcaf81ee32e7af7b7d1870"
],
"version": "==4.3.18"
},
"lazy-object-proxy": {
"hashes": [
"sha256:0ce34342b419bd8f018e6666bfef729aec3edf62345a53b537a4dcc115746a33",
"sha256:1b668120716eb7ee21d8a38815e5eb3bb8211117d9a90b0f8e21722c0758cc39",
"sha256:209615b0fe4624d79e50220ce3310ca1a9445fd8e6d3572a896e7f9146bbf019",
"sha256:27bf62cb2b1a2068d443ff7097ee33393f8483b570b475db8ebf7e1cba64f088",
"sha256:27ea6fd1c02dcc78172a82fc37fcc0992a94e4cecf53cb6d73f11749825bd98b",
"sha256:2c1b21b44ac9beb0fc848d3993924147ba45c4ebc24be19825e57aabbe74a99e",
"sha256:2df72ab12046a3496a92476020a1a0abf78b2a7db9ff4dc2036b8dd980203ae6",
"sha256:320ffd3de9699d3892048baee45ebfbbf9388a7d65d832d7e580243ade426d2b",
"sha256:50e3b9a464d5d08cc5227413db0d1c4707b6172e4d4d915c1c70e4de0bbff1f5",
"sha256:5276db7ff62bb7b52f77f1f51ed58850e315154249aceb42e7f4c611f0f847ff",
"sha256:61a6cf00dcb1a7f0c773ed4acc509cb636af2d6337a08f362413c76b2b47a8dd",
"sha256:6ae6c4cb59f199d8827c5a07546b2ab7e85d262acaccaacd49b62f53f7c456f7",
"sha256:7661d401d60d8bf15bb5da39e4dd72f5d764c5aff5a86ef52a042506e3e970ff",
"sha256:7bd527f36a605c914efca5d3d014170b2cb184723e423d26b1fb2fd9108e264d",
"sha256:7cb54db3535c8686ea12e9535eb087d32421184eacc6939ef15ef50f83a5e7e2",
"sha256:7f3a2d740291f7f2c111d86a1c4851b70fb000a6c8883a59660d95ad57b9df35",
"sha256:81304b7d8e9c824d058087dcb89144842c8e0dea6d281c031f59f0acf66963d4",
"sha256:933947e8b4fbe617a51528b09851685138b49d511af0b6c0da2539115d6d4514",
"sha256:94223d7f060301b3a8c09c9b3bc3294b56b2188e7d8179c762a1cda72c979252",
"sha256:ab3ca49afcb47058393b0122428358d2fbe0408cf99f1b58b295cfeb4ed39109",
"sha256:bd6292f565ca46dee4e737ebcc20742e3b5be2b01556dafe169f6c65d088875f",
"sha256:cb924aa3e4a3fb644d0c463cad5bc2572649a6a3f68a7f8e4fbe44aaa6d77e4c",
"sha256:d0fc7a286feac9077ec52a927fc9fe8fe2fabab95426722be4c953c9a8bede92",
"sha256:ddc34786490a6e4ec0a855d401034cbd1242ef186c20d79d2166d6a4bd449577",
"sha256:e34b155e36fa9da7e1b7c738ed7767fc9491a62ec6af70fe9da4a057759edc2d",
"sha256:e5b9e8f6bda48460b7b143c3821b21b452cb3a835e6bbd5dd33aa0c8d3f5137d",
"sha256:e81ebf6c5ee9684be8f2c87563880f93eedd56dd2b6146d8a725b50b7e5adb0f",
"sha256:eb91be369f945f10d3a49f5f9be8b3d0b93a4c2be8f8a5b83b0571b8123e0a7a",
"sha256:f460d1ceb0e4a5dcb2a652db0904224f367c9b3c1470d5a7683c0480e582468b"
],
"version": "==1.3.1"
},
"mccabe": {
"hashes": [
"sha256:ab8a6258860da4b6677da4bd2fe5dc2c659cff31b3ee4f7f5d64e79735b80d42",
"sha256:dd8d182285a0fe56bace7f45b5e7d1a6ebcbf524e8f3bd87eb0f125271b8831f"
],
"version": "==0.6.1"
},
"more-itertools": {
"hashes": [
"sha256:2112d2ca570bb7c3e53ea1a35cd5df42bb0fd10c45f0fb97178679c3c03d64c7",
"sha256:c3e4748ba1aad8dba30a4886b0b1a2004f9a863837b8654e7059eebf727afa5a"
],
"markers": "python_version > '2.7'",
"version": "==7.0.0"
},
"pluggy": {
"hashes": [
"sha256:19ecf9ce9db2fce065a7a0586e07cfb4ac8614fe96edf628a264b1c70116cf8f",
"sha256:84d306a647cc805219916e62aab89caa97a33a1dd8c342e87a37f91073cd4746"
],
"version": "==0.9.0"
},
"py": {
"hashes": [
"sha256:64f65755aee5b381cea27766a3a147c3f15b9b6b9ac88676de66ba2ae36793fa",
"sha256:dc639b046a6e2cff5bbe40194ad65936d6ba360b52b3c3fe1d08a82dd50b5e53"
],
"version": "==1.8.0"
},
"pycodestyle": {
"hashes": [
"sha256:95a2219d12372f05704562a14ec30bc76b05a5b297b21a5dfe3f6fac3491ae56",
"sha256:e40a936c9a450ad81df37f549d676d127b1b66000a6c500caa2b085bc0ca976c"
],
"version": "==2.5.0"
},
"pyflakes": {
"hashes": [
"sha256:17dbeb2e3f4d772725c777fabc446d5634d1038f234e77343108ce445ea69ce0",
"sha256:d976835886f8c5b31d47970ed689944a0262b5f3afa00a5a7b4dc81e5449f8a2"
],
"version": "==2.1.1"
},
"pylint": {
"hashes": [
"sha256:5d77031694a5fb97ea95e828c8d10fc770a1df6eb3906067aaed42201a8a6a09",
"sha256:723e3db49555abaf9bf79dc474c6b9e2935ad82230b10c1138a71ea41ac0fff1"
],
"index": "pypi",
"version": "==2.3.1"
},
"pytest": {
"hashes": [
"sha256:3773f4c235918987d51daf1db66d51c99fac654c81d6f2f709a046ab446d5e5d",
"sha256:b7802283b70ca24d7119b32915efa7c409982f59913c1a6c0640aacf118b95f5"
],
"index": "pypi",
"version": "==4.4.1"
},
"six": {
"hashes": [
"sha256:3350809f0555b11f552448330d0b52d5f24c91a322ea4a15ef22629740f3761c",
"sha256:d16a0141ec1a18405cd4ce8b4613101da75da0e9a7aec5bdd4fa804d0e0eba73"
],
"version": "==1.12.0"
},
"typed-ast": {
"hashes": [
"sha256:132eae51d6ef3ff4a8c47c393a4ef5ebf0d1aecc96880eb5d6c8ceab7017cc9b",
"sha256:18141c1484ab8784006c839be8b985cfc82a2e9725837b0ecfa0203f71c4e39d",
"sha256:2baf617f5bbbfe73fd8846463f5aeafc912b5ee247f410700245d68525ec584a",
"sha256:3d90063f2cbbe39177e9b4d888e45777012652d6110156845b828908c51ae462",
"sha256:4304b2218b842d610aa1a1d87e1dc9559597969acc62ce717ee4dfeaa44d7eee",
"sha256:4983ede548ffc3541bae49a82675996497348e55bafd1554dc4e4a5d6eda541a",
"sha256:5315f4509c1476718a4825f45a203b82d7fdf2a6f5f0c8f166435975b1c9f7d4",
"sha256:6cdfb1b49d5345f7c2b90d638822d16ba62dc82f7616e9b4caa10b72f3f16649",
"sha256:7b325f12635598c604690efd7a0197d0b94b7d7778498e76e0710cd582fd1c7a",
"sha256:8d3b0e3b8626615826f9a626548057c5275a9733512b137984a68ba1598d3d2f",
"sha256:8f8631160c79f53081bd23446525db0bc4c5616f78d04021e6e434b286493fd7",
"sha256:912de10965f3dc89da23936f1cc4ed60764f712e5fa603a09dd904f88c996760",
"sha256:b010c07b975fe853c65d7bbe9d4ac62f1c69086750a574f6292597763781ba18",
"sha256:c908c10505904c48081a5415a1e295d8403e353e0c14c42b6d67f8f97fae6616",
"sha256:c94dd3807c0c0610f7c76f078119f4ea48235a953512752b9175f9f98f5ae2bd",
"sha256:ce65dee7594a84c466e79d7fb7d3303e7295d16a83c22c7c4037071b059e2c21",
"sha256:eaa9cfcb221a8a4c2889be6f93da141ac777eb8819f077e1d09fb12d00a09a93",
"sha256:f3376bc31bad66d46d44b4e6522c5c21976bf9bca4ef5987bb2bf727f4506cbb",
"sha256:f9202fa138544e13a4ec1a6792c35834250a85958fde1251b6a22e07d1260ae7"
],
"markers": "implementation_name == 'cpython'",
"version": "==1.3.5"
},
"wrapt": {
"hashes": [
"sha256:4aea003270831cceb8a90ff27c4031da6ead7ec1886023b80ce0dfe0adf61533"
],
"version": "==1.11.1"
}
}
}
Performance
itye-msft
All 5 comments
It typically happens when your dependencies include some scientific packages, read #3830 for knowledge of why it is difficult to improve. What you can do at the best now, is to 1) make sure the lockfile is correctly matching the content of Pipfile, so you won't need to lock during docker build, which is super slow, and 2) try another pypi mirror which is relatively faster on your machine, with --pypi-mirror option.
frostming
on 17 Jul 2019
👍1
TL;DR pipenv sync in docker build, not pipenv install
@itye-msft I have a similar issue, and have been tweaking my "pipenv in docker" pattern for years now. I solved this by pipenv syncing in the container build. I would tag the python dependency stage (--target in docker, and target in docker-compose.yml) so that when I do need to lock the Pipfile.lock to update/add to it, I could do it there, and deal with the lock time only when I want to update the lock file.
This both doesn't update packages by "surprise" during build, and it installs the exact versions specified in my lock file, all in addition to the time savings. (There's also the annoying issue that if the Pipfile.lock is updated during build, if you want a copy of that file, you need to perform some docker tricks)
Since I'm only locking at "run" time, I keep the pip and pipenv cache in an internal docker volume, so that the cache drastically speeds up subsequent locks too.
I can provide you my library that sets these things up if you require more information on my pattern.
andyneff
on 6 Aug 2019
👍4
@andyneff I need more information from you to build, can you post it?
928234269
on 13 Aug 2019
@928234269, I'll try... I have this "bash" system I use to build and run my dockers (called just). It's basically just a large case statement that executes commands I want (like a make file would). A lot of the rough edges of having all the various moving pieces are held together using this. I thought it was better to expose all those nasty little details rather than cut everything out I thought was irrelevant because there are usually a few interconnected pieces that I forget are important in the end.
Dockerfile
The Dockerfile has 4 relevant stages
- The dependency stage.
- Install all dependencies here that are needed to _run_ the final result. E.g. for gdal, you need
gdal-bin(dynamic files to link to at runtime).
- Install all dependencies here that are needed to _run_ the final result. E.g. for gdal, you need
- The pipenv dependency stage.
- Install all dependencies here that are needed to _build_ any pipenv packages. E.g. for gdal, you need the gdal devel, python devel, and g++ (plus a few other hacks).
- The important thing here is to
ADD/COPYthePipfileandPipfile.lockhere. This makes it so that when youdocker build/docker-compose buildafter those files have been updated, the cache is missed and thepipenv synccommand (next stage) is run again.
- The Pipenv cache stage.
- This populates the virtual env for the pipenv and also populates the pip/pipenv cache with downloaded packages. This speeds up lock and sync time when running a container. This is where
pipenv syncis used - The pipenv cache directory is moved in the first stage by setting
PIPENV_CACHE_DIR - The virtual env location is moved similarly using
WORKON_HOME - Source only pip packages are compiled here, and the resulting wheel is stored in the cache, to be used later if needed during other
pipenvcommands
- This populates the virtual env for the pipenv and also populates the pip/pipenv cache with downloaded packages. This speeds up lock and sync time when running a container. This is where
- The final stage is the runtime stage.
- Runtime dependencies and other docker image setup goes here. There is an example to install apps here too. The idea is dependencies that you need at run time go here, but if you change them, you don't create a cache miss in stage 1, and have to rebuild everything. This distinction decreases wasted build time for the developer.
- The effects from the virtualenv doesn't affect the runtime stage until the
COPY --from=pipenv_cache /venv /venv.
- This line is put after all major runtime image setup, so that updates to the venv folder don't waste time rebuilting the runtime image that are not affected by the pipenv environment.
- Commands after this are really quite and are most small files being copied in, so that when they change, the venv copy line isn't cache missed too, again to save time on average.
- The weakest part of this pattern is that as
/venvgets really big (10,000's of files?). Every time youdocker build, this line does a SHA check on all of those files. A lot of files can take a significant amount of time, even when there is no change. But I haven't had an environment so truly large that this check is a problem yet. If it was, I might start tarring/untarring things up to save time, but this is an untested future plan.
Dockerfile
FROM vsiri/recipe:gosu as gosu
FROM vsiri/recipe:tini as tini
FROM vsiri/recipe:vsi as vsi
FROM vsiri/recipe:pipenv as pipenv
###############################################################################
FROM debian:stretch as dep_stage
SHELL ["/usr/bin/env", "bash", "-euxvc"]
# Install any runtime dependencies
RUN apt-get update; \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
# # Uncomment for GDAL
# gdal-bin \
python3 tzdata; \
rm -r /var/lib/apt/lists/*
ENV \
# Move all virtualenvs to /venv
WORKON_HOME=/venv \
PIPENV_PIPFILE=/src/Pipfile \
# The pipenv cache is how we avoid recompiling packages at runtime
# when the build dependencies are no longer available
PIPENV_CACHE_DIR=/venv/cache \
# Needed for pipenv shell
PYENV_SHELL=/bin/bash \
# pipenv recommends that these env variables be set to en_US.UTF-8.
# On debian, C.UTF-8 exists by default instead, and seems to work.
# More detail: https://stackoverflow.com/a/38553499/1771778
# Note: en_US.UTF-8 exists by default in Centos/Fedora base images
LC_ALL=C.UTF-8 \
LANG=C.UTF-8
COPY --from=pipenv /tmp/pipenv /tmp/pipenv
RUN /tmp/pipenv/get-pipenv; rm -rf /tmp/pipenv || :
###############################################################################
FROM dep_stage as dep_pipenv
# # Uncomment for GDAL
# # Install any build dependencies
# RUN apt-get update; \
# DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
# libgdal-dev python3-dev g++ ; \
# rm -r /var/lib/apt/lists/*
#
# # GDAL specific hacks
# ENV CPLUS_INCLUDE_PATH=/usr/include/gdal \
# C_INCLUDE_PATH=/usr/include/gdal
ADD Pipfile Pipfile.lock /src/
# Simple packages can be added as dependencies to your project by:
# - Running the container and installing them with pipenv; e.g.,
# just pipenv install --keep-outdated scipy
# # install scipy without updating other packages
# Packages can also be added by
# - Editing the Pipfile and adding lines to the [packages] section
# scipy = "*"
# - Rebuiling the image
# just build
FROM dep_pipenv as pipenv_cache
# GDAL, for example, is a more complicated example
# - GDAL has extra build dependencies. The apt-get pattern above will install
# these dependencies.
# - GDAL's build script needs some customization. The ENV exports
# above accomplish this
# - GDAL needs numpy installed before it is built, otherwise, numpy
# integration will not be compiled
#
# To test this out:
# 1) Uncomment all the "Uncomment for GDAL" sections
# 2) just sync
# 3) Remove the old pipenv install section below
# 4) Edit your Pipfile and add the following lines (or similar)
# gdal = "==2.1.0"
# numpy = "*"
# This will add the latest version of numpy and the version of
# gdal compatible with debian:stretch to your pipenv environment.
# Note: the version of the pypi package should match (as closely as
# possible) to the version of the GDAL-binary dependency (gdal-bin)
# 5) just pipenv install
# 6) just sync (optionally update docker image)
#
# Steps 4 and 5 can also be done by:
# - just pipenv install numpy gdal==2.1.0
# - These can be combined because numpy is already installed at
# just sync time, so numpy was already installed before gdal
# - All other pipenv install arguments can be passed in, such as
# --keep-outdated, --pre, etc...
# # Uncomment for GDAL
# RUN \
# # Get the version marker of numpy specified in the lock file, else blank
# numpy_version="$(python -c "import json; print(json.load(open('${PIPENV_PIPFILE}.lock', 'r'))['default']['numpy']['version'])" 2>/dev/null)" || :; \
# # Install numpy first, so that the gdal install works.
# pipenv run pip install numpy${numpy_version}; \
# # Now install all the packages.
# pipenv sync; \
# rm -rf /src/* /tmp/pip*
RUN \
# Install all packages into the image
pipenv sync; \
# Cleanup and make way for the real /src that will be mounted at runtime
rm -rf /src/* /tmp/pip*
###############################################################################
FROM dep_stage
# Install any additional packages
# RUN apt-get update; \
# DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
# # Example of a package
# qbs-examples; \
# rm -rf /var/lib/apt/lists/*
# Another typical example of installing a package
# RUN build_deps="wget ca-certificates"; \
# apt-get update; \
# DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends ${build_deps}; \
# wget -q https://www.vsi-ri.com/bin/deviceQuery; \
# DEBIAN_FRONTEND=noninteractive apt-get purge -y --autoremove ${build_deps}; \
# rm -rf /var/lib/apt/lists/*
COPY --from=tini /usr/local/bin/tini /usr/local/bin/tini
COPY --from=gosu /usr/local/bin/gosu /usr/local/bin/gosu
# Allow non-privileged to run gosu (remove this to take root away from user)
RUN chmod u+s /usr/local/bin/gosu
COPY --from=pipenv_cache /venv /venv
# Everything after this is specific to my entrypoint, so you can ignore that
COPY --from=vsi /vsi /vsi
ADD bat.env /src/
ADD docker/example.Justfile /src/docker/
ENTRYPOINT ["/usr/local/bin/tini", "--", "/usr/bin/env", "bash", "/vsi/linux/just_entrypoint.sh"]
# Does not require execute permissions, unlike:
# ENTRYPOINT ["/usr/local/bin/tini", "--", "/vsi/linux/just_entrypoint.sh"]
CMD ["example-cmd"]
docker-compose.yml
Fair warning: I primarily use docker-compose to interactively run containers (not up daemon services). So it's a small stretch from the initial purpose, but it works really well. Everything here could be done with normal docker commands and tons of args, but this is how I do it.
Volume magic
The important thing to notice in the docker-compose.yml files is the second volume, /venv. This combines a number of docker features so that:
1. When you have an empty volume (when you start the example service the first time), the directory that was created in the image is copied to the internal docker volume. Now you have a persistent copy you can modify and work with.
- _Note_: This only works when the volume is empty (no files) A freshly created volume is always empty.
1. The first volume bind mounts the Pipfile/Pipfile.lock in. They are referenced to by the PIPENV_PIPFILE env var in the first stage of the Dockerfile
1. Now when you run pipenv install {some package} it updates the Pipfile, Pipfile.lock and adds to the venv, keeping everything in sync. There is no _need_ to rebuild an image, because 99% of the time everyone is happy. Some packages that don't follow proper PEP rules (looking at you gdal) break this, and need special "handling", as you can see in the comments of the Dockerfile of an example on how to handle that
2. _Advanced Note_ Double edge sword: In the container you are free to pip install into the virtual env and bypass pipenv. This should only be done as a "test" to see if a package will work for you or not. This causes the virtualenv to be out of sync with the Pipfile, but developer may really want this ability for quick tests. For this reason when I rebuild (sync) the docker image, I delete my venv volume, which clears it and makes it ready to copy the content upon a new container start and keeps everything in sync.
Services
There are a total of two docker-compose services
- The first service is the main runner
examplein this case. This is where you are to run programs that use your pipenv venv. - The second service
example_pipenvtargets thedep_pipenvstage. This means all the package build dependencies are there ready to be used, such as g++, etc...
- This is where you run
pipenv installthat both installs the packages (and cache) into the/venvvolume and updates yourPipfileandPipfile.lockThis is the only place where locking occurs. - Those packages are immediately ready to be used in the runner
examplecontainer (you don't even have to restart it, because they all share the/venvvolume). No need to rundocker/docker-compose build.
- This is where you run
Environment variables, etc... are there for my particular entrypoint that dynamically creates users with the right ID so file permissions match 100% of the time, so you can ignore that, although it does have the benifit of chowning the /venv dir too.
docker-compose.yml
version: "2.3"
services:
example: &example
build: &example_build
context: .
dockerfile: docker/example.Dockerfile
# prevent different users from clobbering each others images
image: ${BAT_DOCKER_REPO}:example_${BAT_USERNAME}
environment:
# Variables for just_entrypoint_functions
- DOCKER_UID=${BAT_UID}
- DOCKER_GIDS=${BAT_GIDS}
- DOCKER_GROUP_NAMES=${BAT_GROUP_NAMES}
- DOCKER_USERNAME=user
# - DOCKER_HOME=${BAT_HOME}
- DISPLAY
- JUSTFILE=/src/docker/example.Justfile
- JUST_SETTINGS=/src/bat.env
- TZ
# runtime: nvidia # Uncomment for nvidia gpu support
# cap_add:
# - SYS_PTRACE # Useful for gdb
volumes:
- type: ${BAT_SOURCE_DIR_TYPE}
source: ${BAT_SOURCE_DIR}
target: ${BAT_SOURCE_DIR_DOCKER}
- type: volume
source: venv
target: /venv
# - type: volume
# source: home-volume
# target: ${BAT_HOME} # home-volume should be overridable
example_pipenv:
<<: *example
build:
<<: *example_build
target: dep_pipenv
image: ${BAT_DOCKER_REPO}:example_pipenv_${BAT_USERNAME}
volumes:
venv:
labels:
com.vsi.just.clean_action: ask
# home-volume:
TL; I just want to try
If this is all too much and you want to just "try it" (Warning, this is not exactly _ready_ for prime time, so I apologize)
This will create a project skeleton of the files I'm talking about:
- Make a new directory, and cd into it
curl -LO https://raw.githubusercontent.com/VisionSystemsInc/vsi_common/master/linux/new_justbash ./new_just --defaults --setup-gitsource setup.env# Run once per bash session, basically just patches your pathjust sync# Run every time you want to update the docker images (aka every time the source code changes)just pipenv install django# Example of adding a new python package, using theexample_pipenvservicejust run example bash# Drops you into apipenv shell, using theexampleservicepython -c "import django as d; print(d.__version__)"# And that better match thePipfile.lock!
andyneff
on 14 Aug 2019
In my case I add
ENV PIPENV_CACHE_DIR=/path/of/your/choice
will help speeding up from my side
namgivu
on 21 Jan 2020
Was this page helpful?
0 / 5 - 0 ratings
Related issues
hynek
路
3Comments
bgjelstrup
路
3Comments
ipmb
路
3Comments
ansrivas
路
3Comments
Californian
路
3Comments
Most helpful comment
TL;DR
pipenv syncindocker build, notpipenv install@itye-msft I have a similar issue, and have been tweaking my "pipenv in docker" pattern for years now. I solved this by
pipenv syncing in the container build. I would tag the python dependency stage (--targetindocker, andtargetindocker-compose.yml) so that when I do need to lock thePipfile.lockto update/add to it, I could do it there, and deal with the lock time only when I want to update the lock file.This both doesn't update packages by "surprise" during build, and it installs the exact versions specified in my lock file, all in addition to the time savings. (There's also the annoying issue that if the
Pipfile.lockis updated during build, if you want a copy of that file, you need to perform some docker tricks)Since I'm only locking at "run" time, I keep the pip and pipenv cache in an internal docker volume, so that the cache drastically speeds up subsequent locks too.
I can provide you my library that sets these things up if you require more information on my pattern.