Dbt: Running with dbt=0.18.1 Encountered an error: The library Security could not be found

[jtalmi]

Describe the bug

Recently upgraded to Mac OS Big Sur and dbt now returns an error when calling dbt deps and other commands

Steps To Reproduce

On Big Sur, run dbt deps

Expected behavior

Screenshots and log output

If applicable, add screenshots or log output to help explain your problem.

System information

Which database are you using dbt with?

• [ ] postgres
• [ ] redshift
• [ ] bigquery
• [x ] snowflake
• [ ] other (specify: ____________)

The output of dbt --version:
```
installed version: 0.18.1
latest version: 0.18.1

Up to date!

Plugins:

• bigquery: 0.18.1
• snowflake: 0.18.1
• redshift: 0.18.1
• postgres: 0.18.1```

The operating system you're using:
Mac OS Big Sur
The output of python --version:
Python 3.7.1

Additional context

Did some googling and it may be an issue with snowflake and Big Sur compatibility.

[jtcohen6]

Hey @jtalmi, thanks for flagging this. I'm on Big Sur now and I haven't had any issues running dbt deps.

• How did you install dbt? What do you see when you run which dbt?
• Could you try to identify the version of the snowflake-connector-python library that you have installed for use with dbt?

[jtalmi]

Ah you are? Ok I saw some stuff related to Big Sur on stack overflow and I just upgraded so I made the assumption.

I installed dbt using pip:

$ which dbt
/Users/jonathantalmi/.pyenv/shims/dbt

$ pip list  | grep snowflake-connector-python
snowflake-connector-python 2.2.10

[jtcohen6]

Hmm ok, I just did a fresh pip install dbt in a virtualenv, using the same version of snowflake-connector-python (I think we pin it tightly), and I was able to run dbt deps. So there must be something else afoot...

[jtalmi]

If this helps, even after migrating to Big Sur I was able to run dbt 0.18.0 in a previous virtualenv I had installed, managed by pipenv. The pipfile is here:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
fire = "==0.2.1"
pandas = "*"
mock = "*"
permifrost = "*"
snapshottest = "==0.5.1"

[packages]
dbt = "==0.18.0"
python-dateutil = "==2.8.0"

[requires]
python_version = "3.7"

I tried to make changes to this pipfile but kept running into a cryptography dependency issue. Even now, when I try to generate a virtualenv using pipenv based on the following, it fails:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[packages]
dbt = "==0.18.1"

ERROR: Could not find a version that matches cryptography<3,>=2,>=3.2 (from -r /var/folders/w2/12yfgt212f7c6g_tzm_kp4vr0000gn/T/pipenvsc46ijcbrequirements/pipenv-mzhb5zvc-constraints.txt (line 2))
Tried: 0.1, 0.2, 0.2.1, 0.2.2, 0.3, 0.4, 0.5, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.6, 0.6.1, 0.7, 0.7.1, 0.7.2, 0.8, 0.8.1, 0.8.2, 0.9, 0.9.1, 0.9.2, 0.9.3, 1.0, 1.0.1, 1.0.2, 1.1, 1.1.1, 1.1.2, 1.2, 1.2.1, 1.2.2, 1.2.3, 1.3, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.4, 1.5, 1.5.1, 1.5.2, 1.5.3, 1.6, 1.7, 1.7.1, 1.7.2, 1.8, 1.8.1, 1.8.2, 1.9, 2.0, 2.0.1, 2.0.2, 2.0.3, 2.1, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2, 2.2, 2.2.1, 2.2.1, 2.2.2, 2.2.2, 2.3, 2.3, 2.3.1, 2.3.1, 2.4, 2.4, 2.4.1, 2.4.1, 2.4.2, 2.4.2, 2.5, 2.5, 2.6, 2.6.1, 2.6.1, 2.7, 2.7, 2.8, 2.8, 2.9, 2.9, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 3.0, 3.0, 3.1, 3.1, 3.1.1, 3.1.1, 3.2, 3.2, 3.2.1, 3.2.1
There are incompatible versions in the resolved dependencies:
  cryptography>=3.2 (from -r /var/folders/w2/12yfgt212f7c6g_tzm_kp4vr0000gn/T/pipenvsc46ijcbrequirements/pipenv-mzhb5zvc-constraints.txt (line 2))
  cryptography<3,>=2 (from dbt-snowflake==0.18.1->dbt==0.18.1->-r /var/folders/w2/12yfgt212f7c6g_tzm_kp4vr0000gn/T/pipenvsc46ijcbrequirements/pipenv-mzhb5zvc-constraints.txt (line 3))

See thread here: https://getdbt.slack.com/archives/C2JRRQDTL/p1607011130107300

So I ended up stripping away all the other dependencies and installing dbt to my local computer to test, and that's when I hit the Security issue

[jtcohen6]

Hmm. We are definitely upgrading cryptography in dbt v0.19.0, but I don't think we can do it sooner because of the upper bound in the pinned version of snowflake-connector-python here.

What version of pip are you using? For the time being, I wonder if you can use the legacy dependency resolver (--use-deprecated=legacy-resolver). For reference: https://blog.python.org/2020/11/pip-20-3-release-new-resolver.html

[jtalmi]

Retried using pip==20.2.1 (with the old dependency resolver) and still getting the same cryptography issue as above. As to my original issue...still no idea. Hopefully something pops up on stack overflow soon.

[jtalmi]

@jtcohen6 the stackoverflow gods smiled on me today. someone solved this by upgrading their version of oscrypto to 1.2.1:

https://stackoverflow.com/questions/65012259/snowsql-fails-with-the-library-security-could-not-be-found-after-upgrading-to

so i ran pip install oscrypto==1.2.1 and it works fine now.