Kedro: Azure blob requires fsspec >= 0.8 and kedro is still on 0.7

[Lucianois]

Description

Azure blob with authentication via Shared Access Signature requires fsspec >= 0.8

Context

Kedro install does not compile, kedro viz does not work with fsspec 0.8

Steps to Reproduce

1. pip install -U fsspec
2. pip install adlfs
3. kedro run ( with azure blob dataset)

Expected Result

load data from Azure blob storage

Actual Result

  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/lucianoissoe/opt/anaconda3/lib/python3.7/site-packages/adlfs/__init__.py", line 1, in <module>
    from .spec import AzureDatalakeFileSystem
  File "/Users/lucianoissoe/opt/anaconda3/lib/python3.7/site-packages/adlfs/spec.py", line 21, in <module>
    from fsspec.asyn import (
ModuleNotFoundError: No module named 'fsspec.asyn'

kedro.io.core.DataSetError: 
No module named 'fsspec.asyn'.
Failed to instantiate DataSet 'safras' of type `kedro.extras.datasets.pandas.csv_dataset.CSVDataSet`.
Error: 
No module named 'fsspec.asyn'.
Failed to instantiate DataSet 'safras' of type `kedro.extras.datasets.pandas.csv_dataset.CSVDataSet`.

Your Environment

Include as many relevant details about the environment in which you experienced the bug:

• Kedro version used (pip show kedro or kedro -V):kedro, version 0.16.6
• Python version used (python -V):Python 3.7.6
• Operating system and version: Mac

[Lucianois]

this may be a duplicate from #489 and expected to be fixed in 0.17

[Minyus]

I wouldn't say this is a duplicate.

Despite the #489 comment, as of 4th Nov 2020, https://github.com/quantumblacklabs/kedro/blob/develop/requirements.txt#L5 sets fsspec>=0.5.1, <0.8, which means that current develop branch does not support fsspec >= 0.8 and cannot solve this Azure blob issue.

PyTorch Lightning also requires fsspec>=0.8 and got the same problem: https://github.com/quantumblacklabs/kedro/issues/553

[kaemo]

s3fs also, please update.

[lorenabalan]

Hi @Lucianois thanks for raising this. We are aware this is really inconvenient, but it will be made available in version 0.17.0. Commit https://github.com/quantumblacklabs/kedro/commit/151cbcd1acaae25da4633ab87fabbf354d60059b was pushed to develop branch.

[lorenabalan]

Closing this as resolved (in light of comment above).