Dvc: `dvc commit` fails with files imported via `dvc import`

[Cortysus]

Bug Report

Hello! I am importing some files in my main repo from another repo using the command dvc import.
Upon running dvc commit afterward, I get the following error (output of dvc commit -v):

2020-07-14 18:50:54,511 ERROR: unexpected error - 'NoneType' object has no attribute 'scheme'
------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/dvc/1.1.10/libexec/lib/python3.8/site-packages/dvc/main.py", line 53, in main
    ret = cmd.run()
  File "/usr/local/Cellar/dvc/1.1.10/libexec/lib/python3.8/site-packages/dvc/command/commit.py", line 18, in run
    self.repo.commit(
  File "/usr/local/Cellar/dvc/1.1.10/libexec/lib/python3.8/site-packages/dvc/repo/__init__.py", line 36, in wrapper
    ret = f(repo, *args, **kwargs)
  File "/usr/local/Cellar/dvc/1.1.10/libexec/lib/python3.8/site-packages/dvc/repo/commit.py", line 42, in commit
    changes = stage.changed_entries()
  File "/usr/local/Cellar/dvc/1.1.10/libexec/lib/python3.8/site-packages/dvc/stage/__init__.py", line 405, in changed_entries
    changed_deps = self._changed_entries(self.deps)
  File "/usr/local/Cellar/dvc/1.1.10/libexec/lib/python3.8/site-packages/dvc/stage/__init__.py", line 399, in _changed_entries
    return [str(entry) for entry in entries if entry.changed_checksum()]
  File "/usr/local/Cellar/dvc/1.1.10/libexec/lib/python3.8/site-packages/dvc/stage/__init__.py", line 399, in <listcomp>
    return [str(entry) for entry in entries if entry.changed_checksum()]
  File "/usr/local/Cellar/dvc/1.1.10/libexec/lib/python3.8/site-packages/dvc/output/base.py", line 195, in changed_checksum
    return self.checksum != self.get_checksum()
  File "/usr/local/Cellar/dvc/1.1.10/libexec/lib/python3.8/site-packages/dvc/output/base.py", line 181, in get_checksum
    return self.remote.get_hash(self.path_info)
  File "/usr/local/Cellar/dvc/1.1.10/libexec/lib/python3.8/site-packages/dvc/remote/base.py", line 735, in get_hash
    return self.tree.get_hash(path_info, **kwargs)
  File "/usr/local/Cellar/dvc/1.1.10/libexec/lib/python3.8/site-packages/dvc/remote/base.py", line 276, in get_hash
    assert isinstance(path_info, str) or path_info.scheme == self.scheme
AttributeError: 'NoneType' object has no attribute 'scheme'
------------------------------------------------------------

If I perform dvc add on the files imported (effectively "pinning" them to the main repo) the error disappears and the command executes successfully.

As a workaround, adding the following lines before the assert statement in the function get_hash() in dvc/remote/base.py gets rid of the error even for files imported via dvc import:

    def get_hash(self, path_info, tree=None, **kwargs):
        if not path_info:
            return None

        assert isinstance(path_info, str) or path_info.scheme == self.scheme

I never had any problem with DVC<1.0.0
My setup:

DVC version: 1.1.10
Python version: 3.8.3
Platform: macOS-10.14.6-x86_64-i386-64bit
Binary: False
Package: brew
Supported remotes: azure, gdrive, gs, http, https, s3, ssh, oss
Cache: reflink - supported, hardlink - supported, symlink - supported
Repo: dvc, git

[pared]

reproduction script:

#!/bin/bash

set -ex

rm -rf repo other storage
mkdir repo other

main=$(pwd)

pushd other
git init --quiet
dvc init --quiet

echo data >> data
dvc add data
dvc remote add -d str $main/storage

dvc push

git add -A
git commit -am "init"

popd
pushd repo

git init --quiet
dvc init --quiet

git add -A
git commit -am "init"

dvc import $main/other data
dvc -v commit

[charlesbaynham]

Also getting this. Here's my error logs:

Seems related to the db query at https://github.com/iterative/dvc/blob/bbb1c59df2a289332d13baf7ac9fb2c54b72b5ac/dvc/state.py#L345-L356

[charlesbaynham]

I'm on Windows 10, installed via pip. I notice @Cortysus is migrating from pre-1.0, as am I: could be related?

[pared]

@charlesbaynham does your repo contain import-ed assets? If so, could you install dvc from https://github.com/pared/dvc/tree/4205_commit_imported and see if the issue is still present?

[charlesbaynham]

@pared That's pretty much all it contains: some imported from DVC registries and some from git repositories. I'll give it a shot

[charlesbaynham]

@pared Yes that seems to sort it out!

$ dvc --version
1.1.10+f2010f

username@COMPUTER MINGW64 /tmp/pytest-of-username/pytest-70/test_tutorial_no_errors0/registries/processing (master)
$ dvc commit -v
2020-07-15 16:05:07,392 DEBUG: Trying to spawn '['c:\\users\\username\\.conda\\envs\\dvc\\python.exe', 'C:\\Users\\username\\.conda\\envs\\dvc\\Scripts\\dvc', 'daemon', '-q', 'updater']'
2020-07-15 16:05:07,629 DEBUG: Spawned '['c:\\users\\username\\.conda\\envs\\dvc\\python.exe', 'C:\\Users\\username\\.conda\\envs\\dvc\\Scripts\\dvc', 'daemon', '-q', 'updater']'
2020-07-15 16:05:07,634 DEBUG: fetched: [(3,)]
2020-07-15 16:05:07,709 DEBUG: Path 'C:\Users\username\AppData\Local\Temp\pytest-of-username\pytest-70\test_tutorial_no_errors0\registries\processing\ofm.version' inode '7133478682009261073'
2020-07-15 16:05:07,710 DEBUG: fetched: [('1594823450878883584', '33', '50a891df27d0d8efd0532051a0ebc01d', '1594825498180230912')]
2020-07-15 16:05:07,711 DEBUG: Computed stage: 'ofm.version.dvc' md5: 'b613d0b5318adb2a5d0af14f2b3c11a9'
2020-07-15 16:05:07,716 DEBUG: Saving 'ofm.version' to '..\cache\50\a891df27d0d8efd0532051a0ebc01d'.
2020-07-15 16:05:07,718 DEBUG: Path 'C:\Users\username\AppData\Local\Temp\pytest-of-username\pytest-70\test_tutorial_no_errors0\registries\cache\50\a891df27d0d8efd0532051a0ebc01d' inode '7133478682009261073'
2020-07-15 16:05:07,718 DEBUG: fetched: [('1594823450878883584', '33', '50a891df27d0d8efd0532051a0ebc01d', '1594825507710831872')]
2020-07-15 16:05:07,719 DEBUG: cache 'C:\Users\username\AppData\Local\Temp\pytest-of-username\pytest-70\test_tutorial_no_errors0\registries\cache\50\a891df27d0d8efd0532051a0ebc01d' expected '50a891df27d0d8efd0532051a0ebc01d' actual '50a891df27d0d8efd0532051a0ebc01d'
2020-07-15 16:05:07,724 DEBUG: Removing 'C:\Users\username\AppData\Local\Temp\pytest-of-username\pytest-70\test_tutorial_no_errors0\registries\processing\ofm.version'
2020-07-15 16:05:07,728 DEBUG: Cache type 'reflink' is not supported: reflink is not supported
2020-07-15 16:05:07,730 DEBUG: Created 'hardlink': ..\cache\50\a891df27d0d8efd0532051a0ebc01d -> ofm.version
2020-07-15 16:05:07,734 DEBUG: Path 'ofm.version' inode '7133478682009261073'
2020-07-15 16:05:07,736 DEBUG: Path 'ofm.version' inode '7133478682009261073'
2020-07-15 16:05:07,736 DEBUG: fetched: [('1594823450878883584', '33', '50a891df27d0d8efd0532051a0ebc01d', '1594825507719595520')]
2020-07-15 16:05:07,737 DEBUG: Path '..\cache\50\a891df27d0d8efd0532051a0ebc01d' inode '7133478682009261073'
2020-07-15 16:05:07,738 DEBUG: fetched: [('1594823450878883584', '33', '50a891df27d0d8efd0532051a0ebc01d', '1594825507737831168')]
2020-07-15 16:05:07,742 DEBUG: Saving information to 'ofm.version.dvc'.
2020-07-15 16:05:07,759 DEBUG: fetched: [(7,)]

[pared]

@charlesbaynham thank you very much for checking, seems like it might be the solution, need to wait for reviews, as I might be missing something.

[pared]

@charlesbaynham @Cortysus, fix has been merged, release with this issue fixed should be available soon.

[charlesbaynham]

Fantastic, thank you: I'll look out for it!

[Cortysus]

Cheers, thanks a lot!

[skshetry]

It's already available, with the magic number 1.1.11. :smile: