Salt: GITFS w/ pygit2 0.28.2 and a git repo requiring authentication, fails

Created on 21 Jun 2019  路  13Comments  路  Source: saltstack/salt

Description of Issue

I'm running Alpine docker containers. Alpine 3.10 has just been released and pygit2 has been bumped from 0.27.1-r0 to 0.28.2-r0. Now my GITFS repos are failing.

I'm using HTTPS as a transport, but I see the same behaviour using SSH keys.

GitPython does not experience the same issue.

HTTPS

[DEBUG   ] Updating gitfs fileserver cache for the following targets: [('https://git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/salt-states', None)]
[DEBUG   ] Re-using gitfs object for process 41
[DEBUG   ] Set update lock for gitfs remote 'https://git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/salt-states'
[DEBUG   ] Fetching gitfs remote 'https://git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/salt-states'
[DEBUG   ] Removed update lock for gitfs remote 'https://git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/salt-states'
[ERROR   ] Exception caught while fetching gitfs remote 'https://git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/salt-states': list index out of range
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 2333, in fetch_remotes
    if repo.fetch():
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 767, in fetch
    return self._fetch()
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 1702, in _fetch
    origin = self.repo.remotes[0]
  File "/usr/lib/python3.7/site-packages/pygit2/remote.py", line 582, in __getitem__
    return list(self)[name]
IndexError: list index out of range

SSH

[ERROR   ] Exception caught while fetching gitfs remote 'ssh://#################@git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/salt-states': list index out of range
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 2333, in fetch_remotes
    if repo.fetch():
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 767, in fetch
    return self._fetch()
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 1702, in _fetch
    origin = self.repo.remotes[0]
  File "/usr/lib/python3.7/site-packages/pygit2/remote.py", line 582, in __getitem__
    return list(self)[name]
IndexError: list index out of range

Setup

I can reproduce this with a single salt-master and a private git repo. The trigger seems to be the version of pygit2.

gitfs.conf

fileserver_backend:
  - gitfs

gitfs_remotes:
  - https://git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/salt-states:
    - user: ############
    - password: ##########

Steps to Reproduce Issue

The following steps reproduce the issue for me:

docker run -it alpine:3.10
apk add git salt-master
mkdir /etc/salt/master.d
vi /etc/salt/master.d/gitfs.conf
Configure a GITFS with a git repo requring authentication
salt-master -l info

Versions Report

Salt Version:
           Salt: 2019.2.0

Dependency Versions:
           cffi: 1.11.5
       cherrypy: 18.1.1
       dateutil: 2.7.3
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: 0.28.2
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.1
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.28.2
         Python: 3.7.3 (default, May  3 2019, 11:24:39)
   python-gnupg: Not Installed
         PyYAML: 5.1.1
          PyZMQ: 18.0.1
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.2
            ZMQ: 4.3.1

System Versions:
           dist:
         locale: UTF-8
        machine: x86_64
        release: 4.9.0-9-amd64
         system: Linux
        version: Not Installed
Aluminium Bug phase-plan severity-high status-in-prog
Source
picture pmackey2
👍1

Most helpful comment

Does this need to be ported to master? https://github.com/saltstack/salt/pull/51304

picture xeacott on 5 Jun 2020
👍2

All 13 comments

Make sure that the pygit2 version matches the libgit2 version. I had to downgrad back to 0.26 to avoid the same issue.

greut picture greut on 11 Jul 2019

Hi,

Same here on EL7 (libgit2 0.26.8) Only the first sync is in error (repos that have already been sync with previous version still work)

Thanks for the tips, it works with pygit2 0.26

Best regards,

Poil picture Poil on 13 Aug 2019
👍1

I'm getting the same with:

root@salt1:/ # pkg info -x git
git-2.22.0
libgit2-0.28.1
py27-pygit2-0.28.0

and:

root@salt1:/ # cat /usr/local/etc/salt/master.d/override.conf 
interface: 192.168.0.20
ipv6: False
gitfs_provider: pygit2
fileserver_backend:
  - gitfs
  - roots
gitfs_remotes:
  - https://github.com/silenius/jails-formula.git:
    - name: jails
    - update_interval: 120

2019-08-21 14:50:41,831 [salt.utils.gitfs                         :2345][ERROR   ][29418] Exception caught while fetching gitfs remote 'https://github.com/silenius/jails-formula.git': list index out of range
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/salt/utils/gitfs.py", line 2333, in fetch_remotes
    if repo.fetch():
  File "/usr/local/lib/python2.7/site-packages/salt/utils/gitfs.py", line 767, in fetch
    return self._fetch()
  File "/usr/local/lib/python2.7/site-packages/salt/utils/gitfs.py", line 1702, in _fetch
    origin = self.repo.remotes[0]
  File "/usr/local/lib/python2.7/site-packages/pygit2/remote.py", line 584, in __getitem__
    return list(self)[name]
IndexError: list index out of range

(this is on FreeBSD)

silenius picture silenius on 21 Aug 2019

@silenius do you mind trying that?

-origin = self.repo.remotes[0]
+origin = self.repo.remotes['origin']
greut picture greut on 22 Aug 2019

@greut I built Alpine:edge today with Salt 2019.2.0 and changed the file in question with:
sed -i -e "s/self.repo.remotes\[0\]/self.repo.remotes\[\'origin\'\]/g" /usr/lib/python3.7/site-packages/salt/utils/gitfs.py
But now I see a new error trying to do salt '*' state.show_top on my minions.

New error message:

[ERROR   ] Error in function _file_list:
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/salt/master.py", line 1838, in run_func
    ret = getattr(self, func)(load)
  File "/usr/lib/python3.7/site-packages/salt/utils/decorators/__init__.py", line 666, in wrapped
    return function(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/salt/fileserver/__init__.py", line 789, in file_list
    ret.update(self.servers[fstr](load))
  File "/usr/lib/python3.7/site-packages/salt/fileserver/gitfs.py", line 185, in file_list
    return _gitfs().file_list(load)
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 2943, in file_list
    return self._file_lists(load, 'files')
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 2920, in _file_lists
    repo_files, repo_symlinks = repo.file_list(load['saltenv'])
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 1786, in file_list
    tree = self.get_tree(tgt_env)
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 1031, in get_tree
    candidate = func(tgt_ref)
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 1861, in get_tree_from_branch
    'refs/remotes/origin/{0}'.format(ref)).get_object().tree
AttributeError: '_pygit2.Reference' object has no attribute 'get_object'
[ERROR   ] Unable to get SHA of HEAD for git_pillar remote 'master [email protected]:salt.git'
[ERROR   ] Error in function _file_hash:
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/salt/master.py", line 1838, in run_func
    ret = getattr(self, func)(load)
  File "/usr/lib/python3.7/site-packages/salt/fileserver/__init__.py", line 681, in file_hash
    return self.__file_hash_and_stat(load)[0]
  File "/usr/lib/python3.7/site-packages/salt/fileserver/__init__.py", line 667, in __file_hash_and_stat
    load['saltenv'])
  File "/usr/lib/python3.7/site-packages/salt/fileserver/__init__.py", line 623, in find_file
    fnd = self.servers[fstr](path, saltenv, **kwargs)
  File "/usr/lib/python3.7/site-packages/salt/fileserver/gitfs.py", line 155, in find_file
    return _gitfs().find_file(path, tgt_env=tgt_env, **kwargs)
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 2764, in find_file
    blob, blob_hexsha, blob_mode = repo.find_file(repo_path, tgt_env)
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 1818, in find_file
    tree = self.get_tree(tgt_env)
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 1031, in get_tree
    candidate = func(tgt_ref)
  File "/usr/lib/python3.7/site-packages/salt/utils/gitfs.py", line 1861, in get_tree_from_branch
    'refs/remotes/origin/{0}'.format(ref)).get_object().tree
AttributeError: '_pygit2.Reference' object has no attribute 'get_object'

The same Git repo works with HTTP authentication turned on.

/ # salt '*' state.show_top
MINION_ID:
    ----------
    base:
        - win.gpo
        - win.pkgs
        - win.users
pysen picture pysen on 27 Aug 2019

@pysen thanks for the test! The changes on the library are probably way bigger than I thought.

greut picture greut on 27 Aug 2019

@pmackey2 you can try apply change from this PR: https://github.com/saltstack/salt/pull/51304, with change which @greut proposed.

With this setup salt master is working properly with https and ssh repositories

Salt Version:
           Salt: 2019.2.0

Dependency Versions:
           cffi: 1.13.0
       cherrypy: 18.3.0
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: 0.28.2
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.28.2
         Python: 3.6.8 (default, Oct  7 2019, 12:59:55)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-165-generic
         system: Linux
        version: Ubuntu 18.04 bionic
kubadawczynski picture kubadawczynski on 17 Oct 2019

@greut can you create pull request with this? Looks like this is missing part to have this topic fully resolved.

dawidmalina picture dawidmalina on 17 Oct 2019

@kubadawczynski I've applied the patch and it has resolved my issue.

pmackey2 picture pmackey2 on 20 Oct 2019
👍2

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

stale[bot] picture stale[bot] on 7 Jan 2020
😕1

hey stale bot, do you mind keeping it open?

greut picture greut on 7 Jan 2020

Thank you for updating this issue. It is no longer marked as stale.

stale[bot] picture stale[bot] on 7 Jan 2020
👍2

Does this need to be ported to master? https://github.com/saltstack/salt/pull/51304

xeacott picture xeacott on 5 Jun 2020
👍2
Was this page helpful?
0 / 5 - 0 ratings

Related issues

sjorge picture sjorge  路  73Comments
Zogg picture Zogg  路  80Comments
xiaopanggege picture xiaopanggege  路  158Comments
chrismoos picture chrismoos  路  54Comments
leonhedding picture leonhedding  路  57Comments