Salt: spm install -f doesn't update spm info

[daveneeley]

Description of Issue/Question

When an spm package has been previously installed, and spm install -f is used to install a newer version of the package, running spm info doesn't show that the version has been updated.

Steps to Reproduce Issue

Generate a new SPM-enabled formula repo, such as with this cookiecutter: https://github.com/mitodl/saltstack-formula-cookiecutter (formula doesn't have to do anything, just having the format is good enough)
Version should already be set to 1 in the FORMULA file
Run spm build
Run spm install
Run spm info
Make a change to init.sls file in the formula
Bump the version to 2 in the FORMULA file
Run spm build
Run spm update_repo
Run spm install -f
Run spm info
Note that the sls change was made, but the version from spm info was not updated

Versions Report

Salt Version:
Salt: 2016.11.2

Dependency Versions:
cffi: Not Installed
cherrypy: 3.7.0
dateutil: Not Installed
gitdb: 0.6.4
gitpython: 1.0.1
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: 0.21.1
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.7
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 14.7.0
RAET: Not Installed
smmap: 0.9.0
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.0.5

System Versions:
dist: centos 7.2.1511 Core
machine: x86_64
release: 3.10.0-327.22.2.el7.x86_64
system: Linux
version: CentOS Linux 7.2.1511 Core

[gtmanfred]

Can you update to 2017.7?

We made a lot of changes to spm in this release. And I am wondering if this got fixed.

Thanks!
Daniel

[daveneeley]

We're running redhat 6 and last time we attempted that, yum was broken by the python version upgrade. We had to pin the salt repo at 2016.11.

[gtmanfred]

Have you tried 2017.7.1?

We found a few packages that we conflicted with in the .0 repo, but they should all be fixed in 2017.7.1

We would really appreciate it if you opened an issue on https://github.com/saltstack/salt-pack with the output of yum if it breaks on your setup.

Thanks,
Daniel

[gtmanfred]

I have replicated this on 2017.7.1

@techhat do you have any pointers on what might be going on here?

Thanks,
Daniel

[techhat]

Well, there's plenty of INSERTs in salt/spm/pkgdb/sqlite3.py, but no UPDATEs.

[daveneeley]

Not being terribly familiar with python, here's my question. I see this method:

def register_pkg(name, formula_def, conn=None):
    '''
    Register a package in the package database
    '''
    if conn is None:
        conn = init()

    conn.execute('INSERT INTO packages VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)', (
        name,
        formula_def['version'],
        formula_def['release'],
        datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT'),
        formula_def.get('os', None),
        formula_def.get('os_family', None),
        formula_def.get('dependencies', None),
        formula_def.get('os_dependencies', None),
        formula_def.get('os_family_dependencies', None),
        formula_def['summary'],
        formula_def['description'],
    ))

I assume I would just add a try+except if record already exists then issue an update? I see how exceptions are dropped in the caller's _pkgdb_fun instead of in sqlite3.py. Maybe that would be IntegrityError?

Or is the better option to run a SELECT first and then opt for inserting or deleting based on result?

[techhat]

Looks like sqlite supports the REPLACE command.

[gtmanfred]

Dave,

I am going to be working on adding in upgrading and downgrading packages so
that -f won't need to be used for this function.

And in that I will look for why this is happening.

Thanks for reporting,
Daniel

On Fri, Sep 1, 2017 at 7:12 AM, Dave Neeley [e-mail]
wrote:

Not being terribly familiar with python, here's my question. I see this
method:

def register_pkg(name, formula_def, conn=None):
'''
Register a package in the package database
'''
if conn is None:
conn = init()

conn.execute('INSERT INTO packages VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)', (
    name,
    formula_def['version'],
    formula_def['release'],
    datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT'),
    formula_def.get('os', None),
    formula_def.get('os_family', None),
    formula_def.get('dependencies', None),
    formula_def.get('os_dependencies', None),
    formula_def.get('os_family_dependencies', None),
    formula_def['summary'],
    formula_def['description'],
))

I assume I would just add a try+except if record already exists then issue
an update? I see how exceptions are dropped in the caller's _pkgdb_fun
instead of in sqlite3.py. If so, how do I tell what that exception would be
called?

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/saltstack/salt/issues/43239#issuecomment-326576454,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAssoUTD9dVfQumulfa9k1ux8LJCtud7ks5seALSgaJpZM4PFwOU
.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[gtmanfred]

relevant

[stale[bot]]

Thank you for updating this issue. It is no longer marked as stale.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.