Gopass: How to migrate password store from 1.9 to 1.10?

[shibumi]

Summary

Hello,
I really like your roadmap for Gopass 2.0 including age as default(?) backend.
However, what I am missing is a guide for migrating the password store to the new gopass versions.

When I install gopass version 1.10 and I run it, the gopass initialization wizard pops up.
The problem with this is, that It tries to generate a new GPG key, although I would like to further use my GPG key
on my Yubikey.

Steps To Reproduce

Update to gopass 1.10 and just try any operation on an old password store.

Expected behavior

Gopass 1.10 should identify an existing password-store and not try to re-initialize the gopass configuration
or there should be documentation for properly migrating to Gopass 1.10 or the migration should happen automatically.

Environment

• OS: Arch Linux
• OS version: [uname -a] Linux mokoto.shibumi.dev 5.8.2-arch1-1 #1 SMP PREEMPT Thu, 20 Aug 2020 20:45:00 +0000 x86_64 GNU/Linux
• gopass Version: [gopass version] Gopass 1.10 (moved back to gopass 1.9)
• Installation method: repo

[dominikschulz]

You shouldn't need to migrate anything.
Any 1.x release will keep supporting the popular Password Store layout (maybe not 100% of the features, but all common operations should work).

I wonder what's wrong with your setup.

Could you run gopass 1.10.x with GOPASS_DEBUG_LOG=/tmp/gopass.log set?
If you're willing to share that logfile we might be able to debug this.

[shibumi]

Sure, here is the requested output:

2020/08/24 22:13:42.535761 pwrules/aliases.go:52    pwrules.loadCustomAliases   no custom aliases found at /home/chris/.config/gopass/domain-aliases.json
2020/08/24 22:13:42.614792 config/io.go:37  config.loadConfig   Trying to load config from /home/chris/.config/gopass/config.yml
2020/08/24 22:13:42.614846 config/io.go:37  config.loadConfig   Trying to load config from /home/chris/.config/gopass/config.yml
2020/08/24 22:13:42.614879 config/io.go:37  config.loadConfig   Trying to load config from /home/chris/.gopass.yml
2020/08/24 22:13:42.615024 config/io.go:52  config.loadDefault  Loaded default config: &config.Config{AutoClip:false, AutoImport:true, ClipTimeout:45, ExportKeys:true, MIME:true, NoColor:false, NoPager:false, Notifications:true, Path:"/home/chris/.local/share/gopass/stores/root", SafeContent:false, Mounts:map[string]string{}, configPath:"/home/chris/.config/gopass/config.yml", XXX:map[string]interface {}(nil)}
2020/08/24 22:13:42.615441 root/init.go:17  root.(*Store).Initialized   initializing store and possible sub-stores
2020/08/24 22:13:42.615469 root/init.go:65  root.(*Store).initialize    initialize - /home/chris/.local/share/gopass/stores/root
2020/08/24 22:13:42.615484 leaf/store.go:51 leaf.New    Instantiating  at /home/chris/.local/share/gopass/stores/root
2020/08/24 22:13:42.615529 backend/storage.go:74    backend.DetectStorage   Trying gitfs for /home/chris/.local/share/gopass/stores/root
2020/08/24 22:13:42.615567 backend/storage.go:76    backend.DetectStorage   failed to use gitfs for /home/chris/.local/share/gopass/stores/root: no .git
2020/08/24 22:13:42.615593 backend/storage.go:74    backend.DetectStorage   Trying ondisk for /home/chris/.local/share/gopass/stores/root
2020/08/24 22:13:42.615615 backend/storage.go:76    backend.DetectStorage   failed to use ondisk for /home/chris/.local/share/gopass/stores/root: not supported
2020/08/24 22:13:42.615639 backend/storage.go:74    backend.DetectStorage   Trying fs for /home/chris/.local/share/gopass/stores/root
2020/08/24 22:13:42.615661 backend/storage.go:79    backend.DetectStorage   Using fs for /home/chris/.local/share/gopass/stores/root
2020/08/24 22:13:42.615707 fs/loader.go:29  fs.loader.New   Using Storage Backend: fs(v0.1.0,path:/home/chris/.local/share/gopass/stores/root)
2020/08/24 22:13:42.615722 leaf/store.go:62 leaf.New    Storage initialized
2020/08/24 22:13:42.615758 backend/crypto.go:95 backend.DetectCrypto    Trying gpgcli for fs(v0.1.0,path:/home/chris/.local/share/gopass/stores/root)
2020/08/24 22:13:42.615780 fs/store.go:110  fs.(*Store).Exists  Checking if .gpg-id exists at /home/chris/.local/share/gopass/stores/root/.gpg-id: false
2020/08/24 22:13:42.615816 backend/crypto.go:97 backend.DetectCrypto    failed to use crypto gpgcli for fs(v0.1.0,path:/home/chris/.local/share/gopass/stores/root)
2020/08/24 22:13:42.615835 backend/crypto.go:95 backend.DetectCrypto    Trying age for fs(v0.1.0,path:/home/chris/.local/share/gopass/stores/root)
2020/08/24 22:13:42.615868 fs/store.go:110  fs.(*Store).Exists  Checking if .age-ids exists at /home/chris/.local/share/gopass/stores/root/.age-ids: false
2020/08/24 22:13:42.615886 backend/crypto.go:97 backend.DetectCrypto    failed to use crypto age for fs(v0.1.0,path:/home/chris/.local/share/gopass/stores/root)
2020/08/24 22:13:42.615911 backend/crypto.go:95 backend.DetectCrypto    Trying plain for fs(v0.1.0,path:/home/chris/.local/share/gopass/stores/root)
2020/08/24 22:13:42.615930 fs/store.go:110  fs.(*Store).Exists  Checking if .plain-id exists at /home/chris/.local/share/gopass/stores/root/.plain-id: false
2020/08/24 22:13:42.615953 backend/crypto.go:97 backend.DetectCrypto    failed to use crypto plain for fs(v0.1.0,path:/home/chris/.local/share/gopass/stores/root)
2020/08/24 22:13:42.615969 backend/crypto.go:103    backend.DetectCrypto    No valid crypto provider found for fs(v0.1.0,path:/home/chris/.local/share/gopass/stores/root)
2020/08/24 22:13:42.615991 leaf/store.go:68 leaf.New    Crypto initialized
2020/08/24 22:13:42.616016 leaf/store.go:70 leaf.New    Instantiated  at /home/chris/.local/share/gopass/stores/root - storage: &fs.Store{path:"/home/chris/.local/share/gopass/stores/root"} - crypto: <nil>
2020/08/24 22:13:42.616046 root/init.go:70  root.(*Store).initialize    Root Store initialized at /home/chris/.local/share/gopass/stores/root
2020/08/24 22:13:42.616067 fs/store.go:110  fs.(*Store).Exists  Checking if  exists at /home/chris/.local/share/gopass/stores/root: false
2020/08/24 22:13:42.616082 action/init.go:38    action.(*Action).Initialized    Store needs to be initialized
2020/08/24 22:13:46.505612 action/errors.go:58  action.ExitError    Stacktrace: user aborted

Note: My password-store lies in /home/chris/.password-store this is not a problem with Gopass 1.9. With Gopass 1.10, gopass tries to initialize a store at /home/chris/.local/share/gopass/stores? oO

Another note: Looks like gopass 1.9 never generated a config file, while gopass 1.10 seems to need a config file?

[dominikschulz]

Thanks!

It didn't find any config, so it's creating a new one and using the default gopass path.

Where is your existing password store located?
Did you have a gopass config before?

[shibumi]

My existing password-store is in /home/chris/.password-store

I think I know what's going on now. I migrated to an new thinkpad recently and I never migrated the gopass config.yml file to the new notebook. So I just pulled my last config.yml file out of my restic backup and put it in /home/chris/.config/gopass.config.yml now..

Now with the old config, I get this error here:

❯ gopass
Failed to load config *config.Config. unknown fields: [root]
gopass>

With this config:

root:
  askformore: false
  autoclip: false
  autoprint: false
  autoimport: true
  autosync: true
  check_recipient_hash: false
  cliptimeout: 45
  concurrency: 1
  editrecipients: false
  nocolor: false
  noconfirm: false
  nopager: false
  notifications: true
  path: gpgcli-gitcli-fs+file:///home/chris/.password-store
  recipient_hash: {}
  safecontent: false
  usesymbols: false
mounts: {}

I wonder about two things:

1. Why does gopass 1.9 work without any config? :D
2. why does my old gopass config, that was compatible with gopass 1.9, seems not to be compatible with gopass 1.10 anymore? And how do I fix this? :D

[r3k2]

I have the same issue.. you guys broke this I had pass and gopass working with ~/.password-store and now pass works but not gopass :( WTF was migrating to gopass from pass now I had to spend 1 hours moving all my CLI apps back to pass

[r3k2]

I tried this:

gopass config path "~/.password-Store"
path: ~/.password-store

and now I STILL get promp even do I have the stupid config.

 cat config.yml
autoclip: false
autoimport: true
cliptimeout: 45
exportkeys: true
mime: true
nocolor: false
nopager: false
notifications: true
path: ~/.password-store
safecontent: false
mounts: {}

if I go ahead and say yes and choose local I get:

Error: failed to run onboarding wizard: failed to init local store: failed to init store '' at '/home/rek2/.password-store': failed to initialize new sub store: Found already initialized store at /home/rek2/.password-store.

so no kidding! of course I have a password store but gopass does not care even do I am telling it where it is....
and even I created a config file....

sorry if I sound frustrated is because I am.

[shibumi]

I have the same issue.. you guys broke this I had pass and gopass working with ~/.password-store and now pass works but not gopass :( WTF was migrating to gopass from pass now I had to spend 1 hours moving all my CLI apps back to pass

Be a little bit nicer :) They do this in their free time. "You guys broke this" sounds a little bit demanding :)

[shibumi]

@r3k2 did you build the path by yourself? Because I think this should look like this:
gpgcli-gitcli-fs+file:///home/chris/.password-store

[shibumi]

@dominikschulz One addition. I tried modifying the config file now.. after removing some unknown fields I ended up with this one here:

autoclip: false
autoimport: true
cliptimeout: 45
nocolor: false
nopager: false
notifications: true
path: gpgcli-gitcli-fs+file:///home/chris/.password-store
safecontent: false
mounts: {}

when I start gopass with this configuration, it still starts the installation wizard. Here is the debug log:

2020/08/25 02:30:58.676284 pwrules/aliases.go:52    pwrules.loadCustomAliases   no custom aliases found at /home/chris/.config/gopass/domain-aliases.json
2020/08/25 02:30:58.753409 config/io.go:37  config.loadConfig   Trying to load config from /home/chris/.config/gopass/config.yml
2020/08/25 02:30:58.753576 config/io.go:113 config.decode   Trying to unmarshal config into *config.Config
2020/08/25 02:30:58.754153 config/io.go:125 config.decode   Loaded config: *config.Config: &config.Config{AutoClip:false, AutoImport:true, ClipTimeout:45, ExportKeys:true, MIME:false, NoColor:false, NoPager:false, Notifications:true, Path:"gpgcli-gitcli-fs+file:///home/chris/.password-store", SafeContent:false, Mounts:map[string]string{}, configPath:"", XXX:map[string]interface {}(nil)}
2020/08/25 02:30:58.754212 config/io.go:45  config.loadConfig   Loaded config from /home/chris/.config/gopass/config.yml: &config.Config{AutoClip:false, AutoImport:true, ClipTimeout:45, ExportKeys:true, MIME:false, NoColor:false, NoPager:false, Notifications:true, Path:"gpgcli-gitcli-fs+file:///home/chris/.password-store", SafeContent:false, Mounts:map[string]string{}, configPath:"/home/chris/.config/gopass/config.yml", XXX:map[string]interface {}(nil)}
2020/08/25 02:30:58.754557 root/init.go:17  root.(*Store).Initialized   initializing store and possible sub-stores
2020/08/25 02:30:58.754579 root/init.go:65  root.(*Store).initialize    initialize - gpgcli-gitcli-fs+file:///home/chris/.password-store
2020/08/25 02:30:58.754599 leaf/store.go:51 leaf.New    Instantiating  at gpgcli-gitcli-fs+file:///home/chris/.password-store
2020/08/25 02:30:58.754625 backend/storage.go:74    backend.DetectStorage   Trying gitfs for gpgcli-gitcli-fs+file:///home/chris/.password-store
2020/08/25 02:30:58.754661 backend/storage.go:76    backend.DetectStorage   failed to use gitfs for gpgcli-gitcli-fs+file:///home/chris/.password-store: no .git
2020/08/25 02:30:58.754676 backend/storage.go:74    backend.DetectStorage   Trying ondisk for gpgcli-gitcli-fs+file:///home/chris/.password-store
2020/08/25 02:30:58.754700 backend/storage.go:76    backend.DetectStorage   failed to use ondisk for gpgcli-gitcli-fs+file:///home/chris/.password-store: not supported
2020/08/25 02:30:58.754803 backend/storage.go:74    backend.DetectStorage   Trying fs for gpgcli-gitcli-fs+file:///home/chris/.password-store
2020/08/25 02:30:58.754834 backend/storage.go:79    backend.DetectStorage   Using fs for gpgcli-gitcli-fs+file:///home/chris/.password-store
2020/08/25 02:30:58.754859 fs/loader.go:29  fs.loader.New   Using Storage Backend: fs(v0.1.0,path:gpgcli-gitcli-fs+file:/home/chris/.password-store)
2020/08/25 02:30:58.754867 leaf/store.go:62 leaf.New    Storage initialized
2020/08/25 02:30:58.754902 backend/crypto.go:95 backend.DetectCrypto    Trying gpgcli for fs(v0.1.0,path:gpgcli-gitcli-fs+file:/home/chris/.password-store)
2020/08/25 02:30:58.754917 fs/store.go:110  fs.(*Store).Exists  Checking if .gpg-id exists at gpgcli-gitcli-fs+file:/home/chris/.password-store/.gpg-id: false
2020/08/25 02:30:58.754930 backend/crypto.go:97 backend.DetectCrypto    failed to use crypto gpgcli for fs(v0.1.0,path:gpgcli-gitcli-fs+file:/home/chris/.password-store)
2020/08/25 02:30:58.754942 backend/crypto.go:95 backend.DetectCrypto    Trying age for fs(v0.1.0,path:gpgcli-gitcli-fs+file:/home/chris/.password-store)
2020/08/25 02:30:58.754951 fs/store.go:110  fs.(*Store).Exists  Checking if .age-ids exists at gpgcli-gitcli-fs+file:/home/chris/.password-store/.age-ids: false
2020/08/25 02:30:58.754960 backend/crypto.go:97 backend.DetectCrypto    failed to use crypto age for fs(v0.1.0,path:gpgcli-gitcli-fs+file:/home/chris/.password-store)
2020/08/25 02:30:58.754968 backend/crypto.go:95 backend.DetectCrypto    Trying plain for fs(v0.1.0,path:gpgcli-gitcli-fs+file:/home/chris/.password-store)
2020/08/25 02:30:58.754978 fs/store.go:110  fs.(*Store).Exists  Checking if .plain-id exists at gpgcli-gitcli-fs+file:/home/chris/.password-store/.plain-id: false
2020/08/25 02:30:58.754987 backend/crypto.go:97 backend.DetectCrypto    failed to use crypto plain for fs(v0.1.0,path:gpgcli-gitcli-fs+file:/home/chris/.password-store)
2020/08/25 02:30:58.755001 backend/crypto.go:103    backend.DetectCrypto    No valid crypto provider found for fs(v0.1.0,path:gpgcli-gitcli-fs+file:/home/chris/.password-store)
2020/08/25 02:30:58.755010 leaf/store.go:68 leaf.New    Crypto initialized
2020/08/25 02:30:58.755043 leaf/store.go:70 leaf.New    Instantiated  at gpgcli-gitcli-fs+file:///home/chris/.password-store - storage: &fs.Store{path:"gpgcli-gitcli-fs+file:/home/chris/.password-store"} - crypto: <nil>
2020/08/25 02:30:58.755053 root/init.go:70  root.(*Store).initialize    Root Store initialized at gpgcli-gitcli-fs+file:///home/chris/.password-store
2020/08/25 02:30:58.755062 fs/store.go:110  fs.(*Store).Exists  Checking if  exists at gpgcli-gitcli-fs+file:/home/chris/.password-store: false
2020/08/25 02:30:58.755071 action/init.go:38    action.(*Action).Initialized    Store needs to be initialized
2020/08/25 02:31:00.048234 action/errors.go:58  action.ExitError    Stacktrace: user aborted

[shibumi]

Another addition:

I managed to migrate from gopass 1.9 to 1.10 with this configuration here:

autoclip: false
autoimport: true
cliptimeout: 45
nocolor: false
nopager: false
notifications: true
path: /home/chris/.password-store
safecontent: false
mounts: {}

@dominikschulz Do you accept PRs for documentation for migration?

[r3k2]

I have the same issue.. you guys broke this I had pass and gopass working with ~/.password-store and now pass works but not gopass :( WTF was migrating to gopass from pass now I had to spend 1 hours moving all my CLI apps back to pass

Be a little bit nicer :) They do this in their free time. "You guys broke this" sounds a little bit demanding :)

yeah was a bit frustrated, broke my whole evening of work fixing it, it is voluntier work but when so many people uses the software there should be some QA/Testing but yeah you are right.

[r3k2]

Another addition:

I managed to migrate from gopass 1.9 to 1.10 with this configuration here:

autoclip: false
autoimport: true
cliptimeout: 45
nocolor: false
nopager: false
notifications: true
path: /home/chris/.password-store
safecontent: false
mounts: {}

@dominikschulz Do you accept PRs for documentation for migration?

yeah I ended up getting it to work using the "full" path.... they should change that to allow unix style ~/. because I push my configs to git and move around diff laptops with diff usernames.and I am sure lot of other unix hackers do the same.

[dominikschulz]

@shibumi Sure we always accept PRs!

@r3k2 Sorry for breaking your workflow. Remember you can always go back to the previous release if there are regressions.

Overall it looks like the path fallback was broken. We don't expect users to touch their config when upgrading. I've got an idea what's wrong but need to investigate.