vault 🚀 - Provide a way to list secrets

+1 I came to post this as well. I have a few ideas of how we could use vault in our infrastructure. However, all of them require vault entries to be discoverable. Ideally there would also be a way to query/filter secrets as well.

jtdowney on 30 Apr 2015

:+1: I'd like to see this one as well.

DavidWittman on 30 Apr 2015

+1 to be able to List secrets. Ideally as a separate policy (or maybe wrapped into the "sudo" policy).

nmcclain on 1 May 2015

I've been developing a graphical frontend to Vault as an alternative to KeePass called VaultKee which would require listing of available secrets.

In the mean time, I wrote a quick standalone API called VaultDiscover to list available secrets for generic backends that are using the file backend type until Vault natively supports this.

ShadowApex on 7 May 2015

👍2

+1 - I'm currently testing Vault for an in-house password storage and the ability to list secrets would help out a lot.

In the meantime, I''m trying to hack together something like what @ShadowApex is doing with VaultDiscover and also comtemplating wether or not I could solve it with some clever Redis-tinkering.

MadsRC on 7 May 2015

Hi there,

I have marked this issue as "accepted", meaning this is something we want to add to Vault. If you are interested in subscribing or showing support for this issue, please click the "Subscribe" button on the right side.

If you have a question or suggestion, please leave a comment below.

sethvargo on 22 May 2015

Hey, are there any release plans for this feature?

kaelumania on 25 Jun 2015

It's not on the immediate roadmap but something we've already designed into the core and plan on implementing.

armon on 25 Jun 2015

Interesting feature. I have also considered using vault as you are describing it.

pandrew on 25 Jun 2015

After some discussion with @armon about authentication mechanisms, I want to lodge a request to be able to turn this off (probably already on the roadmap, just making it formal), or at least restrict it to sudo and/or root tokens.

jefferai on 30 Jun 2015

I'd vote for (as nmcclain said above) making it a policy - that way it's easy to configure. Make it a separate mount (list/). Have read = list and write = create new secrets. Should fit quite neatly into the design I think. Then policy is simply on "list/my_mount" to be able to create/list in the /my_mount path.

bluecmd on 30 Jun 2015

This feature would be awesome. I understand why it did not exist by default, but right now I have Web UI that queries Vault, and keeps a local list of all secret keys. It works ok, but I hate workarounds. ;-) Thanks!

ghost on 10 Jul 2015

This should extend beyond secrets as well.

I would like a way to see all app-ids and user-ids that have been registered with the app-id auth system.

It would also be nice to list all known tokens in general.

nbrownus on 16 Jul 2015

:+1:

skyzyx on 20 Jul 2015

The use case I have, is using Vault as a secure config service. I would like to version configs. And in a panic, when I want to compare a working config against a broken config, I'd like to be able to list configs.

So I'd like to be able to:

vault write config/service1/v1 config="..."
vault write config/service1/v2 config="..."

# time passes...

vault ls config/service1/*

If I have to fetch a list of _all_ keys and filter them, well okay then. But being able to list keys with a prefix and wildcard (as per policies), that would rock.

sheldonh on 21 Jul 2015

That is exactly the syntax I had in mind as well. I'd like to use Vault to store secrets so that we can pull them down during deployment and inject them into ENV using the results from vault ls myapp/env/*.

I think managing ENV with Vault like this could be a popular use-case, as a lot of developers don't have a great solution and resort to putting secrets in the repo.

lserman on 31 Jul 2015

+1 to:

vault ls path/directory
vault ls path/directory/wild*card

wwalker on 3 Aug 2015

👍9

+1

mbrevoort on 11 Aug 2015

+1 to the ability to list all secret, even if only using a certain policy.

Russell- on 11 Aug 2015

To future readers: you don't need to leave a comment to notify HashiCorp of your interest in the issue. Subscribing to the issue is sufficient. Be aware that the change is significant and will take real work.

sheldonh on 19 Aug 2015

I'm interested in something a little different. It would be great to be able to see what secrets a particular user has access to.

Currently this is very tricky, as afaict there's know what to know what group a token belongs to.

So, as an authed user, a command as simple as 'vault ls' to see what passwords the user can see at their current privilege level.

dalehamel on 24 Aug 2015

Any chance this might make it into the 0.3.0 milestone? Or a later milestone perhaps?

dalehamel on 31 Aug 2015

It is not currently targeted for any specific milestone.

jefferai on 31 Aug 2015

@jefferai this makes Sad Panda Sad

7477846239188242185 jpeg___1_500_1_500_cb94de6a_

parabuzzle on 31 Aug 2015

😄5

It seems like a number of workarounds exist that simply query the storage backend directly. For instance, i write a 10 line sinatra app that for this purpose.

Since vault seems less concerned with the structure of the data stored, perhaps this should be implemented at the storage layer? If each storage backend implemented an API to list secrets, then this would be fairly trivial to implement across vault.

If this issue is accepted, but not currently planned, would the above approach be acceptable as the basis for a PR to add this support?

dalehamel on 31 Aug 2015

@dalehamel Part of the problem is scoping out what should be done. Even looking back at the comments in this issue, there are about 4 or 5 different things that different people are asking for. For any ask it needs to be well understood not only how to implement it and what that interface should look like (plus output format), but also whether there are any possible security implications.

Edit: Also, you are correct in that for the generic backend at least it maps pretty much 1:1 with the underlying storage. So you can easily do this yourself right now by finding the paths in the backend storage but using Vault (with appropriate tokens) to get the actual decrypted values.

jefferai on 31 Aug 2015

@jefferai when a token is generated, you specify which policy that token is associated with. Is there any way, when a token is received, to expose this? What I mean is, does the mapping of token -> policy exist in the vault, or does it just attempt to decrypt, and then fail or succeed?

If it IS the case that there is this mapping, then this in tandem with the ability to provide an interface for each storage provider to implement would allow us to do something like 'show me all secrets I have access to', as we could first enumerate all secrets from all storage backends, and then apply the policy that our token can access to those paths, and return the resulting list.

You are correct though, that this is definitely something that raises a security issue - providing a list of what tokens a credential has access to makes a breach much more damaging, as you now have a treasure map of everything you can steal all at once.

I would suggest that if this feature is implemented, that it should be completely opt-in, probably at the storage provider level (whether or not to enable listing the secrets from the provider - if it returns nothing, then you policy will have nothing to match against).

dalehamel on 31 Aug 2015

+1

This is an essential feature. Sometimes you forget what kind of secrets you stored in the Vault and keeping a standalone documentation is cumbersome. Allowing the root policy (and maybe definable for others) to list the secrets in a given path/mount point would eliminate the need for external documentation especially in small-scale setups.

Vultour on 1 Sep 2015

@Vultour I agree this is the main feature stoping me from utilizing vault.

sstarcher on 1 Sep 2015

I don't think this needs to be complicated.

Feature: Enumerating secrets

  As a security officer
  I want to enumerate secrets
  In order to audit secrets.

  Scenario: Enumeration enabled, root user

    Given secret enumeration is enabled
    And I have a root token
    When request a list of secret paths matching an optionally wildcarded path
    Then I see a list of matching secret paths.

The wildcarding can (and I think should) follow the rules used for policies.

I think that getting cute with fine-grained access control is asking for trouble.

sheldonh on 1 Sep 2015

Hello, hope it will help someone else. I have modified a bit the code provided by ShadowApex in order to have a html frontend that list all the keys available (sorry, the code is ugly but works) :
https://gist.github.com/Seraf/bedb5d779e846c2a71da

There's some values to change in {{ }} (it's an ansible template). There's also some hardcoded things as I use a different backend for each customer we have. It displays a html table which allow us to quickly guess the path and keys available.

Seraf on 1 Sep 2015

+1

jtormey on 10 Sep 2015

+1

mwildehahn on 14 Sep 2015

vault ls secret/path would be awesome

jhaals on 15 Sep 2015

Currently slated for 0.4.

jefferai on 15 Sep 2015

Awesome! I'd love to help out any way I can, this is the last missing piece
for us using vault on a large scale

On Tuesday, September 15, 2015, Jeff Mitchell [email protected]
wrote:

Currently slated for 0.4.

—
Reply to this email directly or view it on GitHub
https://github.com/hashicorp/vault/issues/111#issuecomment-140401070.

dalehamel on 15 Sep 2015

That's great! I do realize 0.3 is not out yet ... what are the target dates for 0.3 and then 0.4?

Russell- on 15 Sep 2015

We don't give hard release dates. Currently 0.3 is scheduled for "quite soon", and look for some news about 0.4 when 0.3 comes out. :-)

jefferai on 15 Sep 2015

Fair enough :-).

Russell- on 15 Sep 2015

As mentioned by @jhaals getting a list of all secrets given a path would be essential for us. We'd like to group our secrets by domain (and possibly other groupings) and we need "admin" users to be able to list all secrets in a domain. So to generalize finding all secrets under any arbitrary path (e.g., secret/path1/.../pathn) would be needed.

Russell- on 18 Sep 2015

For those who want to live on the edge, I'll point you to https://github.com/hashicorp/vault/tree/f-passthrough-list which allows some manner of listing in the generic and cubbyhole backends.

Please note that this is not official and the eventual solution is likely to be different -- it may use a query parameter to GET instead of LIST (or support both), it may work differently to accommodate the needs of other backends, and so on. It also has a known issue in that while you cannot read a path you do not have access to, you will likely be able to see that such a path exists, which may be a violation of your security needs.

However, since it is written and public in its current form, I'm pointing it out for those that really need some manner of support right now. You will be responsible for merging any changes in to your local branch when you like, and I cannot promise any manner of support or stability.

jefferai on 18 Sep 2015

(Note: even if I get the info into the CLI output, don't rely on the CLI output's format to stay consistent. Use the HTTP API for this.)

jefferai on 18 Sep 2015

I'd like this functionality also extended to the auth backends where appropriate. An example would be to be able to list user-id mappings in app-id like this:

vault read auth/app-id/map/user-id/*
user-id       app-id
i-f1762018 foo_app
i-ca3b2324 app_1
...

st33v on 10 Oct 2015

I'm in the same boat as @st33v. Being able to list user-id, so that we can purge non-existing machine UUIDs from the secret store, is the main thing stopping us from using Vault.

I would prefer this to not require the root policy, but maybe a sudo on auth/app-id/map/user-id.

mwitkow on 25 Oct 2015

Also chiming in, would be really useful to be able to list mappings in app-id auth.

ben-nz on 26 Oct 2015

And the same thing for tokens.

lassizci on 28 Oct 2015

"jefferai modified the milestone: 0.4.0, future"

Sadness. :(

Any idea when this will get back on the priority list?

bkrodgers on 6 Nov 2015

In 0.5 we're planning on reworking ACLs. The current plan is to separate out read/write (so write doesn't include read), and along with those two add list and delete permissions. At that point, we can start adding list operations into the backends...probably as they are rewritten to v2.

jefferai on 6 Nov 2015

That's great news. I am sure that a lot of people simply can't adopt vault at this point due to the inability to search secrets.

babilen5 on 11 Nov 2015

+1 for being able to list registered app-id's and user-id's.

geoffreymitchell on 13 Nov 2015

+1 for being able to enumerate vault contents

J-Walling on 16 Nov 2015

i second what @babilen said

DanyC97 on 2 Dec 2015

@babilen @DanyC97 - just FYI, if your backend is consul, you can search the keyspace, by querying consul directly and not using vault:

(curl -v -X GET 'http://localhost:8500/v1/kv/?recurse'; echo) | jq '.[].Key' | grep vault/logical/

This works for secret mount. I currently only use the single secret mount. Not sure how well it plays with other mount types or multiple secret mounts.

Still +5 for getting the search in vault itself!!

wwalker on 28 Dec 2015

❤1 👍1

Most back ends will give you some way to list secrets. What we are asking
for us a common interface via vault so that it's fully supported.

On Sunday, 27 December 2015, Wayne Walker [email protected] wrote:

@babilen https://github.com/babilen @DanyC97
https://github.com/DanyC97 - just FYI, if your backend is consul, you
can search the keyspace, by querying consul directly and not using vault:

(curl -v -X GET 'http://localhost:8500/v1/kv/?recurse'; echo) | jq
'.[].Key' | grep vault/logical/

This works for secret mount. I currently only use the single secret mount.
Not sure how well it plays with other mount types or multiple secret mounts.

Still +5 for getting the search in vault itself!!

—
Reply to this email directly or view it on GitHub
https://github.com/hashicorp/vault/issues/111#issuecomment-167465006.

dalehamel on 28 Dec 2015

how can I decrypt the information in my consul backend?

eladapps on 21 Jan 2016

I'm trying to get my created token information from consul. I can see that I have keys created under token , how can I decrypt the information there?

eladapps on 21 Jan 2016

A note: the original request of this ticket has been fulfilled by the merge. For other listing requests (e.g. "I want to be able to list X in Y backend"), please open separate tickets so those can be tracked properly.

jefferai on 22 Jan 2016

Hi guys, i've released small workaround that can be very helpful.
Here is it: https://github.com/Mykolaichenko/gotools/tree/master/vaulter

It extends vault, and provide methods like 'tree' and 'search' for usability.

Mykolaichenko on 12 Oct 2016

@Mykolaichenko Does it implement anything which #617 doesn't?

JensRantil on 12 Oct 2016

@JensRantil yes, list is very old and simple feature. It shows only that directory containing.
vaulter makes it recursively, can get all tree of objects, and supports regex searching.

Mykolaichenko on 12 Oct 2016

Is it possible to submit a PR instead of forking it please ? That's really nice features :)

Seraf on 12 Oct 2016

Re: displaying the list of secrets as a tree, see https://github.com/hashicorp/vault/pull/2135

$ bin/vault tree cubbyhole/
how/
├── deep/
    ├── does/
        ├── this/
            ├── go
test

It needs work still. But it's a start...

msabramo on 29 Nov 2016

@msabramo: My preference would simply to have a list, irrespective of the tree metaphor.

- how/deep/does/this/go
- how/deep/does/this/need/to/go
- how/deep/does/that/go
- how/deep/do/they/go

skyzyx on 29 Nov 2016

@skyzyx: Oh that's interesting. That is simpler to implement and I could see that being accessible with vault list -recursive how.

Would other folks like that as well? I'd be happy to implement that if there is some consensus and the HashiCorp folks are amenable.

msabramo on 30 Nov 2016

I'm not currently sure whether we are amenable to this (we haven't been in the past) but my immediate thought seeing your PR was the same as @skyzyx -- fancy formatting is not a good idea and the output of such a feature should be a flat list.

jefferai on 30 Nov 2016

I like the recursive feature and would prefer flat list output.
Post-processing into tree format feels like a contrib concern.

On Wed, Nov 30, 2016 at 8:41 AM, Jeff Mitchell notifications@github.com
wrote:

I'm not currently sure whether we are amenable to this (we haven't been in
the past) but my immediate thought seeing your PR was the same as @skyzyx
https://github.com/skyzyx -- fancy formatting is not a good idea and
the output of such a feature should be a flat list.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/vault/issues/111#issuecomment-263788337,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACwKolkqLqnhSW0vS9aPb2r6xVqL4aKks5rDQyTgaJpZM4EMbmv
.

sheldonh on 30 Nov 2016

Cool. I wonder if we should move the comments to the PR: #2135

msabramo on 30 Nov 2016

From the comments, it seems that people prefer something simpler that just prints out all the keys without fancy tree line-drawing characters. I implemented this in #2142. Please leave comments on that PR.

msabramo on 30 Nov 2016

In case it helps anyone else, I provided a working Python snippet of code on a different Vault Github issue. Cross-linking cuz I'm messy like that: https://github.com/hashicorp/vault/pull/2142#issuecomment-336607207

pwhack on 14 Oct 2017

👍1

Is there a way to specificity a limit to a list query ?

zglozman on 16 Jan 2018

Any update on this?

christophetd on 28 May 2018

@christophetd list functionality was implemented long ago

jefferai on 28 May 2018

👎1

vault secrets list

does not list the secrets, it lists the secret engines.

'vault list' implements the required functionality but 'vault list -recursive' doesn't seem to be available yet in vault-0.11.1, even though a commit occurred Nov 2016. https://github.com/hashicorp/vault/pull/2142/commits

frankwaltz on 17 Sep 2018

Hi there,

vault list command does not implement -recursive option. The commit you
are seeing is from a closed PR.

Regards,
Vishal

On Mon, Sep 17, 2018 at 4:42 PM frankwaltz notifications@github.com wrote:

vault secrets list

does not list the secrets, it lists the secret engines.

'vault list' implements the required functionality but 'vault list
-recursive' doesn't seem to be available yet in vault-0.11.1, even though a
commit occurred Nov 2016.
https://github.com/hashicorp/vault/pull/2142/commits

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/vault/issues/111#issuecomment-422163551,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AC6YaP8ytqoURR5ex0dvbYzTXNzodTAvks5ucAkagaJpZM4EMbmv
.