Go-ipfs: /ipfs/cached + /ipfs/pinned fuse dirs

[jbenet]

would be nice to list the hashes stored locally under /ipfs with special (well known) directories:

• /ipfs/pinned for all hashes pinned locally
• /ipfs/cached for all hashes available locally, and not pinned.
• /ipfs/local all hashes available locally (pinned U cached)

[ebuchman]

How best to track cached (non pinned) objects?

• KeyList() (overheard probably too high)
• index file (is the cache flushed? if files are not pinned, will we drop them? if so this may stay small and possible advantage of human readable, otherwise probably no good)
• datastore query (with something like index/<tree#>)

[jbenet]

To go along with the fuse files, we should also have a command. maybe it's just an extension of refs

# outputs all refs stored locally
ipfs refs local 

# outputs only the pinned
ipfs refs --pinned

# complement
ipfs refs --cached

[jbenet]

Update: this is possible to implement now. the blockstore has {AllKeys, AllKeysChan} functions

[whyrusleeping]

Further update:
I want to have every ipfs add make an entry in the files api space, maybe under /pins or something. With that implemented, we could almost (except for the advent of direct pins) remove pinning entirely, and replace it with 'copying to your local filesystem' which is an idea i really like.

[daviddias]

:+1: to this idea (and also making files add a thing)

[whyrusleeping]

in the future, we may even move ipfs add to ipfs files add. This would help with separating the ideas of DAG and unixfs better than we are now.