Go-ipfs: `ipfs ls` doesn't behave as expected on symlinks and regular files

[overbool]

I have two type ipfs path QmRi5n1jzAtqGWQKJbeV4Sv3kGPt3XDfzhWojxiCVpKVdv(is a symlink) and QmYuz8WeF7LbwmQB1enoMsEPgFoB2EYaxrEyxGXwceGsmZ (just a regular file).

$ ipfs ls QmRi5n1jzAtqGWQKJbeV4Sv3kGPt3XDfzhWojxiCVpKVdv
$ ipfs ls QmYuz8WeF7LbwmQB1enoMsEPgFoB2EYaxrEyxGXwceGsmZ

both of them will print nothing.

But

$ ipfs file ls QmRi5n1jzAtqGWQKJbeV4Sv3kGPt3XDfzhWojxiCVpKVdv
Error: cannot list symlinks yet
$ ipfs file ls QmYuz8WeF7LbwmQB1enoMsEPgFoB2EYaxrEyxGXwceGsmZ
QmYuz8WeF7LbwmQB1enoMsEPgFoB2EYaxrEyxGXwceGsmZ

The first command will print a error and second one print itself.

I think ipfs file ls is more reasonable than ipfs ls at this point.

WDYT?

@schomatis @Stebalien @magik6k

https://github.com/ipfs/go-ipfs/issues/5497

[Stebalien]

Ideally, these would use the same logic. But I'll defer to @schomatis on this one.

[Stebalien]

(or @kevina, I just know that schomatis has been working on MFS and unixfs relates stuff lately)

[overbool]

@schomatis
https://github.com/ipfs/go-unixfs/issues/17 and https://github.com/ipfs/go-unixfs/issues/3 are being fixed.

Maybe this issue could wait until those issues mentioned above are fixed.

https://github.com/ipfs/go-ipfs/issues/5497 the same.

[schomatis]

Yes, ipfs ls has always been a mess.

The ipfs ls command works more on the DAG/UnixFS layers whereas the ipfs files ls operates strictly at the MFS layer,

  ls <ref>                       List links from an object
  ipfs files ls [<path>]         List directories in the local mutable namespace.

But before jumping to any fix I would actually need that someone would clarify what's expected from those two commands, they don't seem the same to me (the plain "object" term is actually meaningless to me).

@Stebalien @kevina

[kevina]

@schomatis there is a third command that I didn't even knew existed until a few days ago (while working on the --cid-base flag): ipfs file ls. This is different from ipfs files ls.

I agree ipfs ls needs to be cleaned up so that ipfs file ls is no longer necessary.

[overbool]

Currently, some changes were made to the ipfs ls.

1. file size
Lists file sizes that match the content size, instead of old ipfs ls that will print size including fanout links and the typing information.

$ ipfs ls <dir-path> -v
// ouput
Hash                                           Size Name
QmdyxAUG6ZQ3gXVAmC4bSBNmLoCjePK6Frq7DSjBx4XWSs 14   bar
QmVW1c7cxuAyD49Wv6XxWDBoXFij66xo6U9nazM9QFdV6v 65   hello/

2. symlink:
It will print Error: cannot list symlinks yet just like ipfs file ls.

$ ipfs ls <symlink-path>
// output
Error: cannot list symlinks yet

3. list a regular file:
It will print the regular file hash and size instead of nothing, but the name will be empty(maybe it could be better).

$ ipfs ls <file-path>
// output
QmdyxAUG6ZQ3gXVAmC4bSBNmLoCjePK6Frq7DSjBx4XWSs 14

$ ipfs ls <file-path> -v
// output
Hash                                           Size Name
QmdyxAUG6ZQ3gXVAmC4bSBNmLoCjePK6Frq7DSjBx4XWSs 14

WDYT?

@Stebalien @schomatis @kevina @magik6k

[schomatis]

ipfs file ls. This is different from ipfs files ls.

:fearful:

[rob-deutsch]

Can I suggest that some serious thought should be given to what's trying to be achieved here - and maybe it would be beneficial to take a step back and consider the broad picture: what should be the structure of the commands for ipfs? If there isn't a master plan then we risk running in circles trying to make things logical and consistent.

For example, with respect to this bug report, it could be suggested that it's not a bug. The ipfs --help text clearly states that they _should_ have different output:
BASIC COMMANDS ls <ref> List links from an object
and:
SUBCOMMANDS ipfs file ls <ipfs-path>... - List directory contents for Unix filesystem objects.

That being said, while not a bug, I think that the current command setup is.... counterintuitive. My initial reaction is that:

1. All of ipfs's basic commands (add, cat, get, ls) should operate on the unixfs level, because that is the level that most aligns with ipfs' desired purpose as a filesystem.
2. ipfs refs should be removed - its handled by ipfs object links
3. ipfs file ls should be removed - it will be handled by ipfs ls
4. ipfs files should be renamed to ipfs mfs or ipfs vfs because its purely for the memory/virtual file system.

HOWEVER, I might be misinterpreting things, and ultimately I might be missing things because I'm not looking at the bigger picture.

[schomatis]

Can I suggest that some serious thought has to be given to what's trying to be achieved here - and maybe everyone needs to take a step back and consider the broad picture: what should be the structure of the commands for ipfs. If there isn't a master plan then we risk running in circles trying to make things logical and consistent.

Yes, that's what I meant with my previous comment, but I think you have expressed it much more clearly. Non of the listed issues have any technical complexity, they are all low hanging fruit for someone to fix it, but I'd want to avoid posting help wanted issues just to change our mind in the middle of a submitted PR (where a contributor will have already committed a considerable effort) because the objectives were not clear from the start.

[rob-deutsch]

(You caught me pre-edit, but all did was soften my language a tad :P)

With regards to working towards a solution, I have a suggestion: Should we create a text representation of all ipfs commands in a tree like structure - and then hack away at it until we come up with something elegant and (relatively) future proof?

Maybe a gist would be a good way to to manage revisions and suggestions for it....

[kevina]

Maybe a gist would be a good way to to manage revisions and suggestions for it....

Unless something has changed recently, GitHub does not send notifications out for gist comments, which will make it difficult if not impossible to keep up with. A better idea is to create a p.r. in the form of a documentation. This repo may not be the best place, but I am not sure where a better place may be. An other alternative is to create a new repo _just_ for this purpose, populate it with the initial command list, then create a p.r. to modify it.

Note, work towards this may already be on the way elsewhere. As a non-core memeber it is very difficult for me to keep up with all that is going on.

This work will also effect js-ipfs.

CC @Stebalien @whyrusleeping @diasdavid

[rob-deutsch]

GitHub does not send notifications out for gist comments

That's a good point. A gist probably isn't the right place.

For what its worth, to help look at the bigger bigger picture, I've put together a text document with all of go-ipfs commands in a human readable tree. I've written some very horrible Go code to create this:

https://gist.github.com/rob-deutsch/2a865aea11e05064d2b6e208d787a6bb

[redraw]

Is there a way to list all files/folders added? i.e by ipfs add command

[dokterbob]

Could someone change the title of this issue to be more specific?

[Stebalien]

@redraw in an effort to keep issues on-topic, please ask unrelated questions on https://discuss.ipfs.io.