Go-ipfs: Can pinning get a verbose argument?

[markg85]

Hi,

I'm currently executing ipfs pin add -r <QmHash>. It's been running for hours and i have absolutely no clue if anything is happening at all!

I'm missing a "verbose" flag. I'm sure IPFS is doing all kinds of stuff right now that would be visible if i raised the logging level, but that might be too much information. I'd like to see logs targeted at the current pin action to have some clue of what's happening. Obviously only if i provide the hypothetical -v or --verbose flag.

Cheers,
Mark

[djdv]

There is the --progress flag, which shows how many blocks have been processed. This will at least show you if things are progressing or not.
I would also like to see extended information though. Perhaps display the current block being processed, and maybe warn if there's 0 providers for it after some time.

[michaelavila]

@markg85 @djdv What does the first iteration of this look like? I think if we can get a simple first version described, this would be a good introductory task for someone to take on. Can we start by simply listing out each block we are attempting to pin and then a confirmation that we've received and pinned that block? Something like:

$ ipfs pin add -v -r <QmHash>
...
pinning QmHash
pinned QmHash

Is this too simplistic of a first version?

[djdv]

this would be a good introductory task for someone to take on

:+1:

Is this too simplistic of a first version?

I don't think so, any output seems better than none in this context.

In addition, I encounter people asking "if it's working" often enough, that I wonder if we shouldn't also change the default output. Making it noisy by default, and adding -Q and -q in the same way they appear in ipfs add --help.
At the very least it would improve command consistency.
But this doesn't have to necessarily be part of the first version.

[markg85]

It turned out, in my case, that IPFS did absolutely nothing. Or rather, it did actually pin in 0.4.18 (the current when reporting this issue) and did nothing in 0.4.20 (today's current). Yeah, go figure that one out.

I wholeheartedly with @djdv. Whatever output is figured out, it's better then we have now.
This might be a good start imho:

pinning QmHash
pinned QmHash

The time would just show how long it took to pin the file.
It could be verbose like this or simple like "01.20".

[michaelavila]

@markg85 can you elaborate on what you mean by:

It turned out, in my case, that IPFS did absolutely nothing. Or rather, it did actually pin in 0.4.18 (the current when reporting this issue) and did nothing in 0.4.20 (today's current). Yeah, go figure that one out.

Are you saying that you can't ipfs pin add <hash> as of 0.4.20?

[markg85]

@markg85 can you elaborate on what you mean by:

It turned out, in my case, that IPFS did absolutely nothing. Or rather, it did actually pin in 0.4.18 (the current when reporting this issue) and did nothing in 0.4.20 (today's current). Yeah, go figure that one out.

Are you saying that you can't ipfs pin add <hash> as of 0.4.20?

In general, that would be too bold of a statement.
In my specific case, the answer is currently and unfortunately yes.

I had reported it months ago here: https://github.com/ipfs/go-ipfs/issues/5977
And this comment specifically was from a newer IPFS version then the one i reported it with: https://github.com/ipfs/go-ipfs/issues/5977#issuecomment-480347589

It is know. The cause isn't. The fix isn't either.

Please do note that i did share my setup and ssh keys with another IPFS dev (although that was for CPU utilization issues) which got greatly improved since then (will be the 0.4.21 release i think). I however quit using or trying IPFS for a while as it only seems to give me headaches in either CPU issues or pinning not working or not working properly (aka does pinning, but very VERY slow). So I'ill wait a couple more releases till i give this another shot.

But lets not pollute this issue with the details of #5977 any further :)

[michaelavila]

@markg85 understood, thanks for the follow up

[reinerRubin]

I am going to try this one. I have checked how the current progress option works and think to add more info for pin/command to show.

[michaelavila]

@reinerRubin it is worth discussing your proposed solution here before taking the time to implement it. We can help avoid a lot of rework by getting you the feedback early.

[reinerRubin]

I think to expand the "ipfs/go-merkledag.ProgressTracker" with events and populate it around "ipfs/go-merkledag/EnumerateChildrenAsyncDepth".

Events could be a slice or channel of structs (probably a slice could fit better because of buffer, but idk buffered channel with "default" skip seems also ok). And these structs would consist of a block info, an action, maybe some meta.

So the pin command would track these events and show some additional stat.

idk, if it makes sense.

[michaelavila]

@reinerRubin that sounds reasonable. If I were you, I would do a rough version and get it up in a PR for feedback before diving deep and spending a lot of time.

It looks like we currently report either progress during the pin or the list of pins after completion, but not both at the same time. Here is some code that does some printing of Pins and here is some code that prints progress. I suspect there will be changes in these areas. Maybe a good first version would be to simply print pinned QmHash as they occur rather than all at the end?