borg create --list: when to show a path/filename

[ThomasWaldmann]

borg 1.1.x prints the path/filename after completing the back up of that file.

for rather big files, this might take rather long and the user only sees the previous filenames, not the filename of the big file.

thus, the user might think, borg is hanging / not doing anything. Example: https://github.com/borgbackup/borg/issues/4762#issuecomment-538785655

also, if some file-reading related error or warning is output, it might be unclear to which file the error message refers.

this ticket is about changing this behaviour, to print the path/filename before starting to back up the respective file.

Bounty: https://www.bountysource.com/issues/93406062-borg-create-list-when-to-show-a-path-filename

[ThomasWaldmann]

There is an issue with showing filename first:

We also show the status and while reading the file, the status might change to E)rror, like:

A some_added_file
M some_modified_file
E some_file_that_had_a_read_error

[ThomasWaldmann]

Idea: before borg decides to call chunk_file (which reads, chunks, hashes, ... the file's content) we could emit the latest status (usually A or M for regular files) plus filename and signal via status = None (usually the return value of the file item handler) that no additional output is wanted after completion (needs some other code changes, status None is currently treated as invalid state).

So, if the file is read without errors, there will only be output before reading it.
If the file is read with error, there will be also E filename output when it errors processing it.

3 places to change: regular file, stdin, recreate - all call chunk_file.

Will affect normal log output as well as json log output.

[ThomasWaldmann]

Does anybody see an issue with this slight behaviour change?

[fantasya-pbem]

So the file with error will have double output (M somefile, E somefile)?

[ThomasWaldmann]

Yes, in case of an error.

[Gu1nness]

Wouldn't it be possible in this specific case to rewrite the line using r?
This way, we avoid deduplication, and make it easier to check the number of back-upped files?

[ThomasWaldmann]

Hmm, not sure if that is possible, have to check how that output is generated.

[Gu1nness]

@ThomasWaldmann Can you point me to the place where chunk_file is called ? I cannot find it in the source only in the builds :/
Then I can write the patch with the two lines, or the r if possible

NVM found it the name has changed since your comment, that's why I was unable to find it.

[ThomasWaldmann]

This change should be delayed until after PR #5492 (+ potential backport).

[Gu1nness]

Sure, I'll wait then!

[Gu1nness]

I have started working on this one, still WIP but close to an end

[ThomasWaldmann]

@Gu1nness i've just changed the milestone for this, we first want to have that in master for a bit.

Later we can maybe backport to 1.1-maint.

[ThomasWaldmann]

@Gu1nness please open a PR as soon as you have something, to get early feedback.

[Gu1nness]

Sure, I'll do this.

[ThomasWaldmann]

@Gu1nness are you still working on this?