Notepad3: Remove duplicate lines slightly broken if a duplicate line is the last line

[ritmation]

For example, if we have the following document:

111
111
121

and invoke the "Edit->Lines->Remove Duplicate Lines" function, we get the following result:

111
121

the expected result is

111

@RaiKoHoff wrote about this issue
https://github.com/rizonesoft/Notepad3/issues/418#issuecomment-375634608, but I can't find a bug report, so I'm filing it.

Thank you.

[RaiKoHoff]

Edit->Lines->Remove Duplicate Lines (Ctrl+Alt+D) compares complete lines - including (invisible) line break characters.

• 1st line: "121\r\n"
• last line: "121EOF"

, so both lines are kept as they are different.
To prevent this, you have to maintain an empty line as the last line of those files.
On the other hand, we can change this behavior to come closer to the expected behavior:

• ignoring line break characters
• ignoring multiple space/tab characters (multiple spaces/tabs/mixed seen as one space -> compress space) ?
• ignoring differences between comma and semicolon ?
• ignoring ...

Open for discussion ...

[ritmation]

It seems that this was a misunderstanding on my part - it did not occur to me that the "Remove duplicate lines" compared lines including line breaks.
I would like to keep things simple, so I'm not feeling comfortable with changing the algorithm to ignore some of the characters when comparing lines, as it would probably introduce some other issues.
On the other hand, one has to keep in mind that one needs to add a line break after the last line before invoking the command, a thing which is easily overlooked if a command is used rarely.
Let's leave it as is for the moment, as introducing options into this command would unnecessary complicate things further.
Maybe we can return to this issue in the future, perhaps after a dedicated "Settings" dialog is introduced, as menus are getting longer and more cluttered with time.
I can close the issue if there are no other suggestions and/or objections.
Thank you.

[RaiKoHoff]

@ritmation : Thank you for your opinion. I think, we should leave this issue open to collect some other suggestions.

[lhmouse]

Then how about 123\r\n vs. 123\n? I don't think line breaks should be included.
There is an exception: If the last line is empty i.e. the last character(s) of the document is a line break then it should not be removed.

[RaiKoHoff]

I believe, the normal user's expectation is what @ritmation and @lhmouse expected.
So development build "_develop_4.18.601.999 (Dev_0601/93c0ebfc)" does not take care for line-breaks on comparison (but removes them too, if visible characters match). Please test.

[lhmouse]

looks good to me.

[hpwamr]

Hello @RaiKoHoff,
Tested with "_develop_4.18.604.1000". Indeed, it better matches the expectations of a "normal" user. 😏

[ritmation]

Unfortunately, I don't have access to beta versions and I'm kind of overwhelmed at work, so I can't test this before it makes to release, but I'm sure I'm going to like it. Thank you all.

[hpwamr]

@ritmation ,
In this place: Temp_Betas_Drive, you will find the latest beta files. 🤔

[lhmouse]

@ritmation You can clone RaiKoHoff's repo and checkout the desired branch then build one yourself.

[ritmation]

I've tested the 606.1002 build, everything works fine, except for one moment: in the case when the last line is a duplicate line, after invoking the remove duplicate lines command, document ends with a blank line, whereas if the last line isn't a duplicate line, it ends with that line and an EOF.
For example, if we have the following document:

121\n
111\n
111\n
121\EOF

After pressing Ctrl+Alt+D, we get the following result:

121\n
111\n
\EOF

Now, in second case:

121\n
111\n
111\n
121\n
222\EOF

After pressing Ctrl+Alt+D, we get this:

121\n
111\n
222\EOF

In my opinion, for the sake of consistency, "Remove duplicate lines" function shouldn't leave empty lines at the end of file if they weren't there to begin with.
Please share your opinion on this.
Thank you.

[RaiKoHoff]

@ritmation : Good finding, please test development beta "_4.18.607.1003".

[ritmation]

In the latest beta version everything works as expected.
If there are no other suggestions/objections, the issue can be closed.