Notepad3: Why Notepad3 wants to save file after recode?

[alexantr]

1. Open file
2. Recode (F8) file to any other encoding
3. Try to close file
4. Notepad3 prompts to save file but nothing changed, in fact

[RaiKoHoff]

Notepad3 uses always UTF-8 encoding internally for the document in memory.
The encoding setting is only used as the encoding for the file on the file-system, when it is written.
(... and an encoding analysis/detection is used to read a file from file-system and transform it to the UTF-8 encoding for memory - keeping it as an encoding for writing the file back to the file-system after change.
So, the only reason to change the encoding (while nothing changes in memory), is, to use another encoding on the file-system - so Notepad3 prompts you to use the new encoding for saving the file.
So offering to save the file (using the new encoding) is correct 👍 .

But: If you try to change the encoding to an encoding, which is not capable to encode every character (e.g. Japanese characters transforming to ANSI 1251 (Western European)) you will get garbage:

• Recode (= reload from FS with another byte interpretation): the byte-stream from file-system is interpreted as the bytes of the selected encoding, which may lead to very strange character display (of that encoding).
• Change Encoding in place (Encode -> More...): The UTF-8 characters are translated to the selected encoding - if this encoding has no corresponding character, it is translated to '?'.

[hpwamr]

Hello @alexantr ,
As far as I'm concerned, I think you (as requester) can close this issue...

[alexantr]

Okay

[alekhe]

The encoding setting is only used as the encoding for the file on the file-system, when it is written.
(... and an encoding analysis/detection is used to read a file from file-system and transform it to the UTF-8 encoding for memory - keeping it as an encoding for writing the file back to the file-system after change.
So, the only reason to change the encoding (while nothing changes in memory), is, to use another encoding on the file-system - so Notepad3 prompts you to use the new encoding for saving the file.
So offering to save the file (using the new encoding) is correct 👍 .

As for me, the name of the action “Recode” invoked on F8 was always misleading (it’s like ”change file’s encoding to another one”, but maybe it’s just my bad English). However, I understand it exactly as you described: ”Use another encoding for conversion when reading and writing file.” Nevertheless, let me join the discussion.

That is when I open a file and see garbage I want it to be opened using another conversion function. I don’t want to change its own encoding, I want to re-open it using another encoding function. Actually, file is not changed, it is still the same file but it is opened using another encoding. If I modify the file it should be saved with the new encoding which I’ve chosen with “Recode,” not the one which was used when the file was opened for the first time.

In fact, Notepad2-mod does not change the “modified” flag when you recode it.

[RaiKoHoff]

@alekhe : The alternate way would be to save the file immediately with the new encoding ("recoded").
Then you will not see an "file modified flag", but, if chosen the wrong encoding for "recode", this may destroy the file, which is not acceptable.

On the other hand, there are some encoding, which will not change the byte stream representation of the encoded file (e.g. all ANSI and UTF-8 (no SIG) containing only ASCII (bytes <= 127) characters),
they have the same byte stream in the file, in this case, it does not make sense to show the document modified flag, but it is easier to show the "document modified flag" for any encoding change in general.

[alexantr]

Can we say "Recode is the manual encoding selection with disabled autodetection"? Or Recode is encoding detection but human instead CED or UCHARDET.

[alekhe]

@RaiKoHoff There is no need to save the file with the new encoding, since the byte stream representation does not change. As @alexantr described it, this is just manual encoding selection for language specific representation of the file contents.

For example, I work with texts in Russian which might be cp866 (OEM) or windows-1251 (ANSI) encoded. I want to edit the file using the same encoding as it was used initially. If the file is cp866 encoded but opened with windows-1251 encoding I want to re-open it with the correct encoding (which would be cp866). The byte stream on the disk stays the same but the visual representation (in Cyrillic letters) should be changed.

[alekhe]

I’d like to mention another inconsistency in such a behavior (setting the “modified” flag on). If you reopen file with “recode…”, then save it, it will be identical to the original file.

[RaiKoHoff]

"Recode" is short for "_re-decoding_" - maybe an unlucky wording. The full meaning is "_reload file (byte-stream) and decode it, as if it has been encoded with the selected encoding_". So you manually bypass the encoding detector and force the selected encoding for decoding the byte-stream. Maybe "Reload As <encoding>" would be longer, but more expressive 🤔

After changing the encoding of an already loaded file, the modified-flag is set, because the encoded byte-stream (by new encoding) may differ (will differ in most cases) from the byte-stream of the existing file - a save file is needed.

After forcing a "_re-decoding_" of a file, the _modified-flag_ is set by the "_encoding changed flag_".
After thinking about it, this might be not necessary - I can't imagine a case, where encoding (save file) of a manually "_re-decoded_" will change the byte-stream of the file on saving - even if it has been forced to a wrong encoding 🤔

[alekhe]

I can't imagine a case, where encoding (save file) of a manually "_re-decoded_" will change the byte-stream of the file on saving - even if it has been forced to a wrong encoding 🤔

Usually, I need to _re-decode_ when the automatic detection fails. So it is expected that the manually selected encoding will be correct.

Note that if I select incorrect encoding (when I do _re-decode_) it is remembered in the history and when I open the file the next time, this incorrect encoding is used even if it cannot be used (e.g. decoding with iconv fails). The editor just shows a stream of bytes which cannot be decoded with the selected encoding.

[RaiKoHoff]

Feel free to test the "Portable BETA paf" version "Notepad3Portable_5.20.723.1_BETA.paf" or higher.
See the issue "Notepad3 BETA/RC-channel access #1129" or hereunder:

• Notepad3Portable_5.20.723.1_BETA.paf.exe.7z

_Note1:_ The "Notepad3Portable_paf" version can be used in "2 flavors", see with or without extension ".7z".
_Note2:_ If you follow the instructions of #1105 , you can update your "Notepad3 Setup" version with the latest features/fixes of the "Portable BETA/RC paf" version (see #1129).

[alekhe]

Looks fine. Thanks a lot!

[alexantr]

Try to press Esc with Notepad3Portable_5.20.723.1_BETA.paf. It set modified flag.

[RaiKoHoff]

Upps - yes, the ESC clears the selection (even if it is empty) which populates the (selection-) undo/redo stack. A non empty undo/redo stack triggers the modified flag. I an going to change that. The backside is: a cleared selection (by ESC) can not be restored by undo (Ctrl+Z) 🤔

[RaiKoHoff]

Feel free to test the "Portable BETA paf" version "Notepad3Portable_5.20.723.2_BETA.paf" or higher.
See the issue "Notepad3 BETA/RC-channel access #1129" or hereunder:

• Notepad3Portable_5.20.723.2_BETA.paf.exe.7z

_Note1:_ The "Notepad3Portable_paf" version can be used in "2 flavors", see with or without extension ".7z".
_Note2:_ If you follow the instructions of #1105 , you can update your "Notepad3 Setup" version with the latest features/fixes of the "Portable BETA/RC paf" version (see #1129).

[hpwamr]

Hello @alexantr ,
As far as I'm concerned, I think you (requester) can close this issue...