Vscode: Provide option to opt out of line ending normalisation for files

[ghost]

Upon saving a file edited with VS Code, it appears to perform line ending normalisation automatically and without user notification of any sort. Because of this, I've found myself bitten by large diffs in Git (I'm aware you can circumvent this using the -w flag) where a trivial fix to a single line of source code appears to affect a significant proportion of the file, which makes pull requests and reviews for such changes on GitHub a pain to sift through. Then again, I guess one could argue why a source file with mixed line endings should stay in version control like that anyway!

Visual Studio Community displays a prompt upon opening a file if it has mixed line endings and lets a user decide whether or not to have it fixed.

Version: 0.10.1

(By the way, great work on open sourcing VS Code too, thanks!)

[alexdima]

The current file model normalizes the line endings as soon as the file is read (i.e. each line does not keep track of its EOL), the model only keeps track of one EOL for the entire file (which is computed on file open as the predominantely used EOL character).

@miqid Before we invest a lot into supporting this, when is it desirable to have a file with code that uses mixed EOL sequences? (i.e. why would you want to have a mixed EOL file at all)

[ghost]

Thanks for the insight @alexandrudima - a bit gutted to hear it won't be a trivial change.

Personally, I don't believe having mixed line endings around is a sane thing at all. Where the normalisation becomes a problem for me is when I need to make small edits in source files part of huge code repositories owned by people (usually in the commercial industry) who are _extremely_ stringent on changes they allow through.

I've instead been using the Atom editor to commit changes to files where line endings aren't normalised as it preserves the original line endings of a while on save.

If it is going to be too much effort, I'm happy for the feature request to be closed off as there are alternatives beyond VS Code.

[sleaze]

+1 to that!

[matlimatli]

This would be important when editing files with a binary payload. Think shellscripts with a built-in binary blob, or data files with a plaintext header part.
For me, it would be OK if I could select a number of file extensions for which no conversions are ever applied.

[XFunc]

Clearly, having mixed line endings in source code files is not sane at all. But this is something you often have to deal with in cross-platform development environments (Windows developers not aware at all of the EOL they are inserting in source code files, since the very beginning of the project).
Having the possibility to choose or not EOL normalization would then be great for such scenarios.

Thanks

[gwax]

In my case, I would like to edit a .gitignore file with \n for line endings that contains an entry to ignore OSX Icon\r\r files. These are the only \r in the entire file and stripping them will break the file's behavior.

[smlam]

EOL normalization should be optional feature and off by default.
We do not expect file editor will change non-edited code without user conscious.

Thanks

[shreyasminocha]

@gwax Icon? works for me

[gwax]

@shreyasminocha Yes, Icon? works when people are aware of the issue and remember to watch out for it. The risk is that a .gitignore file exists with a Icon\r\r entry and is edited by someone that does not know to watch out for line ending normalization.

That person comes along, adds an entry to the .gitignore, doesn't see the change from Icon\r\r\n to Icon\r\n or Icon\n, commits it and breaks the file for other people.

The crux of the issue is that the editor is making changes to the rest of the file without warning or explicit configuration request.

Compare to the behavior of the (similar) "Trim Trailing Whitespace". The feature also potentially edits untouched lines of a file but must be enabled by configuration before it will modify anything.

[NoK-Poijes]

Is there a way to turn off this line ending normalization?

[renehamburger]

I've come across another scenario where the automatic normalisation of line endings is not desirable: an SQL export, which contains LF as the main line ending at the end of each row, but in my case also CRLF line endings within fields that contain multi-line text.
I'm able to align these in my case, but I think this example shows that there are valid cases where line endings should not be automatically normalised.

[nour-s]

I have the exact issue when making a pull request. I get surprised that the whole file is changed when I just changed one line!
What makes it even worse, is that I only know that after I create the pull request and someone else review it !

I work on both Mac and Windows, and I believe switching between them is what makes this problem.
Any way to disable this?

[mathias-a-la-basse]

Please fix this issue since this is not just to be more comfortable, this is a mandatory feature of any modern editor:

how do you work with XML having CDATA entries with different lineendings than the surrounding XML document??

I just can't edit such a file in VSCode, since it will thrash the CDATA parts, or the XML part.....
I guess I should stick to Notepad++.....

Please, there should be a mode like in Notepad++:

1. VSCode try to guess the default line ending --> show CRLF / LF.
2. The default is only used when typing ENTER key
3. there is no other alteration of the file
4. VSCode can display the true line endings character (realy this should be core feature, not an extension!), and show up with Toggle Controle Character (because they are, really)
5. the user can choose to normalize the file OR normalize the selection

[chevtek]

Completely agree with the above. This should be a core feature. I shouldn't have to install an extension to see the line endings either. I expected the show control characters feature to show line endings but it did not and I had to hunt for an extension.

[JoeHartzell]

I also would like to agree with the above statements. We for one develop on windows but then use linux docker containers. If we have to write shell scripts for docker using vscode, it usually causes issues. This is especially true if a developer tries to just modify one line, they have to be aware and make sure they set the line ending back to LF.

[sleaze]

Can't believe this still isn't fixed!

I decided to move on to Spacemacs and haven't looked back..

[siqiniao]

ooh, Can't believe this still isn't fixed!

[sonicpro]

Will this option help to see the difference in line endings in the "diff" window as well? Currently that window does not visualize the difference in line endings.

The use case is like the following: I run "d2u <filename>" in bash terminal thus changing the CRLFs to LFs. Then I open diff through "Changes" in the "Source Control" Activity Bar item. The diff window does not visualize the difference, The screenshot is attached.

[ddedkov]

I have to agree, this is undesired behavior, you should make it possible to turn of this normalization , keep it on by default, but give us an option, please.

[rvjc]

I agree, this is the kind of "automatic cleverness" that gives modern technology a bad name.
VSCODE is a great IDE but must resist temptation to get over-complicated.
Programmers need to be in control; they need to know about line-ending issues.
We use unix line-endings on text files across the board, whether developer is running on Windows, Mac or Linux.
We just use editors that work with unix line endings.
Please fix this with great urgency.
Thanks

[vesper8]

I'm entirely unable to submit a PR because of this, as soon as open a file and save it even without any formatting or changes, it renders the entire file as a diff

[Prinzhorn]

I'm not using VS Code, but I'm evaluating Monaco. And all line ending related issues in Monaco point to this, so here I am.

Since Monaco was extracted from VS Code, we are not longer talking about "files". Monaco works on strings. Strings don't have a concept of line endings. I would expect getValue(setValue(s)) === s for every string s.

My current use-case that I'm evaluating Monaco for is editing raw HTTP requests/responses. Obviously I absolutely need to preserve CRLF for the header block. But the body on the other hand might follow completely arbitrary other rules, not related to HTTP. If we're just talking about the headers, I could use getValue({lineEnding: '\r\n'}) to force normalized line-breaks. But that still leaves the body, which can have a mix of everything that needs to be preserved at all costs.

Let's check how close Monaco gets to getValue(setValue(s)) === s.

Let us create a Monaco instance with a string of every single extended ASCII character:

let chars = [];

for (let i = 0; i < 256; i++) {
  chars[i] = i;
}

let value = String.fromCharCode.apply(String, chars);

let editorInstance = monaco.editor.create(editorElement, {
  value
});

console.log(value === editorInstance.getValue());
console.log(JSON.stringify(value));
console.log(JSON.stringify(editorInstance.getValue()));

This will output

"\u0000\u0001\u0002\u0003\u0004\u0005\u0006\u0007\b\t\n\u000b\f\r\u000e\u000f\u0010\u0011\u0012\u0013\u0014\u0015\u0016\u0017\u0018\u0019\u001a\u001b\u001c\u001d\u001e\u001f !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ"

"\u0000\u0001\u0002\u0003\u0004\u0005\u0006\u0007\b\t\n\u000b\f\n\u000e\u000f\u0010\u0011\u0012\u0013\u0014\u0015\u0016\u0017\u0018\u0019\u001a\u001b\u001c\u001d\u001e\u001f !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ"

If you look very carefully, you can spot the difference. \r was turned into \n. Every other character, _including control characters_ (even the zero byte), was preserved. The only thing that was changed was the line ending. It's time for a change. Let's remove the change.

Thanks for coming to my TED talk.

[csatacsirke]

I'm working with PDF files, and i have to view them as text many times. So I had the idea to write an extension with jump to definition functionality, to speed up debugging. It worked wellfor some files, then again, if they contained mixed line endings, which definitely does happen, vscode "corrects" them as soon as i load a file, and i lose the corss-reference table functionality(which works with byte offsets), and thus the parsing capabilities. I would definitely need the option to preserve line endings.

[GeeekerAndy]

I cannot believe it the whole file changed after editing just one line...it's terrible.

[finn-matti]

Concrete examples where this is a problem (from this thread):

• PRs/Changes to repos with mixed line endings become impossible to do (because much bigger diffs are generated by the normalization)
• You might think that mixed line endings are a bad idea, but are not in a position to change current policy
• Makes editing files with binary payloads impossible (shell scripts with binary blob)
• Cross platform dev teams might suffer from normalization ping pong
• editing .gitignore files with Icon\r\r lines in VS Code will break these files behavior (in any case everyone has to be aware of the issue to make the workaround of using Icon? work)
• SQL exports which contain LF as line endings but also CRLF within multi-line text fields
• XML documents with CDATA entries with different line endings cannot be edited savely
• unable to editing raw HTTP requests and responses through the Monaco Editor
• Working on PDF files with mixed lines endings becomes impossible

My own example: files for tests (since some external systems work with differnet line endings) cannot be edited in VS Code. If for some reason I have to edit a testfile with mixed line endings I will lose what a test case is supposed to test.

I don't know if about 10 different examples collected over 5 years is enough to persuade somebody to work on this. But it's a persistent open issue and therefore should at least be reevaluated, imho. I for one would love to hear some member giving this another look.

[genghisun]

OMG!!! What's going on here??? What on earth prevented the fix of this bug???

[chevtek]

Yeah this is long overdue. With the numerous real world examples of why this behavior completely prevents certain kinds of files from being edited in VSCode I would say that this is no longer a feature request. It's a bug. I love VSCode. Please fix this bug so we can finally stop switching to other editors for these situations.

[ddedkov]

can someone from vscode team chime in on why this is not treated as a bug?

[Nuullll]

A quick workaround under a git repo:

git diff > backup.diff   #backup first

git diff --ignore-cr-at-eol > tmp.diff
git checkout -- .
git apply --verbose --ignore-whitespace tmp.diff

You can run this script to recover unexpected EOL modifications made by VSCode.

[alianZhen]

So, is there any new plan for this.

[barak007]

This is really painful for us.
We have a cross os testing for file editing and formatting and this issue cause us to replace line endings every time to get the value from monaco.

[mbomb007]

This is one of the oldest issues in the queue overall. It's a shame to see no progress being made.