Code-server: support other encodings than UTF-8

Created on 21 Sep 2019 · 27Comments · Source: cdr/code-server

I have fixed this issue in my local environment, and I'd like to make a pull request if you want.

enhancement upstream

Source

LeuisKen

Most helpful comment

Maybe we just wait for Microsoft to get it in.

See https://github.com/microsoft/vscode/issues/79275#issuecomment-637337120

Should be in the next couple weeks and their's will be much smaller.

https://bundlephobia.com/[email protected]

In @LeuisKen's PR text-encoding adds 200 KB of minified code! For reference react-dom is 35 KB.

nhooyr on 3 Jun 2020

👍2

All 27 comments

This will require a upstream update.

sr229 on 21 Sep 2019

Related: microsoft/vscode#79275

I have fixed this issue in my local environment, and I'd like to make a pull request if you want.
I also found vscode cannot reopen with other encodings like gbk, could you help me?

SysuXiaoYu on 3 Feb 2020

@SysuXiaoYu feel free to do so and regenerate the patch for us.

sr229 on 3 Feb 2020

@LeuisKen how did you fix it locally?

gyzerok on 12 May 2020

@LeuisKen how did you fix it locally?

https://github.com/LeuisKen/vscode/pull/1/files

LeuisKen on 12 May 2020

@LeuisKen thank you! Does it also allow to switch file encoding?

gyzerok on 12 May 2020

@LeuisKen thank you! Does it also allow to switch file encoding?

I find some way to do this, but this patch need to import another third party lib called text-encoding, and I didn't publish this internal patch. Hope vscode will fix this issue soon.

LeuisKen on 12 May 2020

@LeuisKen, can you also describe what would be the correct way to get your fix into the code-server?

I would like to apply you fix (at least to my local fork of the code-server), but I am afraid it'll take me hours to figure out how to do it properly :)

gyzerok on 12 May 2020

Maybe you can even make a PR to this repo with your patch?

gyzerok on 12 May 2020

Maybe you can even make a PR to this repo with your patch?

I may not have time to do this until weekend, and other features like guess encoding I didn't fix.
I'll send a pull request to code-server this weekend if you really need it.
I think that VS Code team will not fix this problem in my way, and I am worried that this patch will make the code server and VS Code upstream code further and further away.

LeuisKen on 12 May 2020

@code-asher Can you take a look at @LeuisKen's PR?

nhooyr on 12 May 2020

No problem.

code-asher on 13 May 2020

@LeuisKen let me describe you my use-case, so you can tell me if your code will fix it or not (not sure I know enough vscode to understand this myself 😅 )

In the codebase I am working with there are 2 languages used: PHP and JS. Unfortunately *.php files have one encoding and *.js files have another. And currently it will not change in the forseable future. And I need to be able to save the files in the encoding they are currently.

I'll send a pull request to code-server this weekend if you really need it.

So as you can see from my description, I really need it :)

In the desktop version of VSCode I can do it with:

// workspace settings json
"[php]": {
  "files.encoding": "encoding"
  // error: this setting does not support per-language configuration
}

This doesn't work probably because code-server does not support top-level files.encoding setting at all.

How hard do you think it will be to add support for files.encoding setting on top of your changes? Do you think then it will address my use-case?

I think that VS Code team will not fix this problem in my way, and I am worried that this patch will make the code server and VS Code upstream code further and further away.

It seems like the Coder team are ok with the change. And for me this change will be the savior - without it I can't use code-server for half of my project.

So if you can fix this for me you will be my hero 😄

gyzerok on 13 May 2020

I understand and I'll try to help with your case.

LeuisKen on 13 May 2020

https://github.com/cdr/code-server/pull/1650

LeuisKen on 17 May 2020

Thank you for taking care of it!

gyzerok on 18 May 2020

In the related VSCode issue there was just posted an interesting answer: https://github.com/microsoft/vscode/issues/79275#issuecomment-629952916

gyzerok on 18 May 2020

😄1

Maybe we just wait for Microsoft to get it in.

See https://github.com/microsoft/vscode/issues/79275#issuecomment-637337120

Should be in the next couple weeks and their's will be much smaller.

https://bundlephobia.com/[email protected]

In @LeuisKen's PR text-encoding adds 200 KB of minified code! For reference react-dom is 35 KB.

nhooyr on 3 Jun 2020

👍2

Yeah, I've decided to give it a try and contribute the feature.

However so far it doesn't look like the result will be smaller. 200 KB you are referring to is minified and gzipped version. Just minified one is 535 KB according to your link.

Currently iconv-lite, which we are planning to use in the browser is 600 KB minified. As you can see it is even bigger.

So I am not sure the solution I am working on now will result in smaller footprint. However it will probably be more "correct" because it will reuse code between electron version and browser version as well as will use stream interface hence smaller memory footprint.

gyzerok on 4 Jun 2020

Mhmm. Sounds good. Let's wait for your stuff to get into upstream then.

nhooyr on 4 Jun 2020

If it ends up taking a while upstream we can just make text-encoding lazy load when we need to use a non UTF-8 encoding and that'd be ok.

nhooyr on 4 Jun 2020

I am hoping it can go to upstream for June release. At least I’ll try my best to make it happen.

Alternatively if it gets blocked there, I can finish it up anyway and we can just apply my changes here if you’ll be up for it.

Currently it gives me a lot of pain, cause I have to work on half of my project files locally and rsync them back and forth 😅

gyzerok on 4 Jun 2020

@nhooyr finally all of the work needed for this is merged to upstream.

Should be in the next couple weeks and their's will be much smaller.

In the end we were able to make the size much smaller, so you were right 😄

Unfortunately it didn't make in time for the June release. Would it be possible to update code-server so that it includes the fresh release (we seem to be lagging behind now) plus commits needed for this feature to work?

gyzerok on 5 Jul 2020

Would it be possible to update code-server so that it includes the fresh release (we seem to be lagging behind now) plus commits needed for this feature to work?

Definitely! We'll try to get on this at some point but welcome a PR as well.

nhooyr on 5 Jul 2020