Ckeditor5: Spaces are showing up as: Â

[addsimm]

Is this a bug report or feature request? (choose one)

🐞 Bug report

💻 Version of CKEditor: 11.2.0

📋 Steps to reproduce:

open editor and press space

This is a repeat of Stack Overflow:

https://stackoverflow.com/questions/52370562/ckeditor-5-spaces
which has no definitive answer since 9/17/2018

✅ a space

❎ an A with a caret

does this intermittently depending on the key following the space bar depress

[ghost]

I'm getting this error with new lines. This is happening on 5.11. Currently rolling back to 5.10 for the time being.

Error in console will say Uncaught o: model-nodelist-offset-out-of-bounds: Given offset cannot be found in the node list.

[f1ames]

Does it happen on all browsers with any input language. Or there is a specific configuration (browser + input language) which causes this issue? @addsimm @MrXXIV could you share more details on your environment (OS, browser, input language)?

Do you use standard builds (which one) or custom ones (any custom plugins)? If it is a standard CKEditor build without any modifications, could you provide a sample (e.g. on codepen) where this issue could be reproduced?

[addsimm]

I am only using English

it happens in Chrome and Firefox - latest versions

Our site is served on Digital Ocean using a Django 1.11 back end. I am using a single JS file downloaded directly from your latest release on Github. I also have a CSS file that I modified according to your docs to change the colors and font size. The same bug appears when the CSS file is commented out.

Here is my initiation code:

        DecoupledEditor
            .create(document.querySelector('#editor'),
                {
                    fontFamily: {
                        options: [
                            "Abril Fatface",    // Display
                            "Cookie",           // Cursive
                            "Cousine",          // Fixed
                            "Lato",             // Sans
                            "Tinos"             // Serif
                        ]
                    },
                    fontSize: {
                        options: [
                            10,
                            14,
                            'default',
                            26,
                            30
                        ]
                    },
                    toolbar: ['fontFamily', 'fontSize', '|', '|', 'bold', 'italic', 'underline',
                        'strikethrough', '|', '|', 'highlight:greenPen', 'highlight:redPen',
                        'highlight:yellowMarker', 'removeHighlight', '|', '|', "alignment", '|', '|', 'blockQuote'],
                    alignment: {
                        options: ['left', 'center', 'right']
                    },
                    language: 'en',
                    removePlugins: ['Autoformat']
                }
            )
            .then(editor => {
                const toolbarContainer = document.querySelector('#toolbar-container');
                toolbarContainer.appendChild(editor.ui.view.toolbar.element);
                editor.model.document.on('change:data', function () {
                    countWords();
                });
                window.myeditor = editor;
                countWords();
                console.log('editor: ', Array.from(editor.ui.componentFactory.names())); // .join('\n')
            })
            .catch(error => {
                console.error(error);
            });

This morning I also noticed that when i click bold on an empty space:

[oleq]

@addsimm We don't have an access to your instance so we're walking blindl here. There could be dozens of factor contributing to the issue (e.g. OS, browser, extensions, server, content encoding, editor config, your custom code, editor bug). What we can do is eliminate casues step-by-step.

To start off, can you reproduce the issue without Django? A good ol' plain HTML file loading the simple test data and the editor? (served by apache or even straight from the file system)

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="utf-8">
        <script src="https://cdn.ckeditor.com/ckeditor5/11.2.0/decoupled-document/ckeditor.js"></script>
    </head>
    <body>
        <div id="editor">Content with spaces</div>

        <script>
            DecoupledEditor
                .create( document.querySelector('#editor') );
        </script>
    </body>
</html>

It not, extend the configuration step by step until you add everything you posted in https://github.com/ckeditor/ckeditor5/issues/1431#issuecomment-450399693

.create( document.querySelector('#editor'), {
    // ...
} );

• add fontFamily config,
• add fontSize config,
• ...
• add your custom countWords code (what does it look like anyway)?

If still unable to reproduce, then use Django. Serve the same ultra–simple HTML:

• without any config,
• with your full configuration

Also watch for the headers in your dev tools. Maybe the server serves everything with some weird encoding.

I hope this will give us some insight into your problem.

P.S: please make sure you run vanilla browsers (no extensions, no plugins, fresh installations). We had some collisions in the past.

[tomlagier]

I'm having this same issue. I'm using the InlineEditor hosted here: https://cdn.ckeditor.com/ckeditor5/11.2.0/inline/ckeditor.js

My repro case is:

<!DOCTYPE html>
<html>
  <head>
    <script src="https://cdn.ckeditor.com/ckeditor5/11.2.0/inline/ckeditor.js"></script>
  </head>
  <body>
    <div id="editor"></div>
    <script>
      InlineEditor.create(
        document.getElementById('editor')
      );
    </script>
  </body>
</html>

edit: Running in incognito mode with no extensions, and here are my response headers:

cache-control: max-age=0, must-revalidate
content-length: 1097
content-type: text/html
date: Tue, 01 Jan 2019 07:51:51 GMT
etag: ZvCymHutXA7YkHjmwWBYbg==
server: dart:io with Shelf
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block

edit2: Just tested, 11.1.1 works OK, so this must have been introduced in 11.2

[ma2ciek]

Yep, I can confirm it.
The above scenario can show it.

Adding the <meta charset="utf-8"> to the document's head can be a quick win now for somebody facing this problem.

I've tested and for me, a few older versions didn't work too (like the 11.1.1 mentioned above). Although, the 10.0.0 version of the InlineEditor works without setting the encoding.

I've checked and all of these ckeditor.js files are correctly saved using the utf-8 encoding.

I'm still not sure what has changed so now we have the problem that wasn't present in the 10.0.0 version.

[oleq]

Wait... isn't the character encoding declaration mandatory (cc @ma2ciek)?

If it is not declared by the HTTP Content-Type header and not defined by <meta charset="...">, it will obviously fail. One must use either to make the website work properly according to the spec. Doing otherwise is asking for trouble.

Also see https://stackoverflow.com/questions/14669352/is-the-charset-meta-tag-required-with-html5.

[ma2ciek]

Hm... right @oleq.

I was just a little bit confused that the 10.0.0 version works without setting the proper header / meta tag.

[oleq]

Maybe we handled white spaces in a different way in 10.0.0. But all the same, the website (server + page) configuration provided by the user is invalid (risky). Let's wait for the confirmation from @addsimm that it's the same issue before closing.

[addsimm]

Well I have some good news - while I was working on the bug, I started trying different versions off or your cdn ...

All of them do not display the issue, but my local version does, ie local - problem, cdn no problem.

I don't have time to look into it right now, but I attached it and you can take a look if you are interested.

[tomlagier]

Hmm.. I found this issue because I was having this trouble on 11.2, served locally on an app that included the <meta charset="utf-8" />. I have since switched over to a custom build (which I believe is still using 10.x), but I can't reproduce with the combination of 11.2 on the CDN and meta tag.

There must be a header that's different between the locally served version and the CDN one that was causing the discrepancy. Feels like a bug, just because earlier versions work OK, but if not then maybe it's worth a note in the FAQ or readme that the tag & header are preconditions?

EDIT: Just confirmed that running locally, with the <meta charset="utf-8" /> tag, 11.2 presents the issue, but sourced from CDN it does not.

HTML:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <script src="ckeditor/ckeditor-11.2.js"></script>
  </head>
  <body>
    <div id="editor"></div>
    <script>
      InlineEditor.create(
        document.getElementById('editor')
      );
    </script>
  </body>
</html>

Local headers (showing the issue):

cache-control: max-age=0, must-revalidate
content-length: 526581
content-type: application/javascript; charset=utf-8
date: Thu, 03 Jan 2019 04:10:55 GMT
etag: ajlIOJvirlBLPqvXD8f/+g==
server: dart:io with Shelf
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block

Headers when sourced from CDN (src="https://cdn.ckeditor.com/ckeditor5/11.2.0/inline/ckeditor.js") - issue not present

access-control-allow-origin: *
age: 175635
cache-control: max-age=86400
content-encoding: gzip
content-type: application/javascript
date: Tue, 01 Jan 2019 03:26:18 GMT
expires: Wed, 02 Jan 2019 03:26:18 GMT
last-modified: Mon, 10 Dec 2018 22:38:57 GMT
server: Apache
status: 200
vary: Accept-Encoding
via: 1.1 ac2b2afd66b6aef623f9c570e5db36eb.cloudfront.net (CloudFront)
x-amz-cf-id: YT5KiAHLxZDCKhILX7UdnIF9nCDOJ-VXbhWnBK33nOBryFLxU8YneQ==
x-cache: Hit from cloudfront

The big thing that stands out is that charset=utf-8 that my server is returning for content-type.

[oleq]

I could reproduce the issue with 11.2.0 only when I didn't specify the <meta charset> tag and:

1. I used CDN, like

   <!DOCTYPE html>
   <html>
   <head>
       <script src="https://cdn.ckeditor.com/ckeditor5/11.2.0/inline/ckeditor.js"></script>
   </head>
   <body>
       <div id="editor"><p>foo bar</p></div>
       <script>
       InlineEditor.create( document.getElementById( 'editor' ) );
       </script>
   </body>
   </html>
   

   • HTML response headers

     Accept-Ranges: bytes
     Access-Control-Allow-Origin: *
     Connection: Keep-Alive
     Content-Length: 292
     Content-Type: text/html
     Date: Thu, 03 Jan 2019 15:50:17 GMT
     ETag: "124-57e8fb7d27740"
     Keep-Alive: timeout=5, max=100
     Last-Modified: Thu, 03 Jan 2019 15:50:13 GMT
     Server: Apache/2.4.34 (Unix)
     

   • ckeditor.js response headers

     access-control-allow-origin: *
     age: 2010094
     cache-control: max-age=86400
     content-encoding: gzip
     content-type: application/javascript
     date: Tue, 11 Dec 2018 09:29:25 GMT
     expires: Wed, 12 Dec 2018 09:29:25 GMT
     last-modified: Mon, 10 Dec 2018 22:38:57 GMT
     server: Apache
     status: 200
     vary: Accept-Encoding
     via: 1.1 9de9a776d0da209cb66ec4bd03877799.cloudfront.net (CloudFront)
     x-amz-cf-id: IJzZlysbyLWTgll4LPmRBx_bhqxlNTwsxV6x3nGrXHw5sEuoGGrz5g==
     x-cache: Hit from cloudfront
     

2. I used a local ckeditor.js and my server didn't specify the charset in the content-type

   <!DOCTYPE html>
   <html>
   <head>
       <script src="ckeditor.js"></script>
   </head>
   <body>
       <div id="editor"><p>foo bar</p></div>
       <script>
       InlineEditor.create( document.getElementById( 'editor' ) );
       </script>
   </body>
   </html>
   

   • HTML response headers

     Accept-Ranges: bytes
     Access-Control-Allow-Origin: *
     Connection: Keep-Alive
     Content-Length: 341
     Content-Type: text/html
     Date: Thu, 03 Jan 2019 15:54:51 GMT
     ETag: "155-57e8fc8275fc0"
     Keep-Alive: timeout=5, max=96
     Last-Modified: Thu, 03 Jan 2019 15:54:47 GMT
     Server: Apache/2.4.34 (Unix)
     

   • ckeditor.js response headers

     Accept-Ranges: bytes
     Access-Control-Allow-Origin: *
     Connection: Keep-Alive
     Content-Length: 526545
     Content-Type: application/javascript
     Date: Thu, 03 Jan 2019 15:54:51 GMT
     ETag: "808d1-57c4549f601c0"
     Keep-Alive: timeout=5, max=95
     Last-Modified: Wed, 05 Dec 2018 12:11:59 GMT
     Server: Apache/2.4.34 (Unix)
     

   In this case the bug is gone as soon as I force the server to use charset in the .htaccess file

   <Files "*.js">
   Header set Content-Type: "application/javascript; charset=utf-8"
   </Files>
   

   so the ckeditor.js headers become

   Accept-Ranges: bytes
   Access-Control-Allow-Origin: *
   Connection: Keep-Alive
   Content-Length: 526545
   Content-Type: application/javascript; charset=utf-8
   Date: Thu, 03 Jan 2019 15:56:28 GMT
   ETag: "808d1-57c4549f601c0"
   Keep-Alive: timeout=5, max=95
   Last-Modified: Wed, 05 Dec 2018 12:11:59 GMT
   Server: Apache/2.4.34 (Unix)
   

So it all makes sense to me, however it contradicts what @tomlagier reported:

EDIT: Just confirmed that running locally, with the tag, 11.2 presents the issue, but sourced from CDN it does not.

[oleq]

But yes, I confirm that 10.1.0 from CDN is the last version that actually worked in the first case I checked in my previous comment

1. I used CDN, like

I'm not sure why it is so (that's a really good question) but again, a configuration without <meta charset> or charset in response headers is wrong IMO.

[tomlagier]

I'm not sure why it is so (that's a really good question) but again, a configuration without or charset in response headers is wrong IMO.

Right, but I have a reliable reproduction case with both set - is there some way they might be conflicting?

[Reinmar]

The <meta charset> is absolutely obligatory and it must be present before any other tag like <title>. Without it, the browser will try to detect the charset which may lead to strange behaviours which will depend on the actual text used in the page and how the page is served (the server setting, perhaps the client's locale, etc.).

There may be differences between how often and how easily those Â characters appear in specific CKEditor 5 versions due to changes in how often and in which cases we use the inline filler. I would not rely on this. It also requires knowledge about the filler behaviour to reliably verify whether the encoding is set correct.

So, in short – use the <meta charset="utf-8"> tag. If you don't weird things may happen and it makes little sense to analyse them.

However, one thing that I'd check is whether we cannot serve those files from CDN with some additional charset header. cc @vokiel @wwalc

[Reinmar]

PS. I tested it myself and adding <meta charset="utf-8"> fixes the issue which can be reproduced on https://github.com/ckeditor/ckeditor5/issues/1431#issuecomment-450715022.

[oleq]

@Reinmar That makes sense, just like I wrote. But what worries me is this comment indicating that there could be something else going on.

[Reinmar]

I know. It worries me too. But you verified that adding charset=utf-8 to the headers does actually fix this issue and that makes sense. The opposite behaviour would blow my head. I don't say that I'm 100% sure it doesn't happen (it's software, so weird things can happen). But taken the logic, my experience (I see this issue every now and then for the past 7 years) and your tests, I say the chance is extremely low. Unless we'd be able to reproduce this locally, it's also extremely hard to do anything about it.

[vokiel]

We've extended the Content-Type header with the charset. So now the headers looks like:

Access-Control-Allow-Origin: *
Age: 3797
Cache-Control: max-age=86400
Connection: keep-alive
Content-Encoding: gzip
Content-Type: application/javascript; charset=utf-8
Date: Fri, 04 Jan 2019 10:10:56 GMT
Expires: Sat, 05 Jan 2019 10:10:56 GMT
Last-Modified: Mon, 10 Dec 2018 22:38:57 GMT
Server: Apache
Transfer-Encoding: chunked
Vary: Accept-Encoding
Via: 1.1 df792ea3bbbe656e2f5c7b61aa85cc47.cloudfront.net (CloudFront)
X-Amz-Cf-Id: NOjp3GXilQoH77uSqe7pbu9T5zcKit3vb1JUHjzixZGUDSFfyHXV5A==
X-Cache: Hit from cloudfront