Ckeditor5: How to getData to retrieve data with JSON format?

[YingshanDeng]

Hello! I found that we can getData with markdown format. And how to getData with JSON format? Is there a plugin for that now? Thank u😋

[Reinmar]

Hey! There's no such plugin yet because we believe that HTML makes the most sense as the default format. However, the architecture supports outputting whatever you want.

With JSON there's a question of how this format should really look. Rich text can be proposed in multiple ways and it would require a proper analysis to figure out just the right format.

Anyway, we can try to showcase how to output JSON from CKEditor 5, but this will be just a proof-of-concept so the format itself will simply match our internal model pretty closely.

[scofalik]

There are a few ways you can tackle this problem. I will describe three most obvious ones:

1. Get the JSON straight from the model (not recommended).
2. Create an appropriate data processor.
3. Find HTML <-> JSON converter library and convert editor's output.

If you don't want to read about editor architecture you may skip straight to code samples. Still, I recommend reading all of this, as it might be insightful and will let you know about traps you might fall into when developing your own plugins based on CKE5 framework.

JSON-ize the editor's model

_This solution is not recommended but it might be insightful why it is not recommended. You may skip this section if you are interested in a solution you might actually use._

The first idea is that you could provide your own editor class, based on one of CKEditor 5's editor classes, like ClassicEditor. You can find more editor classes running a search on our docs website and pick the one that suits your case.

In this solution, you would have to overwrite editor's getData() and setData() methods. This would let you work straight on the editor's model - an abstract data structure. So, in your database, you could keep exactly what is saved in editor's memory.

The getData() method would have to take the main root (or iterate over all roots) and JSON-ize them JSON.parse( JSON.stringify( root ) ). Model document is available at document property.

For setData() you would have to use Element#fromJSON and Text#fromJSON methods. You can differentiate between elements and texts basing on name property (texts don't have it). Then, use DataController#deleteContent and DataController#insertContent to re-set the model. DataController is available at data property of the editor.

I would not recommend this solution for three reasons:

1. As you already have seen, you are using a lot of low-level stuff.
2. If the model, created by a feature, changes, your saved data becomes invalid. For example - at the moment, we represent headings in the model as an element with name heading1 ... heading6. But we thought about changing this to have one element name: heading and then attribute level with a proper value (1 to 6). If we ever make that change, your data is lost. If you change a feature you wrote - the data is lost. If you will use third party plugins and they change something - your data is lost. So it is better to save, for example, HTML or JSON representation of DOM, because a feature should always be able to understand those and convert them to a correct model structure.
3. Not all the data has to be kept in the model tree. For example, markers are kept in a separate collection. You'd have to save and restore them too. Some features may keep data in their own data structures, maybe private ones.

Create data processor

Data processor is a part of the editor that is responsible for converting data from/to given output format. This happens when you load data to the editor (input string is converted to the editor's model data) and when you get data from the editor (model data is converted to the output string).

The default processor for CKEditor 5 is, of course, HTML processor.

Although it is not a requirement, your processor could use DOM as an intermediate step in the conversion. First, you can convert editor data to DOM and then from DOM to the desired format. Similarly the other way, when loading data. This approach lets you use already existing libraries that work with DOM. Using engine you can generate DOM from our internal view structure (which is, by the way, very similar to DOM).

                 {                       data processor                      }
custom model <-> view (DOM-like structure) <-> DOM <-> DOM-to-format-converter

We don't make a requirement to have an intermediate DOM step for a few reasons:

1. We don't want to force you to have a DOM support in the environment where you run the editor, for example, if you run it in node.js.
2. If you have a need for a custom data processing, you can do it straight from the view structure, which may be easier for you (if you already know CKE5 API, it might be easier for you to deal with it instead of DOM API).
3. Sometimes there are no libraries that convert DOM to your desired output, so, again, you might want to work on the view.

Writing JSON data processor

So, let's write simple JSON data processor. Your data processor should implement DataProcessor interface

As you can see, there are two required methods: toData and toView. By default, the editor reads the data string that is in replaced HTML object. So to simplify the example, let's assume that toData will output a string with JSON data, and toView will load a string with JSON data.

Unfortunately, I haven't found a dom<->json library that worked for me. I've found domjson package on npm, but after installing it, it didn't work for some reason. That's why I'll propose working on directly on the view rather than convert it to DOM.

If you would like to see how DOM can be used in a data processor, take a look at HtmlDataProcessor code, that is available in ckeditor5-engine repo.

Let's implement the frame for JsonDataProcessor:

class JsonDataProcessor {
    toData( viewFragment ) {
        const json = [];

        // Generate JSON.

        return JSON.stringify( json );
    }

    toView( jsonString ) {
        const jsonData = JSON.parse( jsonString );
        const viewFragment = new ViewDocumentFragment();

        // Generate view fragment.

        return viewFragment;
    }
}

Then, you need functions that will actually convert view<->JSON.

function viewToJson( viewElement ) {
    const json = {};

    if ( viewElement.is( 'text' ) ) {
        json.text = viewElement.data;
    } else {
        json.name = viewElement.name;
        json.attributes = {};

        for ( const [ key, value ] of viewElement.getAttributes() ) {
            json.attributes[ key ] = value;
        }

        json.children = [];

        for ( const child of viewElement.getChildren() ) {
            json.children.push( viewToJson( child ) );
        }
    }

    return json;
}

function jsonToView( jsonObject ) {
    if ( jsonObject.text ) {
        return new ViewText( jsonObject.text );
    } else {
        const viewElement = new ViewElement( jsonObject.name, jsonObject.attributes );

        for ( const childJson of jsonObject.children ) {
            const viewChild = jsonToView( childJson );

            viewElement.appendChildren( viewChild );
        }

        return viewElement;
    }
}

And use them in the data processor:

toData( viewFragment ) {
    const json = [];

    for ( const child of viewFragment ) {
        const childJson = viewToJson( child );

        json.push( childJson );
    }

    return JSON.stringify( json );
}

```javascript
toView( jsonString ) {
const jsonData = JSON.parse( jsonString );
const viewFragment = new ViewDocumentFragment();

for ( const childJson of jsonData ) {
    const child = jsonToView( childJson );

    viewFragment.appendChildren( child );
}

return viewFragment;

}

Finally, you have to create your own editor class and use `JsonDataProcessor` in it:

```javascript
export default class JsonClassicEditor extends ClassicEditor {
    constructor( element, config ) {
        super( element, config );

        this.data.processor = new JsonDataProcessor();
    }
}

HTML <-> JSON conversion

If you would find a library that converts HTML from/to JSON you could _maybe_ create an editor class the other way (mind you, I haven't tested it):

class JsonClassicEditor extends ClassicEditor {
    getData() {
        const html = super.getData();

        return htmlJsonConverter.toJson( html );
    }

    setData( jsonData ) {
        const html = htmlJsonConverter.toHtml( jsonData );

        super.setData( html );
    }
}

Of course, htmlJsonConverter is a library that I made up for the example purpose. You have to find one on your own and check its API.

Final solution

I've pushed a branch json-editor to the ckeditor5-engine repo with a manual test with a working solution. The code is here: https://github.com/ckeditor/ckeditor5-engine/commit/91213902a517f4bfe51adf6a6cbb3abb70f058ba. If you want to run it, checkout ckeditor5 repo and follow readme to install CKEditor 5. Then go to packages/ckeditor5-engine and checkout to json-editor branch. Then run manual tests (described in readme).

However, I have to note, that the JSON you get is very similar to HTML. In this case, there is a question, whether you gain anything from saving <ul><li>foo</li></ul> as [ { 'name': 'ul', children: [ { 'name': 'li', children: [ { 'text': 'foo' } ] } ] } ]?

If you were interested in saving the editor's model, then as discussed, there are not enough high-level tools for this and it might not even be possible if some features store some data internally. And it is risky.

[Reinmar]

Apart from getting the view structure as a JSON, we're also asked about retrieving the model structure as a JSON. The goal would be to output/input such content from the editor:

[
    {
        type: 'element',
        name: 'paragraph',
        children: [
            {
                type: 'text',
                data: 'Foo'
            },
            {
                type: 'text',
                data: 'Bar',
                attributes: { bold: true }
            }
        ]
    }
]

This is impossible to do inside a data processor (which operates between the output data format and the view structures) and instead it needs to be done by directly operating on the model. The code can land e.g. in DataController.

@jodator, could you create a POC of that?

[jodator]

So the very basic POC code is ready on the poc/json-data branch in ckeditor5-core:

https://github.com/ckeditor/ckeditor5-core/blob/d0909ce000af16c00120ba5582cc226f3abd11f2/tests/manual/json.js#L22-L42

It basically overrides the DataController#stringify() and DataController#init() methods to have best integration with editor API: editor.getData() and Editor.create().