Ckeditor5: Add support for oEmbed / embedding media (YouTube, Twitter, Vimeo, Instagram etc.)

[wwalc]

We should work on making it easy to insert the most popular media into the editor (e.g. YouTube movies). At the beginning let's support some minimal set of providers and enhance it with time, based on the community feedback.

If you are interested in this feature, add 👍 .

[jonathanlal]

Yes! This is the only advantage ck4 has over ck5 imo. Another thing worth looking into maybe is twitter embeds, it's used quite often on some news websites.

[winzig]

Should this feature be part of a more generic feature for link "unfurling" in general? Something akin to what embed.ly can do, or perhaps a plugin that works with embedly's services? https://embed.ly

Edit: I see now the mention of oEmbed in the title, which does make this more generic as I was hoping. 👍

[Reinmar]

We've run into many problems with oEmbed in the past. CKEditor 4 uses Iframely in its Media Embed feature (check out the demo). In general, few content providers expose native oEmbed services, so you need to work around this by using proxies like embed.ly or Iframe.ly. IDK how the situation looks now, but e.g. for a tweet, Iframely returns:

<blockquote align="center" class="twitter-tweet">
<p>It&rsquo;s me&hellip; landing on a comet &amp; feeling good! MT <a href="https://twitter.com/ESA_Rosetta">@ESA_Rosetta</a>: I see you too! <a href="https://twitter.com/hashtag/CometLanding?src=hash">#CometLanding</a> <a href="http://t.co/DjU0J1Ey4H">pic.twitter.com/DjU0J1Ey4H</a></p>
&mdash; Philae Lander (@Philae2014) <a href="https://twitter.com/Philae2014/status/532547743206875136">November 12, 2014</a></blockquote>
<script async charset="utf-8" src="//platform.twitter.com/widgets.js"></script>

So... you get a <script> tag. IDK, maybe this works fine when used outside the editor, but it's not something that we'd like in the editor and in the data. Especially that embedding 10 tweets adds 10 script tags. IIRC, that wasn't the only problem back then. Therefore, I'd rather see an oEmbed service customised for content editing.

Another aspect is that using an external service makes the implementation a lot more complex due to the asynchronous nature. Plus, if you want to save in your database only such a tag:

<oembed data-url="https://twitter.com/Philae2014/status/610047412036595712" />

instead of the full HTML:

<div data-oembed-url="https://twitter.com/Philae2014/status/532547743206875136">
<blockquote align="center" class="twitter-tweet">
<p>It&rsquo;s me&hellip; landing on a comet &amp; feeling good! MT <a href="https://twitter.com/ESA_Rosetta">@ESA_Rosetta</a>: I see you too! <a href="https://twitter.com/hashtag/CometLanding?src=hash">#CometLanding</a> <a href="http://t.co/DjU0J1Ey4H">pic.twitter.com/DjU0J1Ey4H</a></p>
&mdash; Philae Lander (@Philae2014) <a href="https://twitter.com/Philae2014/status/532547743206875136">November 12, 2014</a></blockquote>
<script async charset="utf-8" src="//platform.twitter.com/widgets.js"></script></div>

then, when you load such a content into the editor, you need to query the oEmbed provider for HTML responses for all the media used in that content. CKEditor 4 does it one by one (cause that was simpler) so if you have 10 media widgets embedded in your content, you get 10 requests to Iframely which isn't good for initial performance and UX.

So, to sum up, implementing a quality and performant oEmbed support is tricky.

Therefore, I'm considering starting now with something really simple. No external oEmbed provider support, no asynchronousity, just a list of URL => HTML converters. The pros: it's simple so we can do it quickly. The cons: we'll be able to handle only services which use iframes (e.g. YT, Vimeo) cause we won't be able to request external service for the whole content. This rules out Twitter unfortunately ;/ Well, unless you'd use https://twitframe.com/ or a similar service.

What I mean is that we'd have something like:

{
    url: /^https://www.youtube.com/watch?v=(\w+)/,
    html: url => `<iframe width="560" height="315" src="https://www.youtube.com/embed/${ videoId }" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>`
},
{
    url: /^https://vimeo.com/channels/staffpicks/(\d+)/,
    html: videoId => `<iframe src="https://player.vimeo.com/video/${ videoId }" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>`
},
{
    url: /^(https://twitter.com/Philae2014/status/\d+)/,
    html: tweetId => `<iframe border=0 frameborder=0 height=250 width=550 
 src="https://twitframe.com/show?url=${ tweetId }"></iframe>
}

Then, we could add real oEmbed support in step 2. It may still happen in the first version of this plugin, but it'll not delay the whole thing if it turns out that we have problems with choosing a quality oEmbed provider or some UX/performance issues.

WDYT? cc @oleq @wwalc @fredck

[Reinmar]

BTW, we were also thinking about the HTML output of this feature. I think that by default it should be something like:

<figure class="media">
  <div data-oembed-url="[[URL]]">
    [[EMBEDDED HTML]]
  </div>
  <figcaption>Support for captions in the future (after MVP)</figcaption>
</figure>

So, it looks very much like the image's plugin output.

Then, in a separate plugin or a configuration option we could allow make it possible to skip the [[EMBEDDED HTML]] part:

<figure class="media">
  <div data-oembed-url="[[URL]]"></div>
  <figcaption>Support for captions in the future (after MVP)</figcaption>
</figure>

Or even:

<figure class="media">
  <oembed data-url="[[URL]]" />
  <figcaption>Support for captions in the future (after MVP)</figcaption>
</figure>

[winzig]

The comments so far make sense to me. How were you guys expecting to "trigger" the embed functionality? For our use case, I was hoping when a URL is detected in the text (e.g. after finishing typing a URL and hitting space/enter, or after pasting in something that contains a URL), the embed functionality would immediately replace the URL with the embed equivalent. And maybe with a configuration option, provide a mechanism for the user to undo the embed if they just want the URL left alone (or preferably, left as a linked URL).

[winzig]

Also, were you thinking the replacement HTML embed would be easily configurable by the developer?

[alexeckermann]

Great insight into the previous experience with CK4. I wouldn't mind the CK maintained plugin be simple and focussed -- limited known providers that provide simple embed options. If theres a need for any of these other services a community-maintained plugin using the other embed tools can be put together easily enough.

[Reinmar]

So, the first implementation is in https://github.com/ckeditor/ckeditor5-media-embed on master. It can handle the media for which we can automatically create HTML previews (like YT, Vimeo, Dailymotion) but it also handles media for which we cannot generate a preview (yet).

Media with previews:

Media without previews:

A documentation should appear on https://ckeditor5.github.io/docs/nightly/ckeditor5/latest/features/index.html in a couple of hours (if CI will stay green). It explains things like how to configure this feature to produce only semantic <oembed> tags (for all kind of media) and how to use Iframely and Embedly to "expand" these tags on the target websites.

I think the next step for this feature will be adding automatic embedding of pasted links. However, there's a whole list of topics on which we'll like to hear a feedback: https://github.com/ckeditor/ckeditor5-media-embed/issues.

[MaffooBristol]

Is there any way to make it work with straight .mp4 files? It just shouts and says it's not a valid URL but it should be easy enough to get to work, surely?

[Reinmar]

Hi @MaffooBristol! Please open a new ticket for the feature that you're describing. You may be right that it's something that can be achieved quite easily with the current media embed feature.

[kfirba]

@Reinmar I find it extremely difficult to define dataDowncast to custom HTML.

I want that whenever I call the getData() method that my media elements will simply return a paragraph containing an iframe with a src attribute:

<!-- This -->
<media url="https://www.youtube.com"></media>

<!-- Converts to: -->
<p><iframe src="https://www.youtube.com"></iframe></p>

Unfortunately, I just can't get it working. After digging around for over 3 hours I just can't get it working. I always end up with some "mysterious" errors. I just can't fully understand the new model. The docs aren't good at explaining those too. The data is too scattered around and it is very difficult to gather it.

I tried something like:

editor.conversion.for('dataDowncast').elementToElement({
    model: 'media',
    view: (modelElement, viewWriter) => {
      const url = modelElement.getAttribute('url');

      return viewWriter.createUIElement('p', null, function(domDocument) {
        const domElement = this.toDomElement(domDocument);
        domElement.innerHTML = `<iframe src="${url}"></iframe>`;

        return domElement;
      });
    },
    priority: 'highest',
  });

Uncaught CKEditorError: view-writer-cannot-break-ui-element

I'm positive I'm just missing something. I'm sure my understanding of the new model is lacking, hence making every needed change an agony to implement.

Also, If possible, I would love to know how would I write an upcast conversion for the desired HTML output I showed above? How would I extract the URL from the iframe and pass it to the media element?

Edit:

Seems like I was able to create the upcast conversion by simply copying the current upcast conversion for the oembed element. The code:

editor.conversion.for('upcast').elementToElement({
    view: {
      name: 'iframe',
      attributes: {
        src: true,
      },
    },
    model: (viewMedia, modelWriter) => {
      const url = viewMedia.getAttribute('src');

      if (mediaRegistry.hasMedia(url)) {
        return modelWriter.createElement('media', {url});
      }
    },
  });

Edit 2:

Made a slight progress... By using the following conversion:

editor.conversion.for('dataDowncast').elementToElement({
    model: 'media',
    view: (modelElement, writer) => {
      const url = modelElement.getAttribute('url');

      const emptyElement = writer.createEmptyElement('iframe', {src: url});
      const paragraph = writer.createContainerElement('p');

      writer.insert(writer.createPositionAt(paragraph, 0), emptyElement);

      return paragraph;
    },
  });

I was able to get the result:

<p><oembed src="..."></oembed></p>

NO idea why the iframe is being changed to oembed... I even tried replacing the iframe with an img tag, but still, it was replaced with an oembed.