Readthedocs.org: Consider supporting OpenGraph?

[malmaud]

As discussed here, OpenGraph support would allow Discourse forums and other popular systems to display inline previews of links to readthedocs pages, especially useful for forums about programming languages and tools.

It just takes a few <meta> tags injected into the rendered HTML, so I don't think it should be too burdensome. Plus Jeff Atwood himself is pushing for it :)

[agjohnson]

Seems like a great addition. It should be trivial to implement on our theme overrides. Does implementing this as an article type make sense, or is there another type that would work best for something like the DIscourse onebox?

[malmaud]

Yes, article type makes sense.

I think "og:description" will be the trickiest tag to get right - it along with "og:title" contain the text which is actually shown in the onebox. According to this, most sites will display between 200 and 300 characters of the og:description value, so it essentially is a 200-character summary of the doc page.

[agjohnson]

Yeah, I'm not entirely sure how to handle the description.

The change for the meta tags is a simple addition to the template overrides RTD maintains. We do have access to the content of the page in the templates, but it's HTML, which rules out a simple truncation operation inside the templates. If we want to be more intelligent about the content truncation, we'll have to write a template filter in a Sphinx extension to break up the HTML content.

For now, this might just be the project description field in the RTD project admin, instead of per-page content for the description.

og:title should be trivial however, this can be provided by the Sphinx template easily.

[malmaud]

That sounds good. And maybe set og:image to the project logo whose name is somehow exposed to the template.

As a concrete reference, here's what the meta tags for say http://docs.julialang.org/en/release-0.4/manual/variables/ could maybe end up looking like:

<meta property="og:title" content="Variables" />
<meta property="og:type" content="article" />
<meta property="og:url" content="http://docs.julialang.org/en/release-0.4/manual/variables/" />
<meta property="og:site_name" content="readthedocs: Julia" />
<meta property="og:description" content="The Julia programming language.." />
<meta property="og:image" content="http://docs.julialang.org/en/release-0.4/_static/julia-logo.svg"/>

[agjohnson]

If we ever start collecting the project logo, that would be another good addition. Currently we aren't collecting that and don't have a great way to determine that outside of project configuration options.

This reference seems like a reasonable place to start at least

[stsewd]

I was able to do a POC here, injecting all the metatags on the rtd theme https://github.com/rtfd/sphinx_rtd_theme.

I was able to make work: title, site_name, url, type, description, image and locale (this one isn't on the same format of language_TERRITORY because of https://www.sphinx-doc.org/en/stable/usage/configuration.html#confval-language).

But, there is a design decision to take here, we depend on the theme_canonical_url value here, but on the rtd code, we inject the canonical url tag from our extension (https://github.com/rtfd/readthedocs-sphinx-ext) rather then injection those values in the html_theme_options or using the value from html_context (we populate this field on the rtd).

So, should this live on the rtd-theme or on the rtd-sphinx-extension?

[stsewd]

I wasn't able to make it work with ngrok, so, I just upload it on github pages, and for some reason the _static dir doesn't work (I guess because of jekyll). But this is kind of how it looks my POC.

this is how it looks without og, I'll need to figureout how to get a better description p:

[stsewd]

I was able to do the .nojekyll thing, but facebook doesn't support svg images p: so, it kind of looks like the same https://developers.facebook.com/tools/debug/sharing/?q=https%3A%2F%2Fstsewd.github.io%2Frtd-test%2F

[stsewd]

At least on twitter it makes a LOT of o difference :grin: https://cards-dev.twitter.com/validator

[humitos]

So, should this live on the rtd-theme or on the rtd-sphinx-extension?

I think that our sphinx extension will be better since it will affect all the themes that are built on Read the Docs, not only our own.

Also, I think that we may sanitize the description probably to make it looks nice (without \n and as a complete sentence). In the example, the title is repeated in the description.

We may want to use the default Read the Docs logo if we don't have a project's image (html_logo in conf.py I suppose).

[agjohnson]

I'd say no logo if the user hasn't specified. It seems odd to brand with our logo

[agjohnson]

So, another thing to point out is that Sphinx already has support for this natively through reST meta directive. It just outputs the meta elements in <head>, but does require authoring these pieces on each page source. Perhaps we can set default page-level meta through standard Sphinx options or and extension?

Ultimately, I want opengraph enabled across all projects. If we are hard coding the template, we'll probably break or at least override anyone using meta for opengraph already. However, setting default meta is probably the cleanest solution either way.

I haven't looked at how sphinx handles this, but my assumption is that the meta elements will be in the doctree after parsing the source, so we can use an extension to add default meta tags based on RTD context data to each document.

http://docutils.sourceforge.net/docs/ref/rst/directives.html#meta

[stsewd]

@agjohnson so, I was editing the extension, with the doctree and everything, but I hit a bug that makes impossible to do the meta tag default thing https://sourceforge.net/p/docutils/bugs/281/, doctutils doesn't support : in the meta directive, even escaping doesn't work as expected.

So, another option is using the builder I guess? I didn't try but, I guess we have access to the dom

[stsewd]

Sphinx issue https://github.com/sphinx-doc/sphinx/issues/2645

[stsewd]

We inject a template already https://github.com/rtfd/readthedocs-sphinx-ext/blob/8e545547c2c1324368406a710dd76e0534674499/readthedocs_ext/readthedocs.py#L117-L121, but I'll try to look if we can have the dom, or block this till the problem is resolved in sphinx

[stsewd]

Another sphinx related issue https://github.com/sphinx-doc/sphinx/issues/6089

[humitos]

doctutils doesn't support : in the meta directive, even escaping doesn't work as expected.

This is already fixed on docutils, but it's not released yet. It will be on 0.15: http://docutils.sourceforge.net/RELEASE-NOTES.html#release-0-15b-dev

[humitos]

docutils 0.15 was released: https://pypi.org/project/docutils/

This is not blocked anymore :tada:

[Daltz333]

Any update on this?

[stsewd]

@Daltz333 sorry, this isn't is our roadmap this month, but I'll try to see if we can prioritize this soon.

[Daltz333]

No rush. It'd be pretty cool though.

[Daltz333]

I went ahead and published https://pypi.org/project/sphinxext-opengraph/

[stsewd]

Just linking to https://github.com/sphinx-doc/sphinx/pull/7974 it may be implemented in sphinx itself. I'll keep this open to see if we can set some values for the user if that happens.

[Daltz333]

In that PR, I've suggested that it may be better to keep it in an external third-party extension, that way we can link straight into READTHEDOCS various environment variables to easily build URLs, instead of having them be static and per language and version.

[stsewd]

So, we already set some variables based on the project from the user, so supporting that wouldn't be hard if the extension is builtin https://github.com/readthedocs/readthedocs.org/blob/f48fb667778a749089cf8adc215a20c4b40e86d3/readthedocs/doc_builder/templates/doc_builder/conf.py.tmpl#L150-L150

[Daltz333]

Currently out extension (one I linked previously) will grab language and version through the READTHEDOCS environment variable, that way the user just has to set the base "domain" URL. It would be great if we could grab the domain URL from Readthedocs somehow.

[stsewd]

You can get the canonical url from the context variable (that should be available for plugins), that URL always points to the default version.

https://github.com/readthedocs/readthedocs.org/blob/21e8237940803f4b05ed883386374fcf444b99bd/readthedocs/doc_builder/templates/doc_builder/conf.py.tmpl#L94-L94