Docusaurus: Fix trailing slash issues (depend on hosting provider)

[slorber]

💥 Proposal

Fix trailing host slash issues: write /myDoc.html instead of /myDoc/index.html

We have a few issues that are related to docs trailing slashes: #2654, #2394, #2134, #3351, #3380

I suspect these issues are related to the Docusaurus output, as for a doc at /docs/myDoc we write /myDoc/index.html instead of /myDoc.html

Always creating a subfolder is probably a signal for the hosting providers (Netlify, Github Pages...) to add a trailing slash automatically to URLs, which can be annoying.

It can break relative links as now the link resolution is different due to the trailing slash, and we end up with broken links in production, not catched at build time (see https://github.com/facebook/docusaurus/issues/3380)

This behavior is provided by a dependency: https://github.com/markdalgleish/static-site-generator-webpack-plugin/blob/master/index.js#L165

We already run on a fork of this dependency. I guess we could as well include a way to write /myDoc.html directly if a route path does not end with a trailing slash.

As this is a risky change, I suggest making this optional. We can make it the default behavior (breaking change) later if it works fine for early adopters.

[slorber]

Seems related to https://github.com/markdalgleish/static-site-generator-webpack-plugin/pull/53

[lunelson]

Always creating a subfolder is probably a signal for the hosting providers (Netlify, Github Pages...) to add a trailing slash automatically to URLs, which can be annoying.

@slorber fwiw this is fixable on Netlify by changing your post-processing settings. They have a UI bug, where the master checkbox to disable post-processing __does not disable the pretty URL setting__, and that is what causes e.g. https://v2.docusaurus.io/docs/2.0.0-alpha.63/installation to 301 redirect to https://v2.docusaurus.io/docs/2.0.0-alpha.63/installation/. You have to just turn them all off individually.

[slorber]

Thanks @lunelson , didn't know about this bug, do you have a link to track this?

Also, I tested these options on my own website, but still have a trailing slash issue 😅 https://sebastienlorber.com/records-and-tuples-for-react

[lunelson]

@slorber If you changed the options and tested your site right away, it might not have worked: the changes seem to take a few minutes to propagate. For me the URL above now works without redirecting to a trailing slash.

And no I don't know of an issue that tracks it at Netlify but I've seen it come up in discussions here and on Gatsby about trailing slash problems.

[lunelson]

@slorber Having pointed that out however, I must say that in my company's websites (including 3 Docusaurus v2 sites and potentially soon more), we will probably turn this Netlify feature back on.

This is because we have had the strong recommendation from an SEO agency that—while neither format (trailing-slash or not) is "better"—there should be 301 redirects to make sure that only one of the two formats is getting crawled and indexed. And because Netlify treats these URL formats identically, and its redirects system therefore doesn't allow you to 301 from one to the other (it would create an infinite loop), our only alternative is to use this pretty URLs feature (which does create a 301) and to accept the decision which is made for us, namely that trailing-slash will be the final format.

This creates a challenge for me right now because currently Docusaurus generates all URL patterns in router/anchor links, sitemap and canonical link tags without trailing slashes 😬

[mpsq]

The sitemap can be generated with trailing slashes, that's what the option trailingSlash is for: https://v2.docusaurus.io/docs/using-plugins#docusaurusplugin-sitemap.

[lunelson]

@mpsq well, that's one down at least 😅

[mpsq]

Yes :) On your list, "URL patterns in router/anchor links" and "canonical link tags" are actually the same thing, they depend on how the permalink prop is generated. All you need is to enable/allow that prop to have a trailing slash.

EDIT:
Maybe this could be a more general setting, rather than a granular one specific to the sitemap plugin.

[lunelson]

Maybe this could be a more general setting, rather than a granular one specific to the sitemap plugin.

YES. This would be ideal.

@mpsq is there any chance you could give me an idea of where I need to intervene in my custom theme, to modify that permalink property in a way that will affect the router, and generated anchor and link tags all together?

[mpsq]

You could do it in your theme by tweaking DocItem / Layout and replace occurrences of permalink with permalink + "/". This is super hacky though, a configuration property in core Docusaurus would be much nicer.

[lunelson]

a configuration property in core Docusaurus would be much nicer

Again: yes

[slorber]

Thanks, didn't notice but it seems my site does not have trailing slashes anymore :)

But anyway we should figure out a portable solution that works even on GithubPages where you don't have many hosting options to tweak... still believe that my initial proposal might be a better solution, need to find time to test this on multiple hosts.

For workarounds to override the permalink, you may be interested by this issue: https://github.com/facebook/docusaurus/issues/3501#issuecomment-702263920

Not sure but something like that could work to customize the permalink:

import React from 'react';
import OriginalLayout from '@theme-original/Layout';
import Head from '@docusaurus/Head';
import {useLocation} from '@docusaurus/router';

export default function Layout(props) {
  const location = useLocation();
  return (
    <>
      <OriginalLayout {...props} />
      <Head>
        <meta
          property="canonical_url"
          content={location.pathname + "/"}
        />
      </Head>

    </>
  );
}

[lunelson]

@slorber thanks, I'll take a look at that too. One thing about the router and this URL format issue that I haven't tested yet, is whether active link class matching will still work if the router paths are also forced to have trailing-slashes...

[slorber]

Notes:

• v1 generates 2 files: /docs/myDoc.html + /docs/myDoc/index.html
• v2 generates 1 file: /docs/myDoc/index.html
• without /docs/myDoc.html file, Github Pages server redirect from /docs/myDoc to /docs/myDoc/

We should probably add 2 new options:

• 1 option to generate the "missing" /docs/myDoc.html file, because some hosting solutions (emm GH pages) can't handle it properly.
• 1 option to add trailing slashes to all site routes, so that it's safe to host your site on all hosts

We can't add a trailing / by default, not everybody is willing to have / at the end of the URLs (legacy/SEO reasons)

Another idea is to resolve all relative links at build time to absolute links, so that the presence of a trailing slash or not does not impact in any way the link target.

On Netlify, disabling the Pretty Url option prevent Netlify from adding the trailing slash, yet if user visits the page with a trailing slash, it is not removed client-side, and still potentially breaks relative links.

[slorber]

Until I figure how to integrate this properly in Docusaurus, here's a PR on the ReactNative website with some ideas to fix trailing slashes issues in userland: https://github.com/facebook/react-native-website/pull/2297

[lunelson]

I've been solving this issue mainly by conforming URLs to the desired format using a function within the Link component—since all internal links have to pass through here at some point; one complexity with this is that if you are using the baseUrl option you have to consider root-based URLs that are outside of the your sub-directory to be external, i.e. static anchors not router-links...

[ukutaht]

Having an option to generate /doc.html instead of /doc/index.html would be ideal for us