Netlify-cms: Entries with complex extensions do not appear in collection view

[ivanreese]

- Do you want to request a feature or report a bug?

bug hunt 👾

- What is the current behavior?

Entries that have a complex extension — eg: html.md — can be created in the CMS, but do not appear in the collection view.

- If the current behavior is a bug, please provide the steps to reproduce.

Set extension: "html.md" and format: frontmatter. In the CMS, create a new entry in that collection. Entry is committed successfully, with the custom extension. Entry does not appear in the collection view.

- What is the expected behavior?

Entry should appear in the collection view.

- Please mention your CMS, node.js, and operating system version.

Netlify CMS version 0.7.6, deployed to Netlify

[tech4him1]

This is caused by our extension filtering function. When it gets the extensions from the file, it only gets the last extension, assuming everything before that is part of the filename. https://github.com/netlify/netlify-cms/blob/9c7c0aeed22cce1fbe01db48e239da49ba3b42ec/src/backends/github/implementation.js#L56 https://github.com/netlify/netlify-cms/blob/834c5f4942ee9f694a900a834c6f672d31761922/src/lib/pathHelper.js#L83-L91

[ivanreese]

Just to lend a bit of context...

These multi-part extensions are a requirement of Middleman as of v4. Here's the issue where they discuss the motivation for this change: https://github.com/middleman/middleman/issues/1211

[erquhart]

We should support this.

[truckyforme]

Same issue here. I'm deploying a multilingual host with HUGO which requires a complex extension. It'd be great to allow this.

[kornichon2000]

Yes, Hugo multilingual requires complex extensions, and it would be great if Netlify CMS could be fixed to support them.

For example, currently example.fr.md is changed to example-fr.md - Hugo requires that the extension remains .fr.md

[erquhart]

We'll take a PR if anyone is up for digging in.

[kornichon2000]

Hi, I'm afraid my JS skills are fairly limited. However I see two solutions:
a) I think the best solution is to leave the . alone. It's the safest solution as unfortunately it's impossible to determine reliably what is a . for a complex extension or just a . in the middle of the filename.
b) in the function posted by tech4him1, use indexOf instead of lastIndexOf to split the file name into name - extension at the first . occurrence instead of the last one. With the possible regression that people using . in their filenames will confuse the system.

[ivanreese]

Is dots in the filename something that people actually do? I'm not sure that that's a common enough practice to be something worth supporting.

[truckyforme]

I'm not an expert and can be easily wrong in my view.
Probably few people, if any, use dots in filenames. However, that shouldn't be a concern. What Kornichon2000 proposes in his first option is a simple solution to the issue of complex extensions and the potential confusion to users who decide to use dots in their filenames.

[tech4him1]

I think a simple fix would be to simple check if the file ends with the extensions specified:

files.filter(file => file.name.endsWith('.' + extension))

We would want to check it with a period prepended to the extension, so that we wouldn't accidently match part of the file name (i_like_tar.gz should not be matched by tar.gz, but i_like.tar.gz should).

[tech4him1]

@kornichon2000 I agree that your last solution is probably not the best option -- we don't really want to introduce regressions here. How do you see implementing your option a?

[kornichon2000]

@tech4him1 Somewhere in the code there is a function that parses the name string to replace invalid url characters with dashes. However the dot . is a valid url character. I think the solution is to modify the parsing routine so it leaves dots unchanged in the filename. I can't think of a negative side effect.

[erquhart]

@kornichon2000 there were issues with static site generators or deployments or something that made us rule out periods. Plus slugs traditionally have periods stripped, I'd consider that widely expected behavior.

@tech4him1 endsWith is exactly what I was thinking, agreed.

[tech4him1]

@erquhart @talves Looking at the fileExtension code:
https://github.com/netlify/netlify-cms/blob/834c5f4942ee9f694a900a834c6f672d31761922/src/lib/pathHelper.js#L66-L80
Do you think any of that "special" logic needs re-created, or do you think just checking endsWith is safe? I don't want to cause regressions, but most of this code seems irrelevant since we're already filtering out folders.

[erquhart]

@tech4him1 since we're only using this check in one place and not for general extension checking, just using endsWith should be fine.

[tech4him1]

@erquhart Hmm, yeah.

Probably not a bad idea to run a sanitize function on it as well, in case a user tries to do something weird like .md/my-file.md. Unless we want that...?
EDIT: For right now I don't think we need to run a sanitize function on the user-supplied extension, since we're not for writes:
https://github.com/netlify/netlify-cms/blob/cfbf31b130172d0f2c44fcfa0bcc4d96bd84ef5f/src/reducers/collections.js#L52
We're also assuming a dev is supplying this extension, so we for the most part want to allow "special" configuration like that, vs a untrusted user setting.

[erquhart]

Yep, I'm good with that.