Hugo: Relative links to non-page files do not include path prefix

[tohojo]

When using the relative files feature (setting the blackfriday sourceRelativeLinksEval option to true), the generated links do not have the path prefix from baseURL included. This means the links break if the site root is not the webserver root.

As an example, say that there are three files in content:

``` content/pages/page1.md
content/pages/page2.md
content/files/image1.png

and page1.md contains the text

link to page 2

```

and the config has baseurl = "http://mysite.com/myroot/".

Then the link to page2.md will be "/myroot/pages/page2/", while the image src will be "/files/image1.png" instead of the correct "/myroot/files/image1.png".

I figure the SourceRelativeLinkFile function needs to learn about prefixes, but I don't know enough about the Hugo data structures to teach it...

This is with Hugo 0.16.

[bep]

@SvenDowideit

[ks888]

Hi I created PR to fix this issue. Please take a look!

[tohojo]

Tested the pull request and it does indeed seem to fix the issue :)

[SvenDowideit]

mmm, this is intentional to our use case.

sourceRelativeLinksEval is intended to allow you to move the files into different file locations without them breaking.

ie - it should not prepend the path from baseUrl.

This also means that we only use a leading / when we want to break out of the relative linking.

[dmd]

Then I'm very confused. If my blog is at http://example.com/myblog/, and my image asset is in the directory /path/to/blog/static/assets/foo.jpg, what is the correct {{< figure src="what" >}} to use?

Using src=/assets/foo.jpg doesn't work, nor does src=assets/foo.jpg.

[tohojo]

@SvenDowideit This is not about using relative or absolute URLs in the markdown - this is about the links being generated in the HTML.

In the example above, it makes no difference whether I change ![](/files/image1.png) to ![](../files/image1.png) - it is still going to result in a link being generated to /files/image1.png instead of the correct /myroot/files/image1.png. Whether the generated link is relative or absolute (according to the realtiveURLs setting), it is still going to be wrong. And the behaviour is _different_ for HTML pages generated from markdown and for non-HTML assets.

So I would request that this issue be reopened and/or @ks888 's pull request above (which fixes the problem) be merged.

[anthonyfok]

@SvenDowideit I think I can reproduce this somewhat buggy behaviour (when sourceRelativeLinksEval = true) with something like this:

hugo server -t hyde  --baseURL=http://localhost:1313/why/what/

When, say, my content/post/hello.md contains this:

Read more about the [Canadian multilingual keyboard](canadian-multilingual-keyboard.md)

The resulting generated HTML becomes:

<p>Read more about the <a href="../../why/what/post/canadian-multilingual-keyboard/">Canadian multilingual keybard</a></p>

Since that page is on http://localhost:1313/why/what/post/hello/, the web browser would interpret that relative URL as:

• http://localhost:1313/why/what/why/what/post/canadian-multilingual-keyboard/

which of course does not exist (404).

Wait, I think this a related but slightly different problem. I see this behaviour when relativeURLs = true. Once I comment that out, the link works.

But, indeed, as @tohojo reports, when it comes to images, it simply does not work. I tried it with ![](image.png), and that became <img src="/post/image.png" alt="" /></p> in Hugo v0.16.

Please take a second look. Thanks!

[SvenDowideit]

In the way we (the docker documentation) use hugo - the baseUrl should never be used at any time when relativeUrls=true - as its supposed to be irrelevant - you're supposed to be able to move the files to a different location on the web server, and they continue to work.

sourceRelativeLinksEval=true should then only change the way that hugo treats links, by looking for files relative to the source markdown file's location (to match how Github's rendering works.

Your example worries me - as it seems to add the baseUrl in - which is definitely wrong.

on the other hand, the way i understand the original report - /files/image.png should (under the way we use hugo) break out of the relativeUrl - its kinda undefined, but whatever happens, the baseUrl would not be used.

[SvenDowideit]

I'm not really trying to say that what we're doing is exactly how it should be - there are not enough tests, nor enough docs to consider the functionality complete - its just what we're relying on to bring our rather complicated docs together at the moment.

[tohojo]

Think of the process as being in two steps:

Step 1: Resolve the actual paths of the assets in the generated file tree. I.e. resolve content/posts/testpost.md to posts/testpost.html and content/files/testimage.png to files/testimage.png. This step is what is influenced by sourceRelativeLinksEval=true.

Step 2: Produce the actual links in the HTML pages. This is affected by relativeURLs=true, in that the relative links between two files are produced by computing relative paths between the paths generated in step 1.

Now, when in theory it shouldn't matter whether the baseURL is included in the paths generated in step 1, or if it is added afterwards. In the current code base it just happens to be included. For relative URLs, the relative positions of two files will stay the same regardless of prefix.

The issue here is that the behaviour of step 1 is inconsistent between HTML pages generated from markdown, and other file assets included in the source tree. So the HTML page paths do include the baseURL prefix, while the asset paths do not. This means that when the relative paths are computed in step 2, they are going to be wrong, and will break out of the baseurl prefix (or, if not using relative paths, the prefix will just be missing).

The patch by @ks888 fixes step 1 to be consistent between HTML pages and non-HTML assets which subsequently makes both relative and absolute URLs in links be correct.

[SvenDowideit]

no, I'm sorry, that makes no sense to me.

at step 1, the webserver's baseUrl has no relevance what so ever - that baseUrl path isn't on the source filesystem at all.

at step 2, if relativeURLs=true the by definition, all URL's that can be considered internal to the html being generated, would also not contain baseUrl, because they're made to be relative to the page being rendered into HTML.

@ks888 's PR @2238 (again, if i read it right) brings the html output baseUrl path into the source file resolution stage - which is very much not the intent of that setting.

content/pages/page.md
content/pages/page2.md
content/files/image1.png

and page1.md contains

[link to page 2](page2.md)
![](/files/image1.png)

might link, if your output HTML is at the root of your web server, but to be truely sourceRelativeLinksEval=true needs to be:

[link to page 2](page2.md)
![](../files/image1.png)

so that hugo can find the file you're talking about and output

<a href="./page2/index.html"><link to page 2</a>
<image src="../files/image1.png" />

As I said earlier, using absolute paths in your markdown with sourceRelativeLinksEval=true means you're indicating that you don't want the file finding to work - and thus when combined with relativeUrl=true, gives you a way to reference url's that are outside hugo's setup, or are going to stay when you move the previously rendered HTML around.

to sum up with an example -

our creation and use of sourceRelativeLinksEval=true is to replicate GitHub's rendering of in repository markdown files - where the baseUrl changes whenever there is a fork, branch or tag, and thus markdown links that are /absolute are almost always outside the current repo

our desired use of relativeUrl=true is to generate the latest docs to /, and then when we release a new version, to move those html files to /v1.3/, and have everything still work.

[SvenDowideit]

all that said - @anthonyfok's comment above looks to me to be a bug in that code - for the reasons i mentioned above, I don't use a BaseUrl

in that example, the output HTML should be

<p>Read more about the <a href="./canadian-multilingual-keyboard/">Canadian multilingual keybard</a></p>

assuming that both sourceRelativeLinksEval=true and relativeUrl=true and that there is a file canadian-multilingual-keyboard.md beside the hello.md one.

But, indeed, as @tohojo reports, when it comes to images, it simply does not work. I tried it with ![](image.png), and that became <img src="/post/image.png" alt="" /></p> in Hugo v0.16.

that is how it should work (hugo should also ERROR - as it can't find that source file) - and why ![](../files/image.png) _should_ be the right thing to do.

[SvenDowideit]

mm, and then reading

In the example above, it makes no difference whether I change to - it is still going to result in a link being generated to /files/image1.png instead of the correct /myroot/files/image1.png. Whether the generated link is relative or absolute (according to the realtiveURLs setting), it is still going to be wrong. And the behaviour is different for HTML pages generated from markdown and for non-HTML assets.

I think this highlights our missunderstanding well - thank you.

there _is_ a difference between ![](/files/image1.png) and ![](../files/image1.png) when using sourceRelativeLinksEval=true. specifically, the absolute path tells the sourceResolution code that this file can't be found relative to the currently rendered page - and so in my usage (with relativeUrl=true, is not checkable by hugo at all. At that point in the code, we're making a list of source file locations - so mixing in the baseUrl that will be used in the next stage of the process isn't the right thing to do.

if the image file isn't found using [](../files/image1.png) then I think that's a bug (I wonder if its found using [](../../files/image1.png))

[SvenDowideit]

well. sorry about that - I'll make a PR that adds some more tests to cover some of the baseUrl cases - the original report is still a corner-case that we use intentionally, but could be defined to work in 2 ways

either:

1. by prefixing all absolute path links with the baseUrl path in relativeUrl=false, and leaving them alone in relativeUrl=true (at the rendering phase, not the sourceRelativeLinnksEval=true` phase
2. leaving all absolute path links as they are in the source, allowing users to break out of the hugo managed URL scheme (the way I use it today)

IMO, baseUrl should _never_ appear in the links when relativeUrls=true, but maybe that's where _I_ am wrong?

[SvenDowideit]

@anthonyfok what we really need, is a way to turn the kind of example you've made into a programatic test - there are way too many options, and i just don't know what they're supposed to do in combinations I don't use.

[SvenDowideit]

@tohojo OH.

Now, when in theory it shouldn't matter whether the baseURL is included in the paths generated in step 1, or if it is added afterwards. In the current code base it just happens to be included. For relative URLs, the relative positions of two files will stay the same regardless of prefix.

and that may be why I am confused.

I'm still failing to see a good way to write tests for what are now at least 4 combinations of options - writing the tests in code isn't going to make it sufficiently obvious what the results should be.

UPDATE - after poking the code most of the day, it looks like my new code adds this baseUrl path to pages (unintentionally, but possible necessarily?) - and that this is not the case with the original linking - where things are much less magical.

[SvenDowideit]

So I think #2277 fixes this properly for most combinations of canonifyURLs, relativeURLs and sourceRelativeLinksEval and baseurl

but because the baseUrl part is added (er, i think) in a transform at the end, I have no idea how to write a test for it)

[tohojo]

Right, so created a minimal hugo site to exercise some of these options (probably should have done that from the beginning).

The repo is here: https://github.com/tohojo/hugo-links-test

And a rendered version of the test page (without the link targets) is here: https://kau.toke.dk/hugo-links-test.html

The latter includes tables for combinations of relativeLinks and baseURL prefix that works and don't pre and post patch. It seems your patch makes things be consistent between how markdown pages and non-markdown assets are handled, which is good. It means that there is now at least one case (relativeURLs=true) that works with a prefix set. However, having things work with a prefix and relativeURLs=false might be good too? :)

[SvenDowideit]

excellent - thank you - we need to make that kind of test an automated part of the hugo test suite

annoying that its not working with relativeURLs=false - it worked for the tests i ran locally - but I got pretty fuzzy after poking different bits of the hugo code base for the last 6 hours.

(aka, yes, I'll look further later)

do you have a script that generates the report?

[anthonyfok]

@SvenDowideit, many thanks for looking into this in depth, and sincere thanks for your solution at #2277.

@spf13, thank you for assigning me some days ago, but I was merely reporting a related bug and not ready to actually solve it. Furthermore, I see the bug is in really good hands now, so I took the liberty of removing my assignment from this issue.

[bep]

This issue has been automatically marked as stale because it has not been commented on for at least four months.

The resources of the Hugo team are limited, and so we are asking for your help.

If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, and you feel that it is still valuable, please open a proposal at https://discuss.gohugo.io/.

This issue will automatically be closed in four months if no further activity occurs. Thank you for all your contributions.

[dmd]

The above reproduction still applies.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.