Core: 301 redirect for new URL (title changes)

[aethior]

When the title is changed a new url (slug) is generated. The old url has to redirect to the new URL using 301 HTTP status code (moved permanently). This is necesary for a better SEO.

Related: https://discuss.flarum.org/d/5069-incorrect-http-status-code-in-redirections

[franzliedke]

As far as I can tell, this problem can be avoided with rel=canonical tags as well - and we already have an issue for that, so I am closing this one.

Feel free to correct me.

[tobyzerner]

You're right, but would it not be worth enforcing a redirect for completeness' sake?

[aethior]

@franzliedke You also need to use 301 redirects. The content was moved to other url, and when the old url is searched by crawlers this url receive a 200 status code. You have to use a 301 code and redirect to the new url to update the search index.

Examples
vBulletin.

Invision Power.

Xenforo.

Discourse.

esoTalk.

[dav-is]

The pretty url portion doesn't matter at all. Easily solved by canocial link without it. For instance forum.example/d/5 will go to the discussion of id 5. Pretty urls aren't needed on search engines as all the context is in the search result. When sharing links with others, a pretty URL makes it a bit more clear on the content you are about to view.

[aethior]

This is not a discussion about pretty urls. When you use a 301 status code, you say to the crawlers that the url changed and they have to update the index. If you change the url without the 301 code, the new url is absolutely new for search engines. The search engines uses the http status code for update (301) or delete (404) the url from search index.

The examples show how the most important forum softwares use this method, because it's the correct method. They also use canonical tags for other uses.

[dav-is]

The point is the URL wouldn't change. If you don't include the title in the URL at all, it says the same. Flarum doesn't parse the title in links. It just goes by the ID so the pretty url portion of it can be omitted for search engines so it never changes.

[aethior]

For Google /d/1-old-url and /d/1-new-url are two different url (for flarum is the same discussion (ID 1), but for crawlers are two different urls). The examples (vBulletin, Invision Power, Xenforo, Discourse, esoTalk, Burning Board...) also creates urls based in the discussion ID and complete the url with the current title. The search engines does not understad portions or ID, crawlers read the complete url.

[dav-is]

The title can be ommited. Set canocial URL to /d/1 and it will work.

[aethior]

I don't want to flood the issue repeating the same argument again and again. You can visit the posted examples and check how they use the canonical tags and redirections (the images are real examples, you can repeat the examples using httpstatus.io).

www.domain.com/d/1-old-or-incorrect-url/ -> HTTP 301 -> www.domain.com/d/1-new-and-correct-url/

In all cases <link rel="canonical" href="http://www.domain.com/1-new-and-correct-url/"> because you want to index www.domain.com/d/1-new-and-correct-url/ (all the url with ID and title).

[dav-is]

I'm not sure you understand what I'm saying.

A user create a discussion title "Test Discussion" and it is assigned an id of 1. The URL shown in the broswer is flarum.example/d/1-test-discussion. The canocial URL is set to flarum.example/d/1. The search engine indexes that URL.

Then the user edits the title of the post. The URL shown in the browser is flarum.example/d/1-new-title although, the canocial URL says the same as flarum.example/d/1.

All traffic from google is sent to flarum.example/d/1 and the title is appended by the javascript after loading the page.

[aethior]

Yes, I understand. The displayed url in Google search results is flarum.example/d/1 (without words) in your example.

This is the easy solution, but not the better (IMHO). The webmasters want to index completely the url (better apearence in search results, keywords for seo,...).

Honestly, if other software solutions use the described method, why not also Flarum?

[tobyzerner]

Yeah, the canonical URL should include the title slug, because we want Google to link to pretty URLs.

If the title slug in the URL is incorrect, Flarum should kindly redirect the user/bot to the correct URL. And let's go back to basics and assume we don't have JavaScript to do it for us. Adding in a HTTP redirect is not a big deal, and it's the correct thing to do, with or without a canonical tag.

[franzliedke]

I still don't see the benefit here, to be honest.

If canonical tags take care of the de-duplication (and they do), why do we still need redirects?

[aethior]

@franzliedke For Google /d/1-old-url and /d/1-new-url are two different url. When the url is changed you use a 301 redirection to alert crawlers about the url change. When the crawler receive a 301 redirect the information is automatically updated in the search index (the update is very fast).

With canonical you are selecting one url between differents urls to be indexed, but with 301 redirect you are updating the old url.

When Google index the url, this url generate page rank and this information should be transfer to the new url. If you dont use 301 redirect the new url is literally new for search engines.

FAQs for all site moves with URL changes
Do I lose credit for links when I redirect to new URLs?
No, 301 or 302 redirects do not cause a loss in PageRank.

https://support.google.com/webmasters/answer/6033049

[franzliedke]

But this is not about moving a site. This is about preventing duplicate content. See one of these:

• https://searchenginewatch.com/sew/how-to/2288690/how-and-when-to-use-301-redirects-vs-canonical
• https://moz.com/blog/301-redirect-or-relcanonical-which-one-should-you-use

[sijad]

from first link you posted:

When to Use 301

• As default – this is the preferred method
• Pages that are being permanently moved or replaced
  ...

when to use Canonical

• When 301s can’t be implemented, or take too much time
• Duplicate content but you want to keep both pages live
  ...

I also think 301 option is better here

[franzliedke]

Why an extra redirect? This is also bad (and still unnecessary) from a performance perspective.

[sijad]

there's no extra redirection, with backend redirection frontend redirection is useless
anyway without 301 redirection browser treats the new URL as a new record in its history and this can make problem for users in a while.

[franzliedke]

@sijad Huh? When the server responds with 301, this causes an additional roundtrip. The client parses the response, and then sends an extra request to the target URL.

301 redirects and canonical URLs both achieve the following goals:

• prevent duplicate content for search engines
• change the URL (so that users can share the correct URL)

Apart from that, 301 is less performant, so I slightly prefer rel=canonical.

That said, implementing rel=canonical is not super trivial, because existing query parameters need to be taken into account (some can be dropped for the canonical URL, some cannot). But that is solvable in a generic way.

More interesting is the question whether to do this on the client or on the server, and what to do on the client when navigating to another page (I read that Google reads those meta tags when navigating a single-page app as well, and it supposedly prefers the values in the DOM over those in the source HTML).

[aethior]

I always understood the 301 redirects as the best way to redirect users and crawlers to the correct url. I understand the canonical tags as a preference among different urls (when different urls are available: sessions, lowercase/uppercase, parameters, etc.). In all case canonicals are recomendations for search engines, and Google reserves the right to ignore them if you use it incorrectly.

301 redirect or canonical?
If you have the choice of doing a 301 redirect or setting a canonical, what should you do? The answer is simple: if there are no technical reasons not to do a redirect, you should always do a redirect. If you cannot redirect because that would break the user experience or be otherwise problematic: set a canonical URL.
https://yoast.com/rel-canonical/#301-redirect-vs-canonical

To be honest, it's curious for me see how the most important software solutions (with long trajectory and good seo optimization) use both methods (301 and canonical). I think it's important to take into account their experience.

[albertalbs]

@aethior is trying to make Valid point here. Whatever he suggests in this discussion is exactly correct.

@dav-is

• When you change the Post Title (Forum Title), the URL slug should automatically change and the OLD URL should do 301 redirect to NEW URL.
• Or also if User want to keep the OLD slug as it is but only the title change, then there will be option to edit the URL slug below the Title field.
• When user click "Publish Post/topic" button this 301 redirect will be executed.

If you think about SEO and ranking in Google, 301 redirect is the best option.

Do not worry about performance issue. Because, Doing one single page URL change will never ever harm the server performance.

Same kind of SEO feature available in all SEO Ready CMS including WordPress and other Forum Software (Discourse).

So it is explained clearly here:

When to Use 301

As default – this is the preferred method
Pages that are being permanently moved or replaced

when to use Canonical

When 301s can’t be implemented, or take too much time
Duplicate content but you want to keep both pages live

Still not convinced please look at the reference links for How WordPress is handing permalinks and URL redirection: (links collection)

• https://codex.wordpress.org/Using_Permalinks
• https://codex.wordpress.org/Function_Reference/sanitize_title#WordPress_Titles
• http://wordpress.stackexchange.com/questions/74415/how-does-wordpress-generate-url-slugs
• https://codex.wordpress.org/Function_Reference/WP_Rewrite
• https://codex.wordpress.org/Rewrite_API
• https://codex.wordpress.org/Function_Reference/wp_update_post
• https://code.tutsplus.com/articles/options-for-301-redirects-in-wordpress--cms-21985

In my experience I've faced lot of clients developing without SEO ready CMS and frond-end websites. They were not ready to listen but experienced the worse from un-seo friendly CMS and then they finally moved to better CMS. Mostly WordPress.
This the reason, WordPress is picked-up in the popular trend along with its ease of customization features.

[meetdilip]

Would have been great if there was some standard to these forum url. Forum makers are a fragmented team. No association where they work on common cause, let alone standards.

[Zeokat]

The preferred solution by most of Search Engines are 301 redirects.

Redirects has many advantages (from SEO point of view):

• Fast reindexing.
• PageRank (aka link juice) transfer.
• Clear webpage URL structure.

If we use canonical tags we lost some of these advantages. The reindexing of new pages will be very slow with canonicals, while 301 are reindexed in seconds _(by Google)_.

If we use canonicals, we always have different slugs pointing to the same contents, which can result into a complex webpage structure. Imagine one unique content that can be accesed through 10 different URLs (a little kaos).

Canonicals are mainly a legacy of the past and they are usefull into a very specific situations. I remember those days when webpages serves two totally different webpage versions depending on the resolution of the device. For example for desktop www.example.com and for mobile devices mobile.example.com.
In this situation, the unique choice is use canonical tags, because boths domains serve same contents but focused for different devices and using redirects will break this system. Nowadays, with responsive designs, this situation is mostly obselete.

I don't know why you want to reinvent the wheel using canonicals while 301 is the recommended and more SEO friendly approach.

[franzliedke]

@Zeokat Thanks for weighing in. However, I would like to see some proof for your claims. Can you provide some links? :-)

I see absolutely no reason why redirects should be treated differently from canonical tags by search engines. Why should they be slower to index, for example?

I also don't understand why you say canonical tags are "legacy". They're not even that old, and were introduced for this specific purpose, as far as I know.

I am more than happy to accept you're right, but I need to see some proof...

[Zeokat]

Ok @franzliedke , i go to try to add some links and examples hoping that everyone understands that canonical URLs are mostly used when two or more URLs of the same content have to coexist. Remember the word coexist because is the key.

We start with one official Google link, a must to understand what we are discussing here: https://webmasters.googleblog.com/2009/02/specify-your-canonical.html

After read that, you can think: Yes! we can use canonicals for the purpose that we are discussing here. And you will be right, but will be like kill flies with a canon because:

• Flarum doesn't needs multiple versions of the same webpage. Don't needs that pages pointing to the same contents coexist. We simply change slugs due title change. We don't add any tracking parameter to the URL or anything usefull that requires keep the old URL online.
• Keeping all URL versions, we will fall into complex URL structure that will make every future webmaster task harder (domain changes, write flarum extensions, etc).

Show a proof to check the indexing speed of 301 redirections vs canonicals is quite hard, because it happens in real time but... you can try this:

• Go to Google Webmasters Console and submit one webpage that redirects to the destination page with a 301 HTTP response. You will see by your own that Google automaticly recognize the 301 response and adds the destination page to the index instantly.

• Then, try to index another webpage from the Google Webmasters Console but this time using canonical. The destination page isn't instantly indexed (at least in my tests).

Well... i think that 301 is a clear solution and keep things simple. When the most famous CMS and forum systems had implemented 301 on title edits, isn't because they ignore canonicals, is because 301 is the right solution here. I can't give more arguments to adopt 301 on title edits, if this isn't enougth i simply surrender.

[aethior]

So if you can do 301 redirects, I would definitely do it way. If you don't have the ability or the option to do 301 redirects that's when rel canonical makes sense.

_Matt Cutts (former head of the webspam team at Google)_.

Source: https://www.youtube.com/watch?v=zW5UL3lzBOA

[franzliedke]

Thanks for the links! I appreciate it.

This also means that using rel=canonical to link from /tags or /all to / (depending on what is configured as home page) is correct, right?

[albertalbs]

I should become SEO developer instead being only as SEO Consultant or Expert ;)

[Zeokat]

@franzliedke good question. Maybe we can start that discussion opening another issue because i'm sure that some parts of Flarum's link structure will need rel=canonical and keep this issue for title edits and 301 redirects.

[franzliedke]

For duplicate home page URLs, see issue #1134.

[buiductuan182]

thanks all . now i know how to 301 redirect
i would 301 redirect quangminhhanoi.com
for new URL quangminhhanoi.com/dieu-hoa-trane-2 .
Sory, my english is bad :(

[franzliedke]

I just pushed a change that adds the canonical tag for the discussions page. Let's see how this works in the wild - we should see positive changes in the Webmaster Console for flarum.org, for example.

Thanks for all your input. Please don't feel ignored. The extra redirect is harder to implement, so I omitted it for now. If it really turns out to be necessary, we can still add it later.