The-seo-framework: More sitemap types

[pedro-mendonca]

Hi,

I would like to suggest adding an optional checkbox to include an archive/category do sitemap.xml.
The user could also choose some categories to include/exclude.

Regards
Pedro Mendonça

[sybrew]

Hi Pedro,

The sitemap's going to be expanded through Extensions.

Free Extension: Categories, Tags, Images, News Sitemap, etc.
~Premium Extension: News sitemap, Image sitemap, etc.~

For now, the basic sitemap included in The SEO Framework is sufficient, or even perfect, for Search Engines.

Adding more onto that gives clutter and increases maintenance time and costs. I also do not want users to be lead into bad SEO practices ("It's in my sitemap, so it's OK...").

The sitemap, in this day and age, mustn't be considered the main factor for getting your pages found.
Your overal website navigation (i.e. theme, post meta and menus) should indicate such existence.

The sitemap should only be used to notify Search Engines for what really matters: Post, public CPT and Page updates.

I hope this clears things up. I do want to expand the sitemap, but only justified for users that know why and how.

Cheers!

[pedro-mendonca]

Thank you for your feedback :+1:

[sybrew]

From ngocdang83's support topic:

Add ability to set sitemap locations.

[jkirker]

Yes, a filter to change the sitemap path would be great - or a settings option. It would make transitioning from Yoast easier as their sitemap url is different. Hate to have to go in and remodify all the various webmaster tools.

**It would also be awesome in the page/post/CPT SEO admin settings to have a checkbox to exclude a page from the sitemap rather than managing the list externally in functions. Just a thought.

[sybrew]

@jkirker You can remove pages with the in-post noindex option. Not having a page indexed is a very good reason to remove it from the sitemap as well and vice versa.

[bigstamp]

Was there any update with a news sitemap? as its crucial, if your listed in Google News

[sybrew]

Google explicitly states it doesn't use the News Sitemap as the only signal.

Google News doesn't favor sites that use News sitemaps in ranking results. Whether there's a News sitemap available or not, Google News uses normal crawl methods to search and index all news sites (homepage and sections).

In the wild, I've also seen sites pop up in the news-section just fine using the regular sitemap and with help of the Articles extension.

With this, I feel no obligation rushing this feature, and I am instead focussing on features that do have an impact: _No ads. No distractions. No nonsense._

[bigstamp]

Yea I get your point, while it may not impact sites once they are indexed. But it does have an impact in the news showing up. With a site map my news articles are listed in google news within a few minute, without it’s a few hours. Dosnt seem much difference but for breaking news the difference is massive.

[sybrew]

Yea, that was in the back of my head too. This is the first time I've seen confirmation, however. I'll dig into it. Thanks!

[sybrew]

Because of the many misconceptions of the sitemap, and as some search engines (Bing, Yandex, Baidu) don't move fast enough, and because some features do give slight benefit in some circumstances, I'm going to transform how the sitemap works and is set up.

I'm going to move TSF to PHP 5.5 very soon. With the move of TSFEM from WordPress.org, I could finally gather useful information via the update server: 99% of the Extension Manager users are at PHP 5.6 or later, of which 60% are at PHP 7.1 or later. 1% is at PHP 5.5.

WordPress will enforce PHP 5.6 from April 2019, too.

I think my users are well aware of the technology they're using, and you are not here to be fooled around. I respect you for this.

The current sitemap's technology is ancient. I built it when I was still learning PHP. I added it in version 2.2.9, back in 2015.

---

Upcoming changes

The sitemap is going to:

1. Get a field*, wherein all "hooks" are prepared; so we can easily expand it.
2. Get an index.
3. Be split up by post type.
4. Get a PHP 5.5-style generator and iterator. This will take the strain off your server.
5. Be optionally listing categories and tags, at the bottom of each related type's sitemap.
6. Need a PR sent to WP Rocket that adheres to these adjustments.

*The field will likely consist of an abstract factory pattern, and something the "WordPress way," with filters and hooks. For example, this list will be filterable, whereafter the factory runs:

[
    'page' => [
        'xml' => {$built-in non-hierarchical factory}, // class name: Attained from get_supported_post_types() loop.
    ],
    'post => [
        'xml' => {$built-in hierarchical factory}, // Attained from get_supported_post_types() loop.
    ]
    'some_cpt' => [
        'xml' => {$built-in hierarchical factory}, // Example, attained from get_supported_post_types() loop.
    ],
    'news' => [
        'xml'      => {$non-default factory},
        'xsl'      => {$non-default factory},
        'supports' => [ // maybe define this at factory level.
            'taxonomy' => false,
            'image'    => true,
        ]
    ],
    'video' => [
        'xml'      => {$non-default factory},
        'xsl'      => {$non-default factory},
        'supports' => [ // maybe define this at factory level.
            'taxonomy' => true,
        ]
    ],
]

ETA: v3.4 4.0.0.

See this comment for implementation: https://github.com/sybrew/the-seo-framework/issues/430#issuecomment-503608385
Also this repo for a working example: https://github.com/sybrew/tsf-term-sitemap

[sybrew]

From Kacper's suggestion:

Add pagination to the sitemap, imposing entry limits; so to reduce server load when the sitemap's building.

I've added this here as it should be considered in building the index map.

Kacper suggested 5000 posts per page. We'll test this, and we might want to transform the current post limit setting to this.

[sybrew]

My comment here explains how to integrate more sitemaps.

Via regular expressions (see first attached commit below), we can now integrate sitemap pagination.

[sybrew]

Here's a proof of concept sitemap generator for terms: https://github.com/sybrew/tsf-term-sitemap
Requires TSF v4.0.0-rc1.5 or later.

[sybrew]

In https://github.com/sybrew/The-SEO-Framework-Extension-Manager/commit/faa6a7b048a7e140c14beb0d5f09166ac6fd993e, we added a Google News sitemap via the Articles extension. It also includes image markup.

Why Articles? Well, it has the capability to annotate posts as NewsArticles 😃

https://github.com/sybrew/the-seo-framework/issues/31#issuecomment-445472929 through https://github.com/sybrew/the-seo-framework/issues/31#issuecomment-447154704 are now marked as resolved.

[sybrew]

Punted to a later update due to limited resources (time).