Magento2: Categories Path for Product URLs not working, issue with getProductUrl on Catalog Module?

[tigerx7]

It seems that getProductUrl() isn't returning a URL with the category in the path in the catalog pages. I'm able to reproduce this with a clean install using my own data and a clean install using the sample data.

Steps:
1) Loaded Magento 2.0.0 via Composer, including sample data
2) Did web based setup wizard. Sample data loaded.
3) Admin -> Stores -> Configuration -> Catalog -> Catalog -> Search Engine Optimization -> "Category URL Suffix" set to "", "Use Categories Path for Product URLs" set to "Yes"
4) Admin -> Products -> Catalog -> SKU "24-MB01" -> Search Engine Optimization -> Verified that URL Key was set to "joust-duffle-bag"
4) Flushed all caches plus static files
5) Navigated to sample data category: http://{domain}/gear/bags
6) Link on first product "Joust Duffle Bag " is to: "http://{domain}/catalog/product/view/id/1/s/joust-duffle-bag/category/4/" instead of the expected: "http://{domain}/gear/bags/joust-duffle-bag.html"

URL Rewrite is functioning since "gear/bags/joust-duffle-bag.html" is in the url_rewrite DB table and http:///gear/bags/joust-duffle-bag.html successfully pulls up the product page. It just seems there's an issue with getProductUrl() on the catalog module inserting the category correctly; unless I have a misunderstanding on something?

[sergei-sss]

+
The command php bin/magento indexer:reindex changes nothing. In the table 'catalog_url_rewrite_product_category' there are no changes...

[ghost]

Yes having the same issue "getProductUrl() isn't returning a URL with the category in the path in the catalog pages"

[vkholoshenko]

Hi, @tigerx7. Thanks for reporting the issue! We've created an internal ticket MAGETWO-46927 to process it.

[tigerx7]

Good to hear. I've been studying the code trying to get my head around it by following the creation of the URL path in the catalog models. I got as far as getUrl() in \Magento\Catalog\Model\Product\Url. I'm able to confirm the following values in my test setup right before an instance of \Magento\Framework\UrlFactory gets created on the last line:

$storeId = 1, $routePath = "catalog/product/view", $routeParams = ['id' => 39, 's' => 'prod-url-key', 'category' => 7, 'query' => []].

At that point it passes off to \Magento\Framework\UrlFactory and that's as far as I got. The parameters seem right, but I'm fairly new to Magento development and wouldn't consider myself too well versed just yet.

[tigerx7]

I did a bit more digging. It seems that $product->hasUrlDataObject() is returning false and so is $product->getRequestPath() in function getUrl so UrlRewrite and setRequestPath never kick in.

When ignoring the false check if statement on $product->getRequestPath(), the following $filterData gets formed in my test:

[filterData] => Array
        (
            [entity_id] => 62
            [entity_type] => product
            [store_id] => 1
            [metadata] => Array
                (
                    [category_id] => 5
                )
        )

However, when that gets passed to $this->urlFinder->findOneByData($filterData); it returns NULL

The product id is correct (62) and it is in category_id 5, which is 3 levels deep.

[GeromeF]

I did it working this way
delete FROM url_rewrite WHERE entity_type = 'product'
take a product and change categories for it
bin/magento indexer:reindex
urls are ok for the updated product, others are still wrong
Don't seems to be a bug, maybe someting went wrong during the install of the sample datas.

[tigerx7]

@GeromeF , I'm experiencing the same issue without sample data loaded. The urlrewrites are correct and I'm able to to load products when typing the rewritten url directly (ie. {domain}/{category}/{subcategory}/{product_url_key}. It brings up the product successfully. However, the url's that are being generated for the products when they're being listed at the category level and use catalog path in url is set to true are failing and generating something like: http://{domain}/catalog/product/view/id/1/s/{product_url_key}/category/4/

Are you able to reproduce successful URL's with the catalog path in the product list anchor tags at the catalog level?

[tigerx7]

While tracking down another issue, I think I found the problem with the Product URL paths. getProductUrl isn't taking into account categories set as anchors.

So say if a product is only assigned to the last child category: {default category}-->{level 1}->{level 2}->{level 3}. If a visitor navigates all the way to {level 3}, then the path for the product is generated correctly: {domain}/{level 1}/{level 2}/{level 3}/{product url key}.

However, if {level 2} is set as an anchor, and the visitor only navigates that far, then that's when getUrlPath() fails and generates the non-url-rewrite path for a product.

[vkholoshenko]

Sorry for the delay. First issue was fixed in e0f560f7d973e03a88f0f71ea932dc3add3599fa. And for issue from your last comment we've created one more internal ticket: MAGETWO-47656
Thanks for your input!

[tigerx7]

@vladimir-k Awesome, I was able to confirm that applying https://github.com/magento/magento2/commit/e0f560f7d973e03a88f0f71ea932dc3add3599fa fixed the issue for category assigned products. It's one step closer, now it's just the anchor categories having the url path issue. Applying the mentioned changes didn't fix the issue in anchor categories.

[rahulanand77]

Whenever we are creating product from Admin, it is doing an insert in "url_rewrite" table(magento2), after doing reindexing it is working fine on the front end. But when we are importing products from csv, then it is creating wrong url's like "catalog/product/view" etc. Even after reindexing its not working.

Also we have noticed, its not doing any insert in "url_rewrite" when doing reindexing. We have noticed that its only inserting into "url_rewrite" when doing insert from admin. Is it default feature of magento now?

[mslabko]

Hello @rahulanand77,
Url rewrite indexer was eliminated in Magento2 and url-rewrites are generated in run-time after saving particular entity.
Could you please provide more details - in what case you receive wrong url after import?

[rahul-parashar]

@mslabko

Same Issue & same case & same diagnosis after importing products via magento import module.

[eworksmedia]

We're having this issue on a brand new install, 2.0.2, and not when importing products but adding them normally. All categories are set to Is Anchor Yes.

• Root Category
  
  
  
  • Category 1
  
  
  • Sub Category 1
  
  
  • Sub Category 2
  
  
  • Category 2
  
  

When creating a product, if I only put it in "Sub Category 1" the url for the product when on the page for "Category 1" is

/category-1/product/view/id/1/s/product-urlkey/category/2/

But if I'm on the page for "Sub Category 1" the url is

/category-1/sub-category-1/product-urlkey.html

The URL should always be /category-1/sub-category-1/product-urlkey.html and never be /category-1/product/view/id/1/s/product-urlkey/category/2/. Has this been fixed yet?

edit: I can provide login details to the dev site if needed and access to anything else that's needed

[goowai]

I'm using Magento ver. 2.0.4,
Still the problem persists,

• To create new categories in magento works.
• If the categories were imported, not the rewrite avvine,
  how you can make a re-indexing?

If I try to change the category in the Key URL field, I get an error:

Notice: Undefined offset: 1 in /var/www/html/shop/vendor/magento/module-catalog-url-rewrite/Model/ProductUrlRewriteGenerator.php on line 166

[andidhouse]

same here - on import urls are messed up.
Seems like this topic has a long story and no solution so far?

[charleskj]

I am having the same issue, after updating products attribute from a csv import, my urls got messed up in the frontend, I am using the latest version 2.0.5.

Any updates on this

[oserediuk]

Thanks for the report! We have created issue MAGETWO-52879

[komsitr]

The issue as mentioned by @eworksmedia still persists on Magento 2.0.7
Further to his feedback, I try to add a product to multiple categories. The URL is only generated correctly when it is being viewed on the assigned categories' page. Please find the following scenarios:
1. Add product to "Sub Category 1" only
- At Category 1 the URL is "/catalog/product/view/id/5/s/product-1/category/3/"
- At Sub Category 1, the URL is correct "category-1/subcategory-1/product-1"
1. Add product to both "Sub Category 1" and "Category 1"
- At Category 1 the URL is correct "category-1/product-1"
- At Sub Category 1, the URL is correct "category-1/subcategory-1/product-1"

[ghost]

Any ideas on when this will be fixed by ? This is currently holding up a site going live.

Thanks.

[oserediuk]

I cannot reproduce this issue on current code version if I run reindex after import. Version 2.1 will be released soon.

[endergreen]

I agree that this looks to be working now for newly created products on the 2.1 version but what has to happen to get the older products to have the correct URL now. Have tried saving a product with reindexing and clearing cache but that still hasn't worked. Is the only way to delete the old products and recreate?

[djha108]

Hi
I have also similar problem
I have my old category name souvenir magneten changed to holland magneten
but the product url http://dev.klompjes.com/magneten/souvenir-magneten/holland-magneten-24-stuks
is not changing to http://dev.klompjes.com/magneten/holland-magneten/holland-magneten-24-stuks
I mean category url key in product url is not changing from old to new can you please suggest me

Thanks

[valibus]

+1 same problem here

[ps202]

I have imported a whole dbase with phpMyAdmin.
Then I turned on SEF-Urls, and created a few new products.
The new products (and all categories, even the old ones) are displayed with SEF Urls, the old products (the imported ones) are displayed without SEF Urls.
I have done reindexing from the command line as well, but that did not help.
Then I just deleted all data from the url_rewrite table where the entity_type='product', and now all of my products are without SEF Urls.
And reindexing from the command line does not help either.

[Krapulat]

I have the same problem.

Using Magento 2.1

[stratocentric]

Has anyone gotten this working? I noticed that in some cases you can control x the sef url, uncheck the rewrite box and then add the sef url back and hit save and it will work... not sure why though.

[developer-lindner]

Still broken in Version 2.1.1. Will there be a fix available for such crucial stuff somewhen?!

[Iazel]

Got this problem today with magento 2.1.2. I think the problem is when MagentoCatalog\Product\Url#getUrl try to find a $requestPath it sets the store_id to the current store_id, however in my case many urls are set only on the global store view 0 and these get ignored. One easy solution is to generate urls for every store, but this can result a little clunky when you add a new store view... The best solution would be to alter the algorithm to include the general store too or to build it on the fly if not found.

UPDATE: It seems like there's no url_rewrite for store_id === 0 and the backend just sync the store view that have Use default value flagged. However I can't understand how the default value thing works yet, any suggestion? I would like to mass update this field.

UPDATE 2: Ok, basically Use default value means it deletes/doesn't insert the attribute for that store view and Magento2 already (re)generate every link upon saving as long it has been modified, correctly handling the default store view. I fixed the missing links through a script and know all seems to work as expected.

[Tristan-N]

@Iazel Can you tell me something more about that script that worked for you? What script did you use, where did you put it and what did it do (in short). It would help me out a lot because I don't want to save 900+ categories one by one.

I have the same problem as mentioned by @djha108 and others. Using Magento 2.1.1 with PHP 7.0.12.

Thanks!

[Iazel]

@Tristan-N I've uploaded the script in this repo:
https://github.com/Iazel/magento2-regenurl

Please notice that this wasn't meant to be published, so it is a little rough and has been developed for my specific use case ^^" However it could be useful as a starting point.

[Tristan-N]

@Iazel Thanks a lot! this is great! I do have some further questions for you. Can i contact you by email for example?

[Iazel]

@Tristan-N if it something about this issue, please write it here, so everyone can benefit from it. If it is something regarding the script, then open an issue on the repo :)

[Morgon]

This is definitely not fixed, @oserediuk . Our base install was 2.1.0, subsequently upgraded to 2.1.2, and the Webserver Rewrites setting is not enabling SEF URLs for products.

[caesarcxiv]

@Iazel - nice script :) --- here are a couple issues i found.

information: all of our sef urls are stored under store_id 0 in the catalog_product_entity_varchar table

1.calling php bin/magento iazel:regenurl is the only way I have found for the script to work.

1. calling the script deletes and regenerates the url even if it exists correctly
2. calling php bin/magento iazel:regenurl s4 1 - deletes the rewrite for store 4, to rebuild I have to run php bin/magento iazel:regenurl which will rebuild the url for s4

[Iazel]

calling php bin/magento iazel:regenurl is the only way I have found for the script to work.

Yep, omitting the -s option is equal to -s0

calling the script deletes and regenerates the url even if it exists correctly

Yes, the original use case was to regenerate only a well known list of products and that's why it also deletes it.

calling php bin/magento iazel:regenurl s4 1 - deletes the rewrite for store 4, to rebuild I have to run php bin/magento iazel:regenurl which will rebuild the url for s4

This is strange. It should delete AND regen the url for the provided store and product. Please note that the correct syntax is -s4 and not s4, but I'll make some test when I have some time.

[miro91]

Hi @Iazel
I tried your script and I can't get URL rewrite for specific product and store.
I have 4 different stores and I want to generate URL's for one product for store with id = 3
php bin/magento iazel:regenurl -s3 3082
After execution I can't find record in my url_rewrite table for entity_id= 3082 and store_id = 3
for store 1 => 8 records (which seems ok because the product is related to 8 categories)
for store 2 => only 1 record which is weird
and for store 3 => 0 records

I also tried to run your script for store 3 for all products and only few products get new URLs

[Iazel]

Hi @miro91
Maybe the 3082 product has "use default" in the url rewrite? All the products that use the default must be regenerated with the default store (don't use the -s option).
Give a try to:

php bin/magento iazel:regenurl 3082

[Gael42]

Thanks a lot @Iazel for your command, it helped me a lot.

Using it i got problem to generate correct full SEO friendly URLS in different languages, ex :

http://domain.com/categories/written/in/french/french_url_key_for_product
AND
http://domain.com/categories/written/in/english/english_url_key_for_product

In case it helps, I solved this issue by setting the current store via the StoreManager before calling @Iazel code, but I did it in a custom admin block code, because I didn't manage to find a good way to get the StoreManager instance in a console command context.

Executing this for every needed store permit me to have full SEO friendly urls for my main store, and my secondary language store view.

[Tristan-N]

@Gael42 Thank you for this update. This sounds a great way to get full and clear URL's using regenurl (which works fine) in combination with the StoreManager.

Can you give me a bit of clarification about the steps you took and how you combined StoreManager with RegenUrl? If possible a little step by step or a summary of the steps you took?

I'm not familiar with StoreManager but my colleagues are.

Kind regards

[miro91]

@Tristan-N I played a little bit with the @Iazel code and add the store manager for regenerating URL's for all stores if the Magento is having multistores. I tried it on Magento with 4 stores and works great. You don't need to add any options to the command just execute php bin/magento iazel:regenurl

Here is the code for RegenerateProductUrlCommand.php

<?php    
namespace Iazel\RegenProductUrl\Console\Command;

use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Magento\UrlRewrite\Service\V1\Data\UrlRewrite;
use Magento\UrlRewrite\Model\UrlPersistInterface;
use Magento\CatalogUrlRewrite\Model\ProductUrlRewriteGenerator;
use Magento\Catalog\Model\ResourceModel\Product\Collection;
use Magento\Store\Model\Store;
use Magento\Store\Model\StoreManagerInterface;

class RegenerateProductUrlCommand extends Command
{
    /**
     * @var ProductUrlRewriteGenerator
     */
    protected $productUrlRewriteGenerator;

    /**
     * @var UrlPersistInterface
     */
    protected $urlPersist;

    /**
     * @var ProductRepositoryInterface
     */
    protected $collection;

    public function __construct(
        \Magento\Framework\App\State $state,
        Collection $collection,
        ProductUrlRewriteGenerator $productUrlRewriteGenerator,
        UrlPersistInterface $urlPersist,
        StoreManagerInterface $storeManager
    ) {
        $state->setAreaCode('adminhtml');
        $this->collection = $collection;
        $this->productUrlRewriteGenerator = $productUrlRewriteGenerator;
        $this->urlPersist = $urlPersist;
        $this->storeManager = $storeManager;
        parent::__construct();
    }

    protected function configure()
    {
        $this->setName('iazel:regenurl')
            ->setDescription('Regenerate url for given products')
            ->addArgument(
                'pids',
                InputArgument::IS_ARRAY,
                'Products to regenerate'
            )
            ->addOption(
                'store', 's',
                InputOption::VALUE_REQUIRED,
                'Use the specific Store View',
                Store::DEFAULT_STORE_ID
            )
            ;
        return parent::configure();
    }

    public function execute(InputInterface $inp, OutputInterface $out)
    {
        if ($this->storeManager->isSingleStoreMode()) {
            $stores = [$this->storeManager->getStore(0)];
        } else {
            $stores = $this->storeManager->getStores();
        }

        foreach ($stores as $store) {
            // $store_id = $inp->getOption('store');
            $store_id = $store->getId();
            $this->collection->addStoreFilter($store_id)->setStoreId($store_id);

            $pids = $inp->getArgument('pids');
            if( !empty($pids) )
                $this->collection->addIdFilter($pids);

            $this->collection->addAttributeToSelect(['url_path', 'url_key']);
            $list = $this->collection->load();
            foreach($list as $product)
            {
                $product->setStoreId($store_id);
                $this->urlPersist->deleteByData([
                    UrlRewrite::ENTITY_ID => $product->getId(),
                    UrlRewrite::ENTITY_TYPE => ProductUrlRewriteGenerator::ENTITY_TYPE,
                    UrlRewrite::REDIRECT_TYPE => 0,
                    UrlRewrite::STORE_ID => $store_id
                ]);
                try {
                    $this->urlPersist->replace(
                        $this->productUrlRewriteGenerator->generate($product)
                    );
                }
                catch(\Exception $e) {
                    $out->writeln('<error>Duplicated url for '. $product->getId() .' store id '. $store_id .'</error>');
                }
            }
        }
    }
}

Hope this helps

[kanduvisla]

Thanks for your effort @miro91

In case someone needs it, here is a CLI command to re-create all (missing) URL rewrites for all categories:

<?php

namespace Vendor\Module\Console;

use Magento\Framework\Exception\AlreadyExistsException;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

/**
 * Class RegenerateUrls.php
 */
class RegenerateUrls extends Command
{
    /**
     * @var \Magento\Store\Model\StoreManagerInterface
     */
    protected $storeManager;

    /**
     * @var \Magento\UrlRewrite\Model\UrlRewriteFactory
     */
    protected $urlRewriteFactory;

    /**
     * @var \Magento\Catalog\Model\ResourceModel\Category\CollectionFactory
     */
    protected $categoryCollectionFactory;

    /**
     * RegenerateUrls constructor.
     * @param \Magento\UrlRewrite\Model\UrlRewriteFactory $urlRewriteFactory
     * @param \Magento\Store\Model\StoreManagerInterface $storeManager
     * @param \Magento\Catalog\Model\ResourceModel\Category\CollectionFactory $categoryCollectionFactory
     * @param string $name
     */
    public function __construct(
        \Magento\UrlRewrite\Model\UrlRewriteFactory $urlRewriteFactory,
        \Magento\Store\Model\StoreManagerInterface $storeManager,
        \Magento\Catalog\Model\ResourceModel\Category\CollectionFactory $categoryCollectionFactory,
        $name = 'regenerate_category_urls'
    )
    {
        $this->urlRewriteFactory = $urlRewriteFactory;
        $this->storeManager = $storeManager;
        $this->categoryCollectionFactory = $categoryCollectionFactory;
        parent::__construct($name);
    }

    /**
     * Configure the command
     */
    protected function configure()
    {
        $this->setName('gb:regenerate_urls');
        $this->setDescription('Regenerate Url\'s for categories');
    }

    /**
     * @param InputInterface $input
     * @param OutputInterface $output
     */
    protected function execute(InputInterface $input, OutputInterface $output)
    {
        // Fetch all categories:
        $categories = $this->categoryCollectionFactory->create()->addAttributeToSelect('url_key');
        /** @var \Magento\Store\Api\Data\StoreInterface $store */
        foreach ($this->storeManager->getStores() as $store)
        {
            echo $store->getCode() . ':';
            /** @var \Magento\Catalog\Model\Category $category */
            foreach ($categories as $category) {

                $urlRewrite = $this->urlRewriteFactory->create();
                $urlRewrite->addData(
                    [
                        'entity_type' => \Magento\UrlRewrite\Controller\Adminhtml\Url\Rewrite::ENTITY_TYPE_CATEGORY,
                        'entity_id' => $category->getId(),
                        'request_path' => $category->getUrlKey(),
                        'target_path' => 'catalog/category/view/id/' . $category->getId(),
                        'redirect_type' => 0,
                        'store_id' => $store->getId(),
                        'description' => null,
                        'is_autogenerated' => 1,
                        'metadata' => null
                    ]
                );
                try {
                    $urlRewrite->getResource()->save($urlRewrite);
                    echo '.';
                } catch (AlreadyExistsException $alreadyExistsException) {
                    echo '-';
                }
            }
            echo "\n";
        }
    }
}

Now if someone can combine these two in one solution to rule them all that would be great.

[pravalitera]

@kanduvisla

I discover this morning that there is no more indexer for url, and you post that script ^^ Thx !
But I just ran it, and i have a pb.

For example i have 1 category with 2 subcategorys:

And the URLs rewritten are
http://www.site.dev/category1
http://www.site.dev/subcategory1
http://www.site.dev/subcategory2

It should be http://www.site.dev/category1/subcategory1

And maybe with the suffix too

I'm looking into it too, posting if i find how to.

Anyway, thx for posting your solution.

Edit: And when i look into the observer of the CSV import, i understand now why importing a lot of category through it is very VERY slow...

Edit2: I'm writing a solution base on the block TopMenu : Get the tree of category and then do the job. i'll try to handle the "move" too...

[pravalitera]

• Full path with URL suffix
• Keep in mind that there are probably a nicer way. Maybe with _Magento\CatalogUrlRewrite\Model\CategoryUrlPathGenerator_

// namespace Yours :) ;

use Magento\Framework\Exception\AlreadyExistsException;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;


/**
 * Class RegenerateUrls.php
 */
class RegenerateUrls extends Command
{
    /**
     * @var \Magento\Store\Model\StoreManagerInterface
     */
    protected $storeManager;

    /**
     * @var \Magento\UrlRewrite\Model\UrlRewriteFactory
     */
    protected $urlRewriteFactory;


    /**
     * @var \Magento\Catalog\Helper\Category
     */
    protected $_categoryHelper;

    /**
     * @var \Magento\Catalog\Model\CategoryFactory
     */
    protected $_categoryFactory;


    /**
     * @var \Magento\Framework\App\Config\ScopeConfigInterface
     */
    protected $scopeConfig;

    /**
     * [category_id] = url_key
     * @var array
     */
    protected $_categoryUrlKeys = [];


    /**
     * [store_id] = category_suffix
     * @var array
     */
    protected $_category_suffixes=[];



    /**
     * RegenerateUrls constructor.
     * @param \Magento\UrlRewrite\Model\UrlRewriteFactory $urlRewriteFactory
     * @param \Magento\Store\Model\StoreManagerInterface $storeManager
     * @param \Magento\Catalog\Model\ResourceModel\Category\CollectionFactory $categoryCollectionFactory
     * @param string $name
     */
    public function __construct(
        \Magento\UrlRewrite\Model\UrlRewriteFactory $urlRewriteFactory,
        \Magento\Store\Model\StoreManagerInterface $storeManager,
        \Magento\Catalog\Helper\Category $categoryHelper,
        \Magento\Catalog\Model\CategoryFactory $categoryFactory,
        \Magento\Framework\App\Config\ScopeConfigInterface $scopeConfig,
        $name = 'regenerate_category_urls'
    )
    {
        $this->urlRewriteFactory = $urlRewriteFactory;
        $this->storeManager = $storeManager;
        $this->_categoryHelper = $categoryHelper;
        $this->_categoryFactory = $categoryFactory;
        $this->scopeConfig = $scopeConfig;

        parent::__construct($name);
    }

    /**
     * Configure the command
     */
    protected function configure()
    {
        $this->setName('gb:regenerate_urls');
        $this->setDescription('Regenerate Url\'s for categories');
    }

    /**
     * @param InputInterface $input
     * @param OutputInterface $output
     */
    protected function execute(InputInterface $input, OutputInterface $output)
    {
        foreach ($this->storeManager->getStores() as $store) {
            echo $store->getCode() . ':';
            $this->_category_suffixes[$store->getId()] = $this->scopeConfig->getValue(
                'catalog/seo/category_url_suffix',
                \Magento\Store\Model\ScopeInterface::SCOPE_STORE,
                $store->getId()
            );
            $this->getChildrenCategories($store);
            echo "\n";
        }
    }

    protected function insertIntoRewrites($id,$url_key,$store_id) {
        $urlRewrite = $this->urlRewriteFactory->create();
        $urlRewrite->addData([
                'entity_type' => \Magento\UrlRewrite\Controller\Adminhtml\Url\Rewrite::ENTITY_TYPE_CATEGORY,
                'entity_id' => $id,
                'request_path' => $url_key,
                'target_path' => 'catalog/category/view/id/' .$id,
                'redirect_type' => 0,
                'store_id' => $store_id,
                'description' => null,
                'is_autogenerated' => 1,
                'metadata' => null
            ]
        );
        try {
            $urlRewrite->getResource()->save($urlRewrite);
            echo '.';
        } catch (AlreadyExistsException $alreadyExistsException) {
            echo '-';
        }

        return $this;
    }



    public function getChildrenCategories($store,$current_category = null)
    {
        if(is_null($current_category))
            $parent_id = $store->getRootCategoryId();
        else {
            $parent_id = $current_category->getId();
            $this->_categoryUrlKeys[$current_category->getId()] = $current_category->getUrlKey();
            $path_exploded = explode('/',$current_category->getPathId());
            $complete_path = [];
            foreach($path_exploded as $path){
                if($path == 1 || $path == $store->getRootCategoryId()){
                    continue;
                    }
                    $complete_path[]= $this->_categoryUrlKeys[$path];
                }

            $complete_path = implode('/',$complete_path) . $this->_category_suffixes[$store->getId()];

            $this->insertIntoRewrites($current_category->getId(),$complete_path,$store->getId());
            }

        $category = $this->_categoryFactory->create();

        $recursionLevel = 0;
        $childrenCategories = $category->getCategories($parent_id, $recursionLevel, false, false, true);
        if(count($childrenCategories)){
            foreach($childrenCategories as $child){
                $this->getChildrenCategories($store,$child);
            }
        }
        return $this;
    }

}

[ps202]

What I have noticed, is that that you should delete all the product related url-rewrites from the database table, and then make this script run!

[Tjitse-E]

@pravalitera your script works, thanks.

Example: www.mainstore.fr/level1/level2/ <-- level 1 has the main store url value, not the store view url value. Do you experience the same problem?

[pravalitera]

@Tjitse-E It's because i suck :D "_categoryUrlKeys" does not contain the store_id, you will have to

something like this:

l.144 $this->_categoryUrlKeys[$store->getId().'_'.$current_category->getId()] = $current_category->getUrlKey();

and just after in the foreach.

$this->_categoryUrlKeys[$store->getId()."_".$path]

I'm close to a very much cleaner script, that rewrites product too. But i have to handle another emergency right now :(

[Tjitse-E]

@pravalitera i've modified the script, according your comment. The strange thing is that the regenerated url's don't have the store view value's, but the main store value's after regeneration. I'm digging in the DB now to find out more...

[pravalitera]

@Tjitse-E I think it's because when the table "url_rewrite" is already generated, you fall into the "catch" of the insert (duplicate). You have "-" instead of "." when you launch it.

I will repost asap with a proper script that does everything right, i'm dealing with another things right now. I'm just astonished that i have to rewrite something to reindex urls...

[pravalitera]

Ok. Here is a new version that rewrite products and category.

• @miro91 Your script does not work with url_key by stores. And seems very slow compared to mine, maybe i done something wrong.
• @Tjitse-E It's maybe because of laking of ->setStoreFilter on collection or ->setStoreId on objects...
• It will always DELETE old urls. So no redirect of old ones.
• I will do another version when i can, to handle params such as $product_ids etc...
• Expect new version soon.
• You will have to write a plugin... because I think i found a bug in Magento 2.

in _Magento\CatalogUrlRewrite\Model\ProductUrlRewriteGenerator;_

l.109
$productCategories = $product->getCategoryCollection()
            ->addAttributeToSelect('url_key')
            ->addAttributeToSelect('url_path')
            ->setStore($storeId) // Line to add !!!
        ;

If you don't do that, the product will not have the proper category path by stores...

My new script :

/**
 * CLI Command to Reindex URLS
 *
 *
 * @author RAVALITERA Pol <[email protected]>
 * @author Tjitse <@github>
 * @author kanduvisla <@github>
 * @author miro91 <@github>
 * @version 0.0.1
 */

namespace Afg\Core\Console\UrlRewriting;

use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Magento\UrlRewrite\Model\UrlPersistInterface;
use Magento\CatalogUrlRewrite\Model\CategoryUrlRewriteGenerator;

/**
 * Class RegenerateUrls.php
 */
class RegenerateUrls extends Command
{
    /**
     * @var \Magento\Store\Model\StoreManagerInterface
     */
    protected $storeManager;

    /**
     * @var \Magento\UrlRewrite\Model\UrlRewriteFactory
     */
    protected $urlRewriteFactory;

    /**
     * @var CategoryUrlRewriteGenerator
     */
    protected $categoryUrlRewriteGenerator;

    /**
     * @var UrlPersistInterface
     */
    protected $urlPersist;

    /**
     * @var \Magento\Catalog\Model\ResourceModel\Category\CollectionFactory
     */
    protected $categoryCollectionFactory;

    /**
     * @var \Magento\CatalogUrlRewrite\Observer\UrlRewriteHandler
     */
    protected $urlRewriteHandler;


    /**
     * RegenerateUrls constructor.
     * @param \Magento\UrlRewrite\Model\UrlRewriteFactory $urlRewriteFactory
     * @param \Magento\Store\Model\StoreManagerInterface $storeManager
     * @param \Magento\Catalog\Model\ResourceModel\Category\CollectionFactory $categoryCollectionFactory
     * @param string $name
     */
    public function __construct(
        \Magento\UrlRewrite\Model\UrlRewriteFactory $urlRewriteFactory,
        \Magento\Store\Model\StoreManagerInterface $storeManager,
        \Magento\CatalogUrlRewrite\Model\CategoryUrlRewriteGenerator $categoryUrlRewriteGenerator,
        \Magento\Catalog\Model\ResourceModel\Category\CollectionFactory $categoryCollectionFactory,
        UrlPersistInterface $urlPersist,
        \Magento\CatalogUrlRewrite\Observer\UrlRewriteHandler $urlRewriteHandler,
        $name = 'regenerate_urls'
    )
    {
        $this->urlRewriteFactory = $urlRewriteFactory;
        $this->storeManager = $storeManager;

        $this->categoryUrlRewriteGenerator = $categoryUrlRewriteGenerator;
        $this->categoryCollectionFactory = $categoryCollectionFactory;
        $this->urlPersist = $urlPersist;
        $this->urlRewriteHandler = $urlRewriteHandler;

        parent::__construct($name);
    }

    /**
     * Configure the command
     */
    protected function configure()
    {
        $this->setName('afg:rewrite_url:category2');
        $this->setDescription('Regenerate Url\'s for categories');
    }

    /**
     * @param InputInterface $input
     * @param OutputInterface $output
     */
    protected function execute(InputInterface $input, OutputInterface $output)
    {

        if ($this->storeManager->isSingleStoreMode()) {
            $stores = [$this->storeManager->getStore(0)];
        } else {
            $stores = $this->storeManager->getStores();
        }

        $start_time = microtime(true);
        foreach ($stores as $store) {
            $output->writeln($store->getCode().'...');
            $categories = $this->categoryCollectionFactory->create()->setStore($store->getId())
                ->addAttributeToSelect(array('url_key', 'url_path', 'is_anchor'))
                ->addAttributeToFilter('level', 2)
                ->addAttributeToFilter('parent_id',$store->getRootCategoryId())
            ;
            foreach ($categories as $category) {
                if ($category->getUrlKey()) {
                    $category->setStoreId($store->getId());
                    $urlRewrites = array_merge(
                        $this->categoryUrlRewriteGenerator->generate($category, true)
                        , $this->urlRewriteHandler->generateProductUrlRewrites($category)
                    );
                    $this->urlPersist->replace($urlRewrites);
                }
            }
        }
        $output->writeln('[DONE] ' . round(microtime(true) - $start_time,2) .' sec');
    }

}

[Gael42]

@Tristan-N there is the code I use in a custom admin route
It's almost @iazel code plus the $this->_storeManager->setCurrentStore($store_id); line

    public function rewrite_regen()
    {
        $store_id = 5; // Store id - mandatory
        $pids = [13879]; // Product id list, if empty all products are processed
        //$pids = [];

        echo "regen urls for store id #".$store_id.':<br>';
        $this->_storeManager->setCurrentStore($store_id);
        $collection = $this->_productCollectionFactory->create();
        $collection->addStoreFilter($store_id)->setStoreId($store_id);


        if( !empty($pids) )
            $collection->addIdFilter($pids);

        $collection->addAttributeToSelect(['url_path', 'url_key']);
        $list = $collection->load();
        foreach($list as $product)
        {
            if($store_id === \Magento\Store\Model\Store::DEFAULT_STORE_ID)
                $product->setStoreId($store_id);

            $this->_urlPersist->deleteByData([
                \Magento\UrlRewrite\Service\V1\Data\UrlRewrite::ENTITY_ID => $product->getId(),
                \Magento\UrlRewrite\Service\V1\Data\UrlRewrite::ENTITY_TYPE => \Magento\CatalogUrlRewrite\Model\ProductUrlRewriteGenerator::ENTITY_TYPE,
                \Magento\UrlRewrite\Service\V1\Data\UrlRewrite::REDIRECT_TYPE => 0,
                \Magento\UrlRewrite\Service\V1\Data\UrlRewrite::STORE_ID => $store_id
            ]);
            try {
                $this->_urlPersist->replace(
                    $this->_productUrlRewriteGenerator->generate($product)
                );
            }
            catch(\Exception $e) {
                echo "error: product id #".$product->getId().': '.$e->getMessage();
            }
        }
    }

And you need to inject \Magento\Catalog\Model\ResourceModel\Product\CollectionFactory and get StoreManager from context in the constructor of your Block class :

    protected $_productCollectionFactory;
    protected $_storeManager;

    public function __construct(
        \Magento\Catalog\Model\ResourceModel\Product\CollectionFactory $productCollectionFactory
    )
    {

        $this->_productCollectionFactory = $productCollectionFactory;
        $this->_storeManager = $context->getStoreManager();

        parent::__construct($context);
    }

[pravalitera]

@Gael42 The _$storeManager->setCurrentStore()_ handle the bug i speak about just over, in _Magento\CatalogUrlRewrite\Model\ProductUrlRewriteGenerator_ Still a bug to me, but it handles it, thx.

I think your code will just be impossible to run on my website: i have more than 100 000 products ! If it's like in Magento 1, the collection will make the script go out of memory :(

i'll try !

And i don't understand the "delete" part: the replace already does that. (Edit: ah ok, to delete even if we moved a product)

@iazel code does not work out of the box : when i do the _setup:upgrade_ , i have:

[Magento\Framework\Exception\LocalizedException]
Area code is already set

I had to comment the _$state->setAreaCode('adminhtml');_ to setup, then decomment it after... Weird.

I imported my 100 000 products with a modified Magmi i have done... A shame that there is no more url indexer...

[Tjitse-E]

@pravalitera @Iazel code works on my store without problems. Although i'm using a slightly older version of https://github.com/Iazel/magento2-regenurl. All the product url's are generated perfectly on each store view (i've tested with 12 store views).

Thanks for your efforts!

@pravalitera i've tested your last generation code (including the fix in _Magento\CatalogUrlRewrite\Model\ProductUrlRewriteGenerator_), on a multistore with 10 store-views in multiple languages. But the RegenerateUrls script seems to mess up the new category rewrites. The store value's just seem random. I've got category URL's like: www.mainsite.com/English_url/Url_ Francais/. Do you have the same problems?

Steps that i've taken to test the script:

1. Remove category rewrites in DB DELETE FROM `url_rewrite` WHERE `entity_type` LIKE 'category';
2. Regenerate catetegory rewrites n98-magerun2 gb:regenerate_urls
3. Reindex/Flush cache.

[pravalitera]

@Tjitse-E It seems ok on my side... i dunno why...
The fix in ProductUrlRewriteGenerator is not mandatory if you put _$this->storeManager->setCurrentStore($store->getId())_; in the "_foreach($store)_

My scripts does not handle product that are not in any categories, that's why @lazel's script was slower than mine... Stupid i am ^^

I just have a question: How many products do you have in your base ? Because making a collection of 100 000 products is a big deal i think.

And the big flaw that we have is that it does not handle 301 redirection for old url... But i think I know how to make it...

[Tjitse-E]

@pravalitera we only have 300 products, 60 categories and 10 store views in different languages, so no big deal there. We've corrected the category rewrites manually, so the problem for this client is resolved. But it would be nice if Magento solves this in 2.2...

[crantron]

Incase this helps anyone. I truncated the url_rewrite table.

• I used @Iazel module to restore all product rewrites.
• Then for categories went to each sub root category, and resaved.
• Then went to SEO and changed everything to not have .html
• now everything is perfect.

thanks for this thread, helped a bunch. cheers!

[MattDelac]

@pravalitera , you said "And the big flaw that we have is that it does not handle 301 redirection for old url... But i think I know how to make it..."

Did you find how to do it ? This would be a nice feature !

Thank you :)

[pixiemediaweb]

juat wanted to stop by and say thanks @Iazel you sir are a life saver.

[jarhody]

I have experienced a similar issue on 2.1.5.

All migrated products work fine and creating new products works fine (URLs correctly formatted) but, if I use the duplicate feature the product url is formatted incorrectly.

No imports. Migrated from 1.9 to 2.1.2.

Correct site URLS
site.com/top-category.html
site.com/top-category/sub-category.html
Use category paths for product URLs = NO
Correct product URL (regardless of category)
site.com/product-url-key.html

Incorrect product URL (made by using Save & Duplicate)
site.com/catalog/product/view/id/2786/s/product-url-key/category/208/

If I manually type in:
site.com/product-url-key.html
...results in a 404.

Admin settings
Use Flat Catalog Category = NO
Use Flat Catalog Product = YES
Product URL suffix = .html
Category URL suffix = .html
Use category paths for product URLs = NO
Create permanent redirects on key change = YES

[skast96]

Is this issue still there in Version 2.1.7 , cause it is pretty annoying.

[Jeordy]

@skast96 That's correct. However we turned off 'Use Categories Path for Product URLs' and now the canonical url is used. That works, but the strange thing is that half of the products have correct url's and the other half don't when enabling the option again. All our products are inserted by hand, so there should be no problem creating url keys.

[skast96]

@Jeordy thank you for your answer, I have been thinking about this issue quiet a while, and came to the conclusion that i just load a collection with one particular product in it and call the addRewriteUrl function on it. It's the easiest way I thought of and it works really well. The best way would be adding this method to the product itself if someone, like me, just needs one particular product url.

[Jeordy]

@skast96 We've solved the issue with some help. It seems we changed the dbStorage with a Github fix that isn't a fix. We reverted this 'fix' and used a module called: URL Rewrite Regeneration from ThLassche (https://marketplace.magento.com/thlassche-regeneraterewrites.html). Great module and I advise you to buy and use it. Great support too from ThLassche himself.

[duckchip]

@magento-engcom-team could you provide the pull request in where this is fixed so I can create a back-port for 2.1 and 2.2 branches. Or provide me the back-port if it already exists :)
thx

[ghost]

Since it is magento and version 2.3 has been released there is still no solution for this matter?
Why doesn't magento-engcom-team move this issue forward in TODO in branch version 6.8-develop so we know it will be repaired in a few years from now. Having this problem since version 2.1.4. I refuse to pay for a matter that is well known and there should be a decent basic and free (still open-source) solution for it already. Hoping for a Miracle or Magento to get there software right.

[Eddcapone]

@koopjesboom We are using Enterprise 2.3.5-p2 and the problem is still not fixed. If I execute php bin/magento indexer:reindex, then the url_rewrite Table does not change.

[hostep]

@Eddcapone: Magento 2 doesn't generate url rewrites with indexers, Magento 1 did that, but Magento 2 has no official way to regenerate url rewrites.
If you want to regenerate them, you need to use an external module (be sure to test one of these thoroughly before running it on production, it could destroy your SEO ranking):

• https://github.com/elgentos/regenerate-catalog-urls/
• https://github.com/olegkoval/magento2-regenerate_url_rewrites
• ...

This module might also be useful to find incorrect data in the url data of your catalog btw:

• https://github.com/baldwin-agency/magento2-module-url-data-integrity-checker

[Eddcapone]

@hostep, Thank you for the info. Do you know why they removed the feature from M2? How are we officialy supposed to auto generate URL Rewrites now without using a 3rd party extension? I thought M2 is supposed to be better than M1.

I tried https://github.com/olegkoval/magento2-regenerate_url_rewrites but it does not generate category urls. I will try the others though. Thanks!

[hostep]

It's bad for SEO if you regenerate a bunch of url rewrites without then also generating 301 redirects from the old to the new url. I think that was the main thought when they rebuild the system for M2. But never saw such an official statement being made, this is only guessing here.

I do agree that it's a feature missing in M2 in certain cases, for example when you aren't in production yet and are building your store and tweaking settings/data to try out various ways of setting up url's. Then it would be handy if you could regenerate all url rewrites to make them consistent. Unfortunately you need 3rd party modules to do this.

In theory all changes should always cause correct url rewrites to be generated (including 301 redirects if you ask for it), but there are way too many bugs in M2 where this fails, from time to time such a bug gets fixed but it goes very slowly and there are still a bunch of bugs open I believe.

Good luck with your endeavours! The url rewrite problems in M2 are probably one of the most frustrating things I had to deal with so far.

[ghost]

@hostep, Thank you for the info. Do you know why they removed the feature from M2? How are we officialy supposed to auto generate URL Rewrites now without using a 3rd party extension? I thought M2 is supposed to be better than M1.

I tried https://github.com/olegkoval/magento2-regenerate_url_rewrites but it does not generate category urls. I will try the others though. Thanks!

I have used this Oleg Koval Extension and yes it generates the URL's for the category urls.

If not use separate commands for products and categories: $> php bin/magento ok:urlrewrites:regenerate --entity-type=category

All the commands you can find here: https://github.com/olegkoval/magento2-regenerate_url_rewrites

I do not know if there is dry run in this olegkoval extension, if not this should a default feature.