Magento2: New bulk schedule doesn't create operations automatically

[duckchip]

Preconditions (*)

1. Magento 2.3.1

Steps to reproduce (*)

1. Create a new bulk schedule

$bulkDescription = $this->getBulkMessage($order);
$bulkUuid = $this->identityService->generateId();

$data = ['just' => 'some dummy'];
$operationData = [
    'data' => [
        'bulk_uuid' => $bulkUuid,
        'topic_name' => self::TOPIC_NAME,
        'serialized_data' => $this->serializer->serialize($data),
        'status' => \Magento\Framework\Bulk\OperationInterface::STATUS_TYPE_OPEN,
    ]
];

$operation = $this->operationsFactory->create($operationData);

$this->bulkManagement->scheduleBulk(
    $bulkUuid,
    [$operation],
    $bulkDescription,
    $this->userContext->getUserId()
);

In this scenario we add 1 operation to a bulk. The moment scheduleBulk is called, a new record inside the magento_bulk table is created.

Even with the queues up and running the operations are not created and stay "Pending"

I think the problem relies in the fact that the operation entities are never saved to the database.

Following function of class Magento\AsynchronousOperationsModel\BulkManagement

/**
     * Publish list of operations to the corresponding message queues.
     *
     * @param array $operations
     * @return void
     */
    private function publishOperations(array $operations)
    {
        $operationsByTopics = [];
        foreach ($operations as $operation) {
+            $this->entityManager->save($operation);
            $operationsByTopics[$operation->getTopicName()][] = $operation;
        }
        foreach ($operationsByTopics as $topicName => $operations) {
            $this->publisher->publish($topicName, $operations);
        }
    }

Adding this line $this->entityManager->save($operation); in the publicOperations function, resolves that problem for me.

Expected result (*)

1. [Screenshots, logs or description]
2. When a bulk is scheduled, the operations are created automatically

Actual result (*)

1. [Screenshots, logs or description]
2. When a bulk is scheduled, the operations are not created

[m2-assistant[bot]]

Hi @duckchip. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• [ ] Summary of the issue
• [ ] Information on your environment
• [ ] Steps to reproduce
• [ ] Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.3-develop instance - upcoming 2.3.x release

For more details, please, review the Magento Contributor Assistant documentation.

@duckchip do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• [ ] yes
• [ ] no

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[vijay-wagento]

@magento give me 2.3-develop instance - upcoming 2.3.x release

[magento-engcom-team]

Hi @vijay-wagento. Thank you for your request. I'm working on Magento 2.3-develop instance for you

[magento-engcom-team]

Hi @vijay-wagento, here is your Magento instance.
Admin access: https://i-23958-2-3-develop.instances.magento-community.engineering/admin
Login: admin Password: 123123q
Instance will be terminated in up to 3 hours.

[duckchip]

I will close this issue since there's a workaround (thanks to @nuzil for sending me in the right direction). This should be documented in the devdocs as well (will create a PR for that later.)

The trick for saving those operations is by calling the Operations Repository.

Given the example of the devdocs

public function execute($operationData)
    {
        $operationCount = count($operationData);
        if ($operationCount > 0) {
            $bulkUuid = $this->identityService->generateId();
            $bulkDescription = 'Specify here your bulk description';

            $operations = [];
            foreach ($operationData as $operation) {

                $serializedData = [
                    //this data will be displayed in Failed item grid in ID column
                    'entity_id' => $operation['entity_id'],
                    //add here logic to add url for your entity(this link will be displayed in the Failed item grid)
                    'entity_link' => $this->urlBuilder->getUrl('your_url'),
                    //this data will be displayed in Failed item grid in the column "Meta Info"
                    'meta_information' => 'Specify here meta information for your entity',//this data will be displayed in Failed item grid in the column "Meta Info"
                ];
                $data = [
                    'data' => [
                        'bulk_uuid' => $bulkUuid,
                        //topic name must be equal to data specified in the queue configuration files
                        'topic_name' => '%your_topic name%',
                        'serialized_data' => $this->jsonHelper->jsonEncode($serializedData),
                        'status' => OperationInterface::STATUS_TYPE_OPEN,
                    ]
                ];

                /** @var OperationInterface $operation */
                $operation = $this->operationFactory->create($data);
                $operations[] = $operation;

            }
            $userId = $this->userContext->getUserId();
            $result = $this->bulkManagement->scheduleBulk($bulkUuid, $operations, $bulkDescription, $userId);
            if (!$result) {
                throw new \Magento\Framework\Exception\LocalizedException(
                    __('Something went wrong while processing the request.')
                );
            }
        }

We'll have to call use the MassSchedule:

$this->massSchedule->publishMass( 'async.V1.inventory.bulk-product-source-transfer.POST', $operations, null, $userId );

instead of

$this->bulkManagement->scheduleBulk($bulkUuid, $operations, $bulkDescription, $userId);

since the MassSchedule uses the OperationRepository to save the Operation entities.

[martinog22]

Hello,
i have still the problem in magento 2.3.2
I tried it with the solutions here but no chance.
Is there maybe another solution or a step by step documentation to check it again?
thanks!

[zw1111]

Hello,
i have still the problem in magento 2.3.2
I tried it with the solutions here but no chance.
Is there maybe another solution or a step by step documentation to check it again?
thanks!

I am also having this issue with Magento 2.3.2. The message still states "Task "Update attributes for 4 selected products": 1 item(s) have been scheduled for update." even after trying to make the code modifications. It seems that the change does not take into account updates that are already queued. How do you get rid of or cancel a queued update?

[arion-p]

This issue has been closed but the problem remains: Magento bulk actions (the ones offered by magento not custom ones) simply don't work. Yet there is no other open issue to track this.
So will this ever be fixed or is it considered expected behaviour?

[hostep]

@arion-p: if there is no issue tracking your problem, I would suggest you open one 😉
This issue here is indeed something different and not related.

@zw1111: your issue also doesn't sound related, you might find more help here: https://github.com/magento/magento2/issues/23450#issuecomment-520134243

[arion-p]

@hostep: there was an issue tracking my problem: this one but it was closed. I may be missing something here but here are the facts:

• @duckchip posted this https://magento.stackexchange.com/a/283837/39637 in response to this thread in magento stackexchange: https://magento.stackexchange.com/questions/281723/magento-2-bulk-actions-not-starting
• Then he created this ticket and the related PR.
• The PR was abandoned and a different fix was suggested
• Then this ticket was closed without any other ticket tracking the issue and without any fix (either the original or the second one) being applied.

The end result is that the issue described in the 1st post is not going to be fixed because there is no longer an open ticket about it.

btw @zw1111's issue is also exactly the same as the one described in the 1st post (and in the stack exchange thread mentioned above as well as the this thread in magento community: https://community.magento.com/t5/Magento-2-x-Admin-Configuration/Bulk-Actions-issue/td-p/138226)

[hostep]

@arion-p: I can't reproduce the issue from stackexchange or the community forums on a vanilla 2.3.2 installation with a working cronjob and working message queue consumers.

This issue here is for custom written bulk actions, not for the ones included in Magento by default.

So what I suspect is that something is preventing the message queue consumers to spawn somehow, can you read https://github.com/magento/magento2/issues/23450#issuecomment-520134243 and do some debugging and try to figure out why they aren't spawning in your case?

Helping in finding the cause might get us closer to figuring out how to solve it for everybody, thanks! 🙂

[arion-p]

In my case cron and the consumers are running.
What I did is similar to the stack exchange issue:

• I selected some products in the product grid
• in the actions menu selected 'update'
• updated an attribute and clicked 'save'

This resulted in an entry added in magento_bulk but nothing in magento_operations. A message is also added in queue_message.
Note that I am not using RabbitMQ.

Looking at Magento/Catalog/Controller/Adminhtml/Product/Action/Attribute/Save::publish it uses:

$this->bulkManagement->scheduleBulk($bulkUuid, $operations, $bulkDescription, $userId);

instead of using MassSchedule as suggested earlier. As a result it should suffer from the same issue.

[arion-p]

Further update:
I cannot reproduce this on clean 2.3.2 with sample data.

On my normal installation it occasionally works but usually it does not. When it works magento_operations is updated much later (probably upon processing the queue message?). I don't know if that makes sense.

I cannot figure out the conditions when it fails and logging is a bit slim. This is clearly something else. I will investigate further and open a new issue if I find something concrete

[ebodor]

The same thing happened to me on 2.3.2. If you are running on Magento Cloud, the support team was able to provide a solution that got them back running for me:

Please update .magento.env.yaml and add these lines (see https://devdocs.magento.com/guides/v2.3/cloud/env/variables-deploy.html#cron_consumers_runner):

deploy:
CRON_CONSUMERS_RUNNER:
cron_run: true
max_messages: 1000
consumers: []
Then commit the changes and redeploy.

Sounds like it may be related to the RabbitMQ Message Queues, can follow this for non Magento Cloud environments:

https://devdocs.magento.com/guides/v2.3/config-guide/mq/manage-message-queues.html

[gaiterjones]

Until this problem gets fixed how can I manually remove the stuck bulk update task? The admin notification bar with the task info is very annoying as until the task completes the notification message cannot be dismissed and it hides other more important system notifications.

[Sendurrr]

Until this problem gets fixed how can I manually remove the stuck bulk update task? The admin notification bar with the task info is very annoying as until the task completes the notification message cannot be dismissed and it hides other more important system notifications.

i did it with deleting the entries in the magento_bulk table. Also facing this problem. Cannot bulk edit anything.

[onepack]

Same here. Bulk edit has actually never worked. I'm following all issues but they all get closed without solution. I now fix these bulk edits by working with database queries but this should not be necessary.
The suggested fix by @duckchip by adding the extra line in the publishOperations function did not change the result except it changed the bulk message from 1 in the queue to 0 in the queue.

[niners52]

Also having this issue

[hostep]

Ok guys, can anybody who has "this issue" please open a new ticket with exact steps to reproduce?

"This issue" here was created for writing your own custom bulk schedule jobs for which the documentation wasn't very clear. It was closed by the author because the solution was found for this custom implementation.

"This issue" was not for running into default Magento bulk job problems.
So if somebody can very clearly state in a new issue what steps you should perform to trigger this problem, that would be very much appreciated! 🙂

[daddyhenk]

This worked for me:
https://magento.stackexchange.com/questions/281723/magento-2-bulk-actions-not-starting

[jaimiedijstra]

I applied the solution already linked previous post, https://magento.stackexchange.com/questions/281723/magento-2-bulk-actions-not-starting, edit file: vendor/magento/module-asynchronous-operations/Model/BulkManagement.php adding "$this->entityManager->save($operation);":

private function publishOperations(array $operations)
 {
        $operationsByTopics = [];
        foreach ($operations as $operation) {
            $this->entityManager->save($operation);
            $operationsByTopics[$operation->getTopicName()][] = $operation;
        }
        foreach ($operationsByTopics as $topicName => $operations) {
            $this->publisher->publish($topicName, $operations);
        }
 }

This got bulk changes working for what I needed and I was able to run the same update process from magento's admin (updates to new to, new from, and product status enable).

Now I still have the message of:
Task "Update attributes for 5 selected products": 1 item(s) have been scheduled for update.

This is from the failed bulk product update which appears like it will never run. I found it in the tables below:

and the acknoledged table:

I'm hesitant to dive in and just delete a row, is there any other tables involved or code, and can I simply delete the row from "magento_bulk"?

Realise this topic is closed, but would be handy to have a clear fix here.

[jaimiedijstra]

I applied the solution already linked previous post, https://magento.stackexchange.com/questions/281723/magento-2-bulk-actions-not-starting, edit file: vendor/magento/module-asynchronous-operations/Model/BulkManagement.php adding "$this->entityManager->save($operation);":

private function publishOperations(array $operations)
 {
        $operationsByTopics = [];
        foreach ($operations as $operation) {
            $this->entityManager->save($operation);
            $operationsByTopics[$operation->getTopicName()][] = $operation;
        }
        foreach ($operationsByTopics as $topicName => $operations) {
            $this->publisher->publish($topicName, $operations);
        }
 }

This got bulk changes working for what I needed and I was able to run the same update process from magento's admin (updates to new to, new from, and product status enable).

Now I still have the message of:
Task "Update attributes for 5 selected products": 1 item(s) have been scheduled for update.

This is from the failed bulk product update which appears like it will never run. I found it in the tables below:

and the acknoledged table:

I'm hesitant to dive in and just delete a row, is there any other tables involved or code, and can I simply delete the row from "magento_bulk"?

Realise this topic is closed, but would be handy to have a clear fix here.

Well, interestingly for me this just cleared up by itself 26 hours later in the back end, but the two database tables remain the same, apparently out of sync?

[davidrthomson]

I am still getting this issue on 2.3.1. My cron is configured ok server side and crons are running. The queued jobs just never start. Did anyone find a definitive cause/fix for this issue?

I triggered some attribute updates and these are correctly listed in the bulk log screen but they stay on pending. I removed legacy entries from the magento_bulk table.

Server side I ran php bin/magento queue:consumer:start product_action_attribute.update and then grepped out for the process and it was listed so looked to be being triggered ok.

Any help would be good.

[hostep]

@davidrthomson: make sure your app/etc/env.php file doesn't have the cron_run set to false.

Also: since you are using Magento 2.3.1, you shouldn't have the product_action_attribute.update consumer, it was introduced in Magento 2.3.2, updating product attributes using queues was not available yet in 2.3.1 and should just work as is, so I'm a bit confused here...
Running bin/magento queue:consumers:list should show which consumers you can use.

If you are running Magento 2.3.2 or higher, make sure the app/etc/env.php file does not only have a couple of consumers configured to run, if you leave those consumers empty, it should execute all of them. More info in the docs.

Also, if you are talking about the default consumers Magento ships with which aren't running automatically, then please follow https://github.com/magento/magento2/issues/23450 which contains some useful hints in various comments.
This issue here was about implementing a custom bulk schedule by yourself in a custom module, so you are probably in the incorrect thread here.

[VivekShingala]

@davidrthomson: make sure your app/etc/env.php file doesn't have the cron_run set to false.

Also: since you are using Magento 2.3.1, you shouldn't have the product_action_attribute.update consumer, it was introduced in Magento 2.3.2, updating product attributes using queues was not available yet in 2.3.1 and should just work as is, so I'm a bit confused here...
Running bin/magento queue:consumers:list should show which consumers you can use.

If you are running Magento 2.3.2 or higher, make sure the app/etc/env.php file does not only have a couple of consumers configured to run, if you leave those consumers empty, it should execute all of them. More info in the docs.

Also, if you are talking about the default consumers Magento ships with which aren't running automatically, then please follow #23450 which contains some useful hints in various comments.
This issue here was about implementing a custom bulk schedule by yourself in a custom module, so you are probably in the incorrect thread here.

This worked for me

[vipin12117]

Facing issue in 2.3.4, was it fixed ?

[Anantkprajapati]

Hello Guys,
Still This is not fix and facing same issue in Magento2.4 also.

[hostep]

@Anantkprajapati: if you are building your custom bulk scheduling system (which this issue is about), then the answer was already given above.

If you are having problems with the default queues of Magento, you need to either search for other issues, or create a new issue where you provide steps of how to reproduce the problem you are running into (For Magento 2.4.0, there are some known problems: https://github.com/magento/magento2/issues/29718 & https://github.com/magento/magento2/issues/29797 for example)

[forcecodema]

The problem with Bulk Actions is still present in 2.4
If you try to bulk change product attributes it doesn't work - it creates new entry in Bulk Actions Log but doesn't create operation units. Solution given above by @duckchip with additional line of code:

$this->entityManager->save($operation);

solves the problem for admin panel, but introduces another with bulk actions via REST API asynchronous bulk operations. Batch job gets created with operations, but never finishes. Operations seem to be executed, but the job stays "In progress" forever.

[hostep]

@forcecodema, https://github.com/magento/magento2/pull/29814 should fix the status of job issues in 2.4.0 if I understand it correctly (see https://github.com/magento/magento2/issues/29718 or https://github.com/magento/magento2/issues/29797 for more info).