Magento2: Executing `grunt refresh` or `grunt exec:all` in production mode, leads to error "Unable to retrieve deployment version of static files from the file system."

[Eddcapone]

Preconditions (*)

1. Magento 2.3.5-p2

Steps to reproduce (*)

1. Make the folder pub your project root
2. Put magento in production mode. php bin/magento deploy:mode:set production
3. Create a theme.
4. Setup grunt.
5. Execute grunt exec:all

Expected result (*)

1. It should compile.

Actual result (*)

1. You will get Unable to retrieve deployment version of static files from the file system. and the command fails and the file pub/static/deployed_version.txt gets deleted.

This error is getting logged:

[2020-08-17 05:25:31] report.CRITICAL: Can not load static content version. [] []
[2020-08-17 05:25:31] report.ERROR: Unable to retrieve deployment version of static files from the file system.
#0 vendor/magento/framework/App/View/Deployment/Version.php(67): Magento\Framework\App\View\Deployment\Version->readValue('production')
#1 vendor/magento/module-theme/Model/Url/Plugin/Signature.php(87): Magento\Framework\App\View\Deployment\Version->getValue()
#2 vendor/magento/module-theme/Model/Url/Plugin/Signature.php(65): Magento\Theme\Model\Url\Plugin\Signature->renderUrlSignature()
#3 vendor/magento/framework/Interception/Interceptor.php(146): Magento\Theme\Model\Url\Plugin\Signature->afterGetBaseUrl(Object(Magento\Store\Model\Store\Interceptor), 'https://relaunc...', 'static', $
#4 vendor/magento/framework/Interception/Interceptor.php(153): Magento\Store\Model\Store\Interceptor->Magento\Framework\Interception\{closure}('static', false)
#5 generated/code/Magento/Store/Model/Store/Interceptor.php(143): Magento\Store\Model\Store\Interceptor->___callPlugins('getBaseUrl', Array, Array)
#6 vendor/magento/framework/Url.php(482): Magento\Store\Model\Store\Interceptor->getBaseUrl('static', false)
#7 vendor/magento/framework/View/Asset/Repository.php(284): Magento\Framework\Url->getBaseUrl(Array)
#8 vendor/magento/framework/View/Asset/Repository.php(233): Magento\Framework\View\Asset\Repository->getFallbackContext('static', NULL, 'frontend', 'company/fresh', 'de_DE')
#9 vendor/magento/module-developer/Console/Command/SourceThemeDeployCommand.php(172): Magento\Framework\View\Asset\Repository->createAsset('css/styles-m.le...', Array)
#10 vendor/symfony/console/Command/Command.php(255): Magento\Developer\Console\Command\SourceThemeDeployCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Con$
#11 vendor/magento/framework/Interception/Interceptor.php(58): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output$
#12 vendor/magento/framework/Interception/Interceptor.php(138): Magento\Developer\Console\Command\SourceThemeDeployCommand\Interceptor->___callParent('run', Array)
#13 vendor/magento/framework/Interception/Interceptor.php(153): Magento\Developer\Console\Command\SourceThemeDeployCommand\Interceptor->Magento\Framework\Interception\{closure}(Object(Symfony\Componen$
#14 generated/code/Magento/Developer/Console/Command/SourceThemeDeployCommand/Interceptor.php(104): Magento\Developer\Console\Command\SourceThemeDeployCommand\Interceptor->___callPlugins('run', Array,$
#15 vendor/symfony/console/Application.php(1000): Magento\Developer\Console\Command\SourceThemeDeployCommand\Interceptor->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Componen$
#16 vendor/symfony/console/Application.php(271): Symfony\Component\Console\Application->doRunCommand(Object(Magento\Developer\Console\Command\SourceThemeDeployCommand\Interceptor), Object(Symfony\Comp$
#17 vendor/magento/framework/Console/Cli.php(115): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput$
#18 vendor/symfony/console/Application.php(147): Magento\Framework\Console\Cli->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 bin/magento(23): Symfony\Component\Console\Application->run()
#20 {main} [] []

---

Please provide Severity assessment for the Issue as Reporter. This information will help during Confirmation and Issue triage processes.

• [x] Severity: S0 _- Affects critical data or functionality and leaves users without workaround._
• [ ] Severity: S1 _- Affects critical data or functionality and forces users to employ a workaround._
• [ ] Severity: S2 _- Affects non-critical data or functionality and forces users to employ a workaround._
• [ ] Severity: S3 _- Affects non-critical data or functionality and does not force users to employ a workaround._
• [ ] Severity: S4 _- Affects aesthetics, professional look and feel, “quality” or “usability”._

[m2-assistant[bot]]

Hi @Eddcapone. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

  :warning: According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

:clock10: You can find the schedule on the Magento Community Calendar page.

:telephone_receiver: The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

:movie_camera: You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

:pencil2: Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[Eddcapone]

I figured out that grunt exec calls this code: grunt --force clean:company_fresh. This calls the script dev/tools/grunt/configs/clean.js.

It looks like this script is deleting the file pub/static/deployed_content.txt and so the errors Unable to retrieve deployment version of static files from the file system and Can not load static content version. are showing.

So I modified the script and removed all lines where "deployedVersion" is showing.

Original:

/**
 * Copyright © Magento, Inc. All rights reserved.
 * See COPYING.txt for license details.
 */

'use strict';

var themes = require('../tools/files-router').get('themes'),
    _      = require('underscore');

var themeOptions = {};

_.each(themes, function(theme, name) {
    themeOptions[name] = {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                    "<%= path.tmp %>/cache/**/*",
                    "<%= combo.autopath(\""+name+"\", path.pub ) %>**/*",
                    "<%= combo.autopath(\""+name+"\", path.tmpLess) %>**/*",
                    "<%= combo.autopath(\""+name+"\", path.tmpSource) %>**/*",
                    "<%= path.deployedVersion %>"
                ]
            }
        ]
    };
});

var cleanOptions = {
    "var": {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                    "<%= path.tmp %>/cache/**/*",
                    "<%= path.tmp %>/generation/**/*",
                    "<%= path.tmp %>/log/**/*",
                    "<%= path.tmp %>/maps/**/*",
                    "<%= path.tmp %>/page_cache/**/*",
                    "<%= path.tmp %>/tmp/**/*",
                    "<%= path.tmp %>/view/**/*",
                    "<%= path.tmp %>/view_preprocessed/**/*"
                ]
            }
        ]
    },
    "pub": {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                "dot": true,
                "src": [
                    "<%= path.pub %>frontend/**/*",
                    "<%= path.pub %>adminhtml/**/*",
                    "<%= path.deployedVersion %>"
                ]
            }
        ]
    },
    "styles": {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                    "<%= path.tmp %>/view_preprocessed/**/*",
                    "<%= path.tmp %>/cache/**/*",
                    "<%= path.pub %>frontend/**/*.less",
                    "<%= path.pub %>frontend/**/*.css",
                    "<%= path.pub %>adminhtml/**/*.less",
                    "<%= path.pub %>adminhtml/**/*.css",
                    "<%= path.deployedVersion %>"
                ]
            }
        ]
    },
    "markup": {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                    "<%= path.tmp %>/cache/**/*",
                    "<%= path.tmp %>/generation/**/*",
                    "<%= path.tmp %>/view_preprocessed/html/**/*",
                    "<%= path.tmp %>/page_cache/**/*"
                ]
            }
        ]
    },
    "js": {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                    "<%= path.pub %>**/*.js",
                    "<%= path.pub %>**/*.html",
                    "<%= path.pub %>_requirejs/**/*",
                    "<%= path.deployedVersion %>"
                ]
            }
        ]
    }
};

module.exports = _.extend(cleanOptions, themeOptions);

Fix:

/**
 * Copyright © Magento, Inc. All rights reserved.
 * See COPYING.txt for license details.
 */

'use strict';

var themes = require('../tools/files-router').get('themes'),
    _      = require('underscore');

var themeOptions = {};

_.each(themes, function(theme, name) {
    themeOptions[name] = {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                    "<%= path.tmp %>/cache/**/*",
                    "<%= combo.autopath(\""+name+"\", path.pub ) %>**/*",
                    "<%= combo.autopath(\""+name+"\", path.tmpLess) %>**/*",
                    "<%= combo.autopath(\""+name+"\", path.tmpSource) %>**/*",
                ]
            }
        ]
    };
});

var cleanOptions = {
    "var": {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                    "<%= path.tmp %>/cache/**/*",
                    "<%= path.tmp %>/generation/**/*",
                    "<%= path.tmp %>/log/**/*",
                    "<%= path.tmp %>/maps/**/*",
                    "<%= path.tmp %>/page_cache/**/*",
                    "<%= path.tmp %>/tmp/**/*",
                    "<%= path.tmp %>/view/**/*",
                    "<%= path.tmp %>/view_preprocessed/**/*"
                ]
            }
        ]
    },
    "pub": {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                "dot": true,
                "src": [
                    "<%= path.pub %>frontend/**/*",
                    "<%= path.pub %>adminhtml/**/*",
                ]
            }
        ]
    },
    "styles": {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                    "<%= path.tmp %>/view_preprocessed/**/*",
                    "<%= path.tmp %>/cache/**/*",
                    "<%= path.pub %>frontend/**/*.less",
                    "<%= path.pub %>frontend/**/*.css",
                    "<%= path.pub %>adminhtml/**/*.less",
                    "<%= path.pub %>adminhtml/**/*.css",
                ]
            }
        ]
    },
    "markup": {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                    "<%= path.tmp %>/cache/**/*",
                    "<%= path.tmp %>/generation/**/*",
                    "<%= path.tmp %>/view_preprocessed/html/**/*",
                    "<%= path.tmp %>/page_cache/**/*"
                ]
            }
        ]
    },
    "js": {
        "force": true,
        "files": [
            {
                "force": true,
                "dot": true,
                "src": [
                    "<%= path.pub %>**/*.js",
                    "<%= path.pub %>**/*.html",
                    "<%= path.pub %>_requirejs/**/*",
                ]
            }
        ]
    }
};

module.exports = _.extend(cleanOptions, themeOptions);

Now if I call grunt refresh && php bin/magento setup:static-content:deploy && php bin/magento setup:static-content:deploy de_DE then it works. So why is the clean script deleting the file by default and causing the bug? It only happens in production mode.

[luccame]

From: https://devdocs.magento.com/guides/v2.4/frontend-dev-guide/tools/using_grunt.html

Prerequisites

Make sure that you set your Magento application to the developer or default mode.

[Eddcapone]

@luccame Thx. Then how are we supposed to compile our theme in production mode?
Are we supposed to compile on a develop instance and deploy the generated content to the production server?
How is the normal workflow supposed to look like?

[ihor-sviziev]

@Eddcapone
for production you should use bin/magento setup:static-content:deploy command. More info here https://devdocs.magento.com/guides/v2.4/config-guide/cli/config-cli-subcommands-static-view.html#config-cli-subcommands-staticview

[mrtuvn]

i believe in production we should not use grunt . Every changes need to update by ultimate commands di compile and deploy static

[Eddcapone]

@Eddcapone
for production you should use bin/magento setup:static-content:deploy command. More info here https://devdocs.magento.com/guides/v2.4/config-guide/cli/config-cli-subcommands-static-view.html#config-cli-subcommands-staticview

But this does not compile your themes LESS files, so it makes no sense?

[Eddcapone]

i believe in production we should not use grunt . Every changes need to update by ultimate commands di compile and deploy static

The commands bin/magento setup:di:compile and bin/magento setup:static-content:deploy are not compiling your themes LESS files, so how are we supposed to compile the Theme without using grunt?

[ihor-sviziev]

It is done on php. Yes this is not logical and much slower than it could
be, but that’s what we have in magento

On Tue, 18 Aug 2020 at 11:43 Eddcapone [e-mail] wrote:

>
>
>
>

i believe in production we should not use grunt . Every changes need to
update by ultimate commands di compile and deploy static

The commands bin/magento setup:di:compile and bin/magento
setup:static-content:deploy are not compiling your themes LESS files, so
how are we supposed to compile the Theme without using grunt?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/magento/magento2/issues/29570#issuecomment-675346407,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAOJOUK7N2L4BGQGIBRSEG3SBI5JZANCNFSM4QBKB63Q
.

[hostep]

@Eddcapone: you can use https://github.com/baldwin-agency/magento2-module-less-js-compiler which uses nodejs to compile less code instead of php, which - in the early days of Magento was much more performant, but recently the scales have tipped. But the most important feature of that module, is that the outputted css code should be 100% the same between local development (using grunt) and production (using setup:static-content:deploy) which is not the case if you use the build-in php less compiler which Magento ships with.

[Eddcapone]

i believe in production we should not use grunt . Every changes need to update by ultimate commands di compile and deploy static

The commands bin/magento setup:di:compile and bin/magento setup:static-content:deploy are not compiling your themes LESS files, so how are we supposed to compile the Theme without using grunt?

I just tested it and php bin/magento maintenance:enable && php bin/magento setup:upgrade && php bin/magento setup:di:compile && php bin/magento setup:static-content:deploy && php bin/magento setup:static-content:deploy de_DE && php bin/magento maintenance:disable actually does compile your Theme. It works.
But I still don't understand why grunt only works for default/developer mode? It just makes no sense for me.
I applied my fix and I will use grunt in production, until someone gives a valid reason why we should not use grunt in production.

[ihor-sviziev]

Php compiler gives a bit different result time to time, that’s the main
reason - you might see the issues that no one else see

On Tue, 18 Aug 2020 at 13:09 Eddcapone [e-mail] wrote:

>
>
>
>
>
>

i believe in production we should not use grunt . Every changes need to
update by ultimate commands di compile and deploy static

The commands bin/magento setup:di:compile and bin/magento
setup:static-content:deploy are not compiling your themes LESS files, so
how are we supposed to compile the Theme without using grunt?

I just tested it and bin/magento setup:di:compile acutally does compile
your Theme. It works.

But I still don't understand why grunt only works for default/developer
mode? It just makes no sense for me.

I applied my fix and I will use grunt in production, until someone gives a
valid reason why we should not use grunt in production.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/magento/magento2/issues/29570#issuecomment-675389884,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAOJOUIVA2VYTDL4C6M4ZW3SBJHO3ANCNFSM4QBKB63Q
.

[Eddcapone]

Php compiler gives a bit different result time to time, that’s the main reason - you might see the issues that no one else see
…
On Tue, 18 Aug 2020 at 13:09 Eddcapone @.*> wrote: i believe in production we should not use grunt . Every changes need to update by ultimate commands di compile and deploy static The commands bin/magento setup:di:compile and bin/magento setup:static-content:deploy are not compiling your themes LESS files, so how are we supposed to compile the Theme without using grunt? I just tested it and bin/magento setup:di:compile acutally does compile your Theme. It works. But I still don't understand why grunt only works for default/developer mode? It just makes no sense for me. I applied my fix and I will use grunt in production, until someone gives a valid reason why we should not use grunt in production. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#29570 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOJOUIVA2VYTDL4C6M4ZW3SBJHO3ANCNFSM4QBKB63Q .

So grunt uses the php compiler? Then which compiler is used by setup:di:compile? Is it working better than grunt compiler?

[hostep]

@Eddcapone

1) setup:di:compile has nothing to do with generating static assets
2) setup:upgrade also implicitly executes setup:static-content:deploy (unless you specify the --keep-generated flag)
3) the main reason for wanting to use setup:static-content:deploy on production is because it does a lot more than simply compiling less code. It's also merging/bundling/minifying css and javascript and html if you have those settings enabled. It's also responsible for generating javascript translations and might do some more tasks. I don't know if grunt can also do all these tasks by default in Magento? I've discussed this before in another ticket.

If you want steps for a production deploy, here are the basic ones which we perform:

composer install --optimize-autoloader --prefer-dist --no-dev
npm install --production # only needed if you need nodejs dependencies on your build/production server
php bin/magento deploy:mode:set --skip-compilation production
php bin/magento setup:di:compile
composer dump-autoload --optimize --no-dev
touch pub/static/deployed_version.txt # to work around an old bug https://github.com/magento/magento2/issues/6426 which is not clear if it was fixed or not yet
php bin/magento setup:static-content:deploy --theme=Custom/theme nl_NL fr_FR en_US ... # depending on which languages you need in the frontend
php bin/magento setup:static-content:deploy --theme=Magento/backend en_US ... # depending on which languages you need in the backend
php bin/magento setup:upgrade --keep-generated

These are just basic steps, we do a lot of complex other things due to how our workflow is setup, but hopefully this can put you in the correct direction 🙂

[Eddcapone]

I made a color change in a less file of my theme and then executed php bin/magento setup:static-content:deploy && php bin/magento cache:flush but the change was not visible after it.

I always have to call the whole set of commands to make it work:

php bin/magento maintenance:enable && php bin/magento setup:upgrade && php bin/magento setup:di:compile && php bin/magento setup:static-content:deploy && php bin/magento setup:static-content:deploy de_DE && php bin/magento maintenance:disable

[hostep]

That's because you start from dirty directories, Magento uses previously generated files in their SCD procedures and doesn't always overwrite them. If you start from a clean environment (like we always do with every single new deploy to production) it should work.

In your case, please try removing these files before running setup:static-content:deploy:

• rm -R var/view_preprocessed/*
• rm -R pub/static/*

(the * is important and intentionally, so you won't accidentally remove the .htaccess files in those directories which is important to keep if you use Apache as your webserver)

[Eddcapone]

I see thank you. I just tried php bin/magento maintenance:enable && grunt clean && php bin/magento setup:static-content:deploy && php bin/magento setup:static-content:deploy de_DE && php bin/magento maintenance:disable instead.

That worked.