Magento2: Return path e-mail variable system/smtp/return_path_email not used anywhere?

[flecxie]

Trying to configure the return path e-mail as described on https://docs.nexcess.net/article/how-to-set-the-return-path-email-in-magento-2.html on Magento CE 2.1.0 but failing to do so.

I can see that both system/smtp/set_return_path and system/smtp/return_path_email are configured correctly in my core_config_data table but when I look for these variables in the source I don't seem them used anywhere. The only references I could find for return_path_email are these:

vendor/magento/module-email/Model/Template.php:    const XML_PATH_SENDING_RETURN_PATH_EMAIL = 'system/smtp/return_path_email';
vendor/magento/module-backend/etc/adminhtml/system.xml:                <field id="return_path_email" translate="label" type="text" sortOrder="80" showInDefault="1" showInWebsite="0" showInStore="0">

Const XML_PATH_SENDING_RETURN_PATH_EMAIL doesn't seem to be used anywhere either. Could someone please confirm that these settings are in fact used in Magento CE 2.1.0 & if so point me to the correct file for debugging?

Preconditions

1. Magento CE 2.1.0 on Debian jessie

   Steps to reproduce

2. Configure return path as described on https://docs.nexcess.net/article/how-to-set-the-return-path-email-in-magento-2.html

3. Place order

   Expected result

4. Return path being set correctly

   Actual result

5. Return path remains [email protected]

[SchumacherFM]

We have the same problem in 2.1.0 and some customers (e.g. those with a GMX account) don't get any emails.

Magento does not set the return path because the the configuration will be not be used because it's not implemented.

Will this fix be included in 2.1.1 ?

Edit: Work around in php.ini:

sendmail_path = "sendmail -t -i -F [email protected] -f [email protected]"

[SchumacherFM]

This bug or missing feature has not been fixed/added in 2.1.1

[MarkB42]

Also having this issue and unable to get emails through.
Mage 2.1.1

[MarkB42]

I have spent some time looking at this and based on the following I think Magento is not setting the php mail options anymore because the return path refuses to change.

1) Hard coded a call to $this->setReturnPath("[email protected]"); in the constructor of vendor / magento / module-email / Model / Template.php

2) Call setReturnPath() when the setReplyTo() method is called

3) Finally thought maybe the setReturnPath() is being called somewhere and it's just being set wrong so I modified the setReturnPath() method to always set the return path to [email protected] and still no changes.

Can this get bumped up the list? We literally cannot use Magento in a multi store mode as the emails are getting classified as spam, predominately by SPF.

Mage 2.1.1
Ubuntu
PHP 7.0.11

[PascalBrouwers]

Same for 2.1.2

[paales]

@victor-v-rad Can you please check this issue? :)

[PascalBrouwers]

I see there is a function setReturnPath(). Maybe we can use that in MagentoSalesModelOrderEmailSenderBuilder->send() and then do $this->transportBuilder->setReturnPath() ?

[heldchen]

ran into this issue as well today.

[ktruehl]

The problem lies with the Zend_Mail_Transport_Sendmail class. It requires the return path as a parameter in its constructor (see https://framework.zend.com/manual/1.12/en/zend.mail.introduction.html#zend.mail.introduction.sendmail). So using the setReturnPath() method won't work.

Now the return path must be set in the constructor of \Magento\Framework\Mail\Transport. The constructor of Transport has a paramter called $parameters. One option would be to use dependency injection to inject the necessary parameter. However, that does not allow specifying a different return path for each website or store. Admittedly that is currently not possible anyway, since the system/smtp/set_return_path and system/smtp/return_path_email both have global scope. However, in a multi-website environment it is easy to conceive of a need to have different return paths per store.

So, I suggest getting the necessary settings in the constructor of \Magento\Framework\Mail\Transport and handing them to the parent constructor. The following patch does just that:
fix-setting-return-path-according-to-settings.diff.zip

[dooblem]

Same here.
To my mind this is an important issue !
Many store are sending mails that are classified as spam without knowing it...

[schmengler]

Looks like the feature has been removed intentionally (see: https://github.com/magento/magento2/blob/b1e56ed3b6f13f1ab780d8eddc33fc04cb6ecf75/dev/tests/static/testsuite/Magento/Test/Legacy/_files/obsolete_constants.php#L189) but they missed removing the configuration field.

On a plus side, Magento 2 should be unaffected by the current exploit ZF2016-04

[tecjam]

I agree with @dooblem
I can't see any reason why it should be removed. Having www-data@yourserver as your Return-Path in your email header makes it pretty certain it will be classified as spam by the majority of email providers.
Issue still persists in 2.1.3

[heldchen]

removing it is a bad idea. our larger shop owners have dedicated return-path and reply-to addresses that are screened by their staff - set on the store level so that return mails from the german storeview can be screened by the german team, french by the frenchs etc.

using the common base os sender email is not an option.

[bh-ref]

issue is still present in 2.1.4

[Noewt]

Used @ktruehl solution, but seems it stopped working in 2.1.6. It worked perfectly for all e-mails in 2.1.5
Currently all e-mails have the same Return-path which is set on server side. Magento configuration to set a specific Return-path is not working.

[ktruehl]

Good to know, @Noewt. We haven't upgraded to 2.1.6 yet. Sounds like fun. Once we upgrade and I have to port my patch, I will post the new patch here.

[justanewbie00]

So I upgraded from 2.0.3 to magento 2.1.6, it seems that the customer is no longer receiving email. Like New Order or Registration Email.

[matin73]

same problem in 2.1.7 - any solution yet?

[Joker2099]

Bump. Did anyone find a workaround?

[Joker2099]

Anyway, I went ahead and added this line
$parameters .= "-f [email protected]";
Right before:
parent::__construct($parameters);
in Magento/Framework/Mail/Transport.php

It's really dirty but does the trick for now.

[Noewt]

That dirty trick was already provided ktruehl, but is also currently not working here in 2.1.8. All transactional e-mails still have the set return-path ignored.

[Joker2099]

@Noewt Sorry, yes I used @ktruehl's way to fix it on 2.1.4. I also read something about that the return-path was removed in one of the versions completely. I may be wrong though.

[matin73]

same problem in magento 2.2 - realy poor in year 2017 that magento is not even supporting sending emails via smtp!!!

[magento-engcom-team]

@flecxie, thank you for your report.
We've created internal ticket(s) MAGETWO-82311 to track progress on the issue.

[dverkade]

I was working on this issue for the #SQUASHTOBERFEST and found out the following.

• Setting the return path to sendmail should be done by adding "-f [email protected]" to the mail() function parameters parameter.
• The Transport model can be changed to achieve this.

The problem I found is that the included Zend Framework 1 package has changed. In the Zend/Mail/Transport/Sendmail.php class the _sendMail() function has been altered and does not check if the parameter variable has been set. So this issue is not fixable right now, because we need to changed the included Zend Framework that comes with Magento. Unfortunately, this is a seperate package and can not be changed in the Magento Github respository. @magento-engcom-team please share your thoughts on how to go forward with this issue.

Sendmail class code in Magento:

    public function _sendMail()
    {
        set_error_handler(array($this, '_handleMailErrors'));
        $result = mail(
            $this->recipients,
            $this->_mail->getSubject(),
            $this->body,
            $this->header);
        restore_error_handler();

        if ($this->_errstr !== null || !$result) {
            /**
             * @see Zend_Mail_Transport_Exception
             */
            #require_once 'Zend/Mail/Transport/Exception.php';
            throw new Zend_Mail_Transport_Exception('Unable to send mail. ' . $this->_errstr);
        }
    }

Sendmail class code in Zend Framework:

    public function _sendMail()
    {
        if ($this->parameters === null) {
            set_error_handler(array($this, '_handleMailErrors'));
            $result = mail(
                $this->recipients,
                $this->_mail->getSubject(),
                $this->body,
                $this->header);
            restore_error_handler();
        } else {
            if(!is_string($this->parameters)) {
                /**
                 * @see Zend_Mail_Transport_Exception
                 *
                 * Exception is thrown here because
                 * $parameters is a public property
                 */
                require_once 'Zend/Mail/Transport/Exception.php';
                throw new Zend_Mail_Transport_Exception(
                    'Parameters were set but are not a string'
                );
            }
            set_error_handler(array($this, '_handleMailErrors'));
            $result = mail(
                $this->recipients,
                $this->_mail->getSubject(),
                $this->body,
                $this->header,
                $this->parameters);
            restore_error_handler();
        }
        if ($this->_errstr !== null || !$result) {
            /**
             * @see Zend_Mail_Transport_Exception
             */
            require_once 'Zend/Mail/Transport/Exception.php';
            throw new Zend_Mail_Transport_Exception('Unable to send mail. ' . $this->_errstr);
        }
    }

[Will2JCommerce]

Still an issue in Magento OS 2.2.

Despite above workarounds my return path value is still not updating and is the following:

[email protected]

[drashmk]

Looks like issue in ZendMailTransportSendmail class. When return path is passed in the constructor, -f'[return_email]' is added as parameter but also once again -f'[from_email]'is added in prepareParameters method which is called in send method. Later PHP is sending both of these parameters as command line parameters so sendmail in mailHandler method, ex. /usr/sbin/sendmail -f'[return_email]' -f'[from_email]' ....
As a workaround I would recommend using 3rd party smtp extension and send emails trough email service for high volume shops to avoid getting emails classified as spam.

[jzembala]

Magento based on forked zendframework/zf1 with some fixes and here is a commit which prevents the possibility of adding parameters to mail() funciton.
https://github.com/magento/zf1/commit/57a60da47c5de32c6d61bcb385c5aea7f8447779

[PieterCappelle]

Any update on this one? Having same problem. :)

[gewaechshaus]

@PieterCappelle - you can finetune postfix if you are using it...

• Create a file canonical in /etc/postfix
• Paste in www-data [email protected]
• Do a #postmap hash:/etc/postfix/canonical
• And finaly place canonical_maps = hash:/etc/postfix/canonical in main.cf
  

  - Restart or reload the service

You should also set the sendmail_path to /usr/sbin/sendmail -t -i -f'[email protected]' for php.
You can check the setting in your phpinfo file or use the console php -i | grep sendmail to find the currently applied setting. If you are using php-fpm, don't forget to set it also in the fpm related php.ini file. Recheck if the settings were applied successfully.

[lfluvisotto]

@magento-engcom-team I think this pull request in zend framework will fix the issue:

https://github.com/magento/zf1/pull/15

[jasondobry]

Has anyone confirmed this is indeed working as expected?

[darshan-khatri-16]

i am working on it #dmcindia1

[m2-assistant[bot]]

Hi @darshan-khatri-16. Thank you for working on this issue.
Looks like this issue is already verified and confirmed. But if you want to validate it one more time, please, go though the following instruction:

• [ ] 1. Add/Edit Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 2. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento-engcom-team give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!
  

• [ ] 3. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento-engcom-team give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• [ ] 4. If the issue is not relevant or is not reproducible any more, feel free to close it.

[m2-assistant[bot]]

Hi @Krielkip. Thank you for working on this issue.
Looks like this issue is already verified and confirmed. But if you want to validate it one more time, please, go though the following instruction:

• [x] 1. Add/Edit Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [x] 2. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!

• [ ] 3. If the issue is not relevant or is not reproducible any more, feel free to close it.

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[Krielkip]

Reproduced on 2.4-develop;

Since https://github.com/magento-engcom/php-7.2-support/pull/136 is not been merged. magento/zendframework1 is still being used?

Instead of waiting for the new solution, let's fix the "old" code and help many people at this moment 😄

[ghost]

:white_check_mark: Confirmed by @Krielkip
Thank you for verifying the issue! :+1: Your confirmation will help us to acknowledge and process this report.

[Krielkip]

😱 something with fix commit and close ticket..

[Krielkip]

Can be closed if this is merged: https://github.com/magento/zf1/pull/26

[m2-assistant[bot]]

Hi @Krielkip. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [x] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• [x] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• [x] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [x] 4. Verify that the issue is reproducible on 2.4-develop branch
  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[Krielkip]

https://github.com/magento/zf1/pull/26
This is still open for this issue. 😺