Cms: CraftConnector getEmails TypeError

[juban]

Description

Hi,

I'm trying to setup acceptante tests using Codeception and email part of craft\test\Craft module.

However, when using the seeEmailIsSent assertion, the following error is thrown:
[TypeError] Return value of craft\test\CraftConnector::getEmails() must be of the type array, null returned

For some reason, the return statement doesn't seem to return the expected array.

I've tried various mailer configurations with no success.

Steps to reproduce

Use $I->seeEmailIsSent assertion in acceptance test

Additional info

• Craft version: 3.4.24
• PHP version: 7.4
• Database driver & version: MariaDB 10.2
• Plugins & versions: Elements Panel 1.0.0, Redactor 2.6.1, Retcon 2.1.0, SEO 3.6.5.1, Super Table 2.5.1, Twigpack 1.2.2, Typography 1.1.38

[angrybrad]

Potentially related to https://github.com/craftcms/cms/pull/6263, which will be included in the next release. Want to pull that in early and see if it's related?

[juban]

Hi Brad. Indeed, looks like it's related because I'm setting dynamically the mailer configuration in order to activate SMTP authentification depending on environment. I'll try the fix and let you know. Thx.

[juban]

Unfortunately, I tried to use the new CraftConnector version and it doesn't change anything for me.

I'm using the following acceptance suite configuration:

# Codeception Test Suite Configuration
#
# Suite for acceptance tests.
# Perform tests in browser using the WebDriver or PhpBrowser.
# If you need both WebDriver and PHPBrowser tests - create a separate suite.

actor: AcceptanceTester
modules:

  enabled:
    - PhpBrowser:
        url: https://some-domain.test/
    - \craft\test\Craft:
        part:
         - init
         - orm
         - email
        cleanup: false
        transaction: false
    - \Helper\Acceptance
  step_decorators: ~

I'm only using parts of craft\test\Craft here. Could this be related to the issue?

[juban]

Hi @angrybrad
FYI, still not working as of Craft 3.4.27.
I totally removed overloading of mailer configuration, but no luck so far.

[angrybrad]

@juban Can you share your codeception.yml file as well as your Cest (and anything you think would help us reproduce) over at [email protected]?

[juban]

@angrybrad Done. I've sent you the whole codeception test folder.
Let me know if you need more informations.
Thank you for your help.

[angrybrad]

@juban I just pushed up a fix for that particular error, but if you try it you'll notice that $I->seeEmailIsSent(1) will still fail because it's returning 0 instead of the 1 you're expecting.

The reason for that is that for acceptance tests, behind the scenes, Codeception is using curl to make entirely separate web requests back to the server (e.g. when it hits $I->click("Register"); that have no context of what the server is actually doing behind the scenes. Think of it in the context of a browser, where if you load a page, the browser has no way to tell if an email was sent during the request.

If you did something like this:

    $I->amOnPage('/');
    Craft::$app->getMailer()
        ->compose()
        ->setTextBody('test')
        ->setTo('[email protected]')
        ->send();

    $I->seeEmailIsSent(1);

It would work because that's all within the same request.

What you might be looking for instead, for this particular case, is a functional test: https://codeception.com/docs/04-FunctionalTests

Now that we’ve written some acceptance tests, functional tests are almost the same, with one major difference: Functional tests don’t require a web server.

Because they don't require a web server.

looks like it's related because I'm setting dynamically the mailer configuration in order to activate SMTP authentification depending on environment.

I'll also note that by doing this, you're effectively overriding Craft's/Yii's/Codeception's mocked mailer implementation

https://github.com/craftcms/cms/blob/develop/src/test/TestMailer.php
https://github.com/craftcms/cms/blob/develop/src/test/CraftConnector.php#L66-L76

So things like $I->seeEmailIsSent(1); won't work natively.

[juban]

Hi @angrybrad,

Thank you so much for your answer and the fix.

I get that acceptance test use a pseudo browser request (that’s the goal). But what’s a bit confusing is for example an assertion like seeRecord works just fine.
Anyway, for seeEmailIsSent, there are no means to persist that email was sent in that scenario.
Functional test is probably the way to go. I got some issues using that because of URLs/routes issues and could not make a login action work properly. Will have to try again :)

Thanks again.

[angrybrad]

But what’s a bit confusing is for example an assertion like seeRecord works just fine.
Anyway, for seeEmailIsSent, there are no means to persist that email was sent in that scenario.

Exactly... seeRecord is persisted to the database, so both requests can see it fine.

seeEmailIsSent is just memoized on the first request.

[brandonkelly]

Craft 3.4.28 is out now with this fix.