Yii2: Codeception test fails out of the box

[evil1]

Install fresh yii2 advanced template via composer
Install codeception
Execute codeception run, and get the following error:

cept run
Codeception PHP Testing Framework v2.2.8
Powered by PHPUnit 4.8.32 by Sebastian Bergmann and contributors.

Common\tests.unit Tests (3) ---------------------------------------------------------------------------------------
✔ LoginFormTest: Login no user (0.04s)
✔ LoginFormTest: Login wrong password (0.53s)

✔ LoginFormTest: Login correct (0.53s)

Frontend\tests.functional Tests (12) ------------------------------------------------------------------------------
E AboutCest: Check about (0.01s)

• ContactCest: Check contact

  PHP User Error 'yii\base\ErrorException' with message 'Exception (Invalid Configuration) 'yii\base\InvalidConfigException' with message 'The directory does not exist: ./assets'

in /var/www/ott/vendor/yiisoft/yii2/web/AssetManager.php:211

| Q | A
| ---------------- | ---
| Yii version | 2.0.10
| PHP version | 5.4.16
| Operating system | CentOS 7.3

[sztyvny]

Have you tried
php init
in your main project directory?
It looks like you don't have your assets directory. Is your project running correctly in browser?
And maybe paste your ContactCest.php file here.

[evil1]

Here is ContactCept contents:

<?php
namespace frontend\tests\functional;

use frontend\tests\FunctionalTester;

/* @var $scenario \Codeception\Scenario */

class ContactCest
{
    public function _before(FunctionalTester $I)
    {
        $I->amOnPage(['site/contact']);
    }

    public function checkContact(FunctionalTester $I)
    {
        $I->see('Contact', 'h1');
    }

    public function checkContactSubmitNoData(FunctionalTester $I)
    {
        $I->submitForm('#contact-form', []);
        $I->see('Contact', 'h1');
        $I->seeValidationError('Name cannot be blank');
        $I->seeValidationError('Email cannot be blank');
        $I->seeValidationError('Subject cannot be blank');
        $I->seeValidationError('Body cannot be blank');
        $I->seeValidationError('The verification code is incorrect');
    }

    public function checkContactSubmitNotCorrectEmail(FunctionalTester $I)
    {
        $I->submitForm('#contact-form', [
            'ContactForm[name]' => 'tester',
            'ContactForm[email]' => 'tester.email',
            'ContactForm[subject]' => 'test subject',
            'ContactForm[body]' => 'test content',
            'ContactForm[verifyCode]' => 'testme',
        ]);
        $I->seeValidationError('Email is not a valid email address.');
        $I->dontSeeValidationError('Name cannot be blank');
        $I->dontSeeValidationError('Subject cannot be blank');
        $I->dontSeeValidationError('Body cannot be blank');
        $I->dontSeeValidationError('The verification code is incorrect');
    }

    public function checkContactSubmitCorrectData(FunctionalTester $I)
    {
        $I->submitForm('#contact-form', [
            'ContactForm[name]' => 'tester',
            'ContactForm[email]' => '[email protected]',
            'ContactForm[subject]' => 'test subject',
            'ContactForm[body]' => 'test content',
            'ContactForm[verifyCode]' => 'testme',
        ]);
        $I->seeEmailIsSent();
        $I->see('Thank you for contacting us. We will respond to you as soon as possible.');
    }
}

I did run init and initialized the development environment.

Project is running in browser well using both apache webserver and yii serve command, which starts PHP's internal web-server.
But php serve by default cannot detect the docroot, so I have to manually pass it via --docroot parameter.
By default codeception configuration uses localhost:8080 url which (I guess) means that codeception uses php serve command during testing?
Anyway, I tried both - chaging the url to the one, served by apache and the defaul one.
The error is still there.

[samdark]

Try creating assets directory.

[evil1]

Where? It is present under frontend/assets containing php bundles and also frontend/web/assets containing styles and scripts required for the project. The last one is accessible to the browser.

[evil1]

chmod 777 - R on the last dir mentioned above also changed nothing.

[samdark]

Hmm... it looks it searches for assets in tests which doesn't seem correct.

[evil1]

Is there any temporary workaround to fix this?

[evil1]

Creating assets directory in the project root did resolve the issue.

Another thing I faced before is when I was upgrading yii from 2.0.6 to 2.0.10 none of the tests where downloaded. I guess this chould be moved to another issue?
php composer.phar global require "fxp/composer-asset-plugin:^1.2.0"
php composer.phar update yiisoft/yii2 yiisoft/yii2-composer bower-asset/jquery.inputmask
did upgrade yii, but the file structure was not upgraded.

[evil1]

Now the problem is that LoginCest fails with valid login and SignupCest fails.
I tried to add some debug messages to test code.
user erau is actually being added to test DB correctly.
I disabled Fixture unload (so that test DB was not erased after the fixture was used), and manually logged in successfully with the credentials provided in fixture.

[samdark]

Creating assets directory in the project root did resolve the issue.

I guess it's a good idea to add empty assets directories to both basic and advanced templates by default...

[samdark]

but the file structure was not upgraded

It should not. Template is what it is. You start with it and then it's all yours. It's not meant to be updated.

[evil1]

Update:
After checking usertable I found that user tester which is added during SignupCest was not removed after the test (should that be done automatically?).
After manual removal, SignupCest is fine.
The problem still exists with LoginCest with valid credentials erau and password password_0.

[evil1]

It look like Fixture is not working in _before function of LoginCest.
I modified the checkValidLogin method, to check wether the user exists in database, and found that it's not there.
public function checkValidLogin(FunctionalTester $I) { $I->seeRecord('common\models\User', [ 'username' => 'erau', 'email' => '[email protected]', ]); }

[evil1]

Yes, Fixture is not correctly applied in LoginCest _before method.
After manually adding the user to the test database all test are ok.

[evil1]

Some kind of magic happens.
I investigated ActiveFixture class (put some echo for debug purposes) both methods, loadand unload are called properly, but the user is not added to the database, and $this->resetTable() in the load section does not actually deletes any data in table.
@samdark do you have any idea, why this might happen?

[evil1]

Trying to load fixture with ./yii fixture/load also not woking:

If I run ./yii fixture/load User without specifying the namespace - yii tries to find fixture in unknown namespace @tests/unit/fixtures. Specifying the correct namespace also doesn't work.

[evil1]

@samdark I think the problem is related to PostgreSQL only.
I modified the load method of ActiveFixture and came to some interesting information:

and here is the output of the echo command:

My PostgreSQL version is postgresql-server-9.2.18-1.el7.x86_64
It seems like user is a reserved operator in PostgreSQL.
SELECT * FROM user returns current user instead of user table contents.
For sure this is not Yii-caused bug, but should be considered by developers.
Running the command in psql console also led to exactly the same results:

[samdark]

So that's specific to your application?

[evil1]

That is specific to database used.
I guess I'm not the only person, who uses PostgreSQL as a database server.

[samdark]

What I've meant is "are you using AR only to access user model or you did exactly that command via plain SQL"?

[evil1]

As I've mentioned in the begingin of the issue these are the tests out of the box, so only pure advanced template and codeception are installed. No coding was made.

[evil1]

So I believe only AR is used with no plain SQL used

[samdark]

That's a bit weird then since table names should be escaped properly.

[evil1]

My fault. When I put table name inside the quotes SELECT query works well
SELECT * FROM "user" returns propper output
I tried to enable DB query loggin, and see the following:

LOG:  execute pdo_stmt_00000007: INSERT INTO "user" ("username", "auth_key", "password_hash", "password_reset_token",
 "created_at", "updated_at", "email") VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING "id"
DETAIL:  parameters: $1 = 'erau', $2 = 'tUu1qHcde0diwUol3xeI-18MuHkkprQI', $3 = '$2y$13$nJ1WDlBaGcbCdbNC5.5l4.sgy.OME
KCqtDQOdQ2OWpgiKRWYyzzne', $4 = 'RkD_Jw0_8HEedzLk7MM-ZKEFfYR7VbMr_1392559490', $5 = '1392559490', $6 = '1392559490',
$7 = '[email protected]'
LOG:  statement: DEALLOCATE pdo_stmt_00000007
LOG:  statement: ROLLBACK

For some reason ROLLBACK is called.
I've put echo for debugging into rollBack() method of yii\db\Transation class and found that rollBack is called in each test

If I put $this->db->pdo->commit(); return; the problem is solved and the tests pass well.
I will try to investigate further, and find the cause of the problem, why the rollBack() method is called, and what for?
One more thing I've noticed, when $this->resetTable() is called, the following queries being executed:

LOG:  execute pdo_stmt_00000003: DELETE FROM "user"
LOG:  statement: DEALLOCATE pdo_stmt_00000003
LOG:  execute pdo_stmt_00000006: SELECT SETVAL('"user_id_seq"',1,false)
LOG:  statement: DEALLOCATE pdo_stmt_00000006

According to PostgreSQL docs https://www.postgresql.org/docs/9.2/static/sql-truncate.html it's much more efficient to use truncate to reset table in a single query:
TRUNCATE TABLE "user" RESTART IDENTITY
I'm not sure if this information is helpful, and will be used by Yii team, but still at least you will know that this approach also exists.

[evil1]

Just for testing purposes, I tried to change DB server to MySQL (MariaDB), and all the tests pass without any errors. So the problem is purely related to PostgreSQL

[evil1]

Overriding the load() method in common\fixtures\User did solve the problem, but this is just a temporary workaround, not a solution:

public function load()
{
    parent::load();
    $this->db->transaction->commit();
}

[samdark]

Could be related to https://github.com/Codeception/Codeception/pull/3973

[samdark]

https://github.com/yiisoft/yii2-app-advanced/issues/221

[evil1]

@samdark I don't think this is a codeception issue. I did some debugging and from the codeception side it seems to be ok. From the codecetion perspective the process is not database dependant at all, but from Yii perspective it is. When i put mysql: to DSN everything works perfectly. But if I use pgsql: fixtures are not applied (actually they do, but after that rollback is called and all the changes are reversed). The problem is somewhere in Data Access Layer.

[samdark]

Well, the rollback happens within Codeception.

[evil1]

But it does nothing in case you are using MySQL, and rolls back the transaction in case you are using PostgreSQL

[samdark]

Well, yes. I can reproduce failures using PostgreSQL but these are different than yours so I'm not sure what and why happens in your case.

[evil1]

I can make a freshly installed vmware image, and upload it somewhere to demonstrate the error presence.

[evil1]

The steps are as simple as possible:

composer global require "fxp/composer-asset-plugin:^1.2.0"
composer create-project --prefer-dist yiisoft/yii2-app-advanced yii-application
cd yii-application
./init

select 0 (Development environment)
configure database in common/config/mail-local.php
configure test database name in common/config/test-local.php

./yii migrate
./yii_test migrate
composer require "codeception/codeception"
alias cept='./vendor/bin/codecept'
mkdir assets
cept run

I tried deleting the project, and starting from scratch few times, and all the result were the same,

[purocean]

append to test config file.

    'components' => [
        'assetManager' => [
            'basePath' => __DIR__ . '/../web/assets',
        ],
    ],

[Nicolai6120]

I think purocean code should be included to next advanced template version. It`s clean and simple!

[sashsvamir]

Also have same issue after update codeception to 2.2.9:
[yii\base\InvalidConfigException] The directory does not exist: ./assets
problem solved after append code with assetManager's basePath like as @purocean showing above.
@Nicolai6120 this code already has been added https://github.com/yiisoft/yii2-app-advanced/commit/7f58ca1f53aab55bb15f8e8feb11aac4af4991f2#diff-3cc128d31956e25113f3d136ae06fdd0 on Nov 13, 2016

[dynasource]

as stated above, this problems seems to be resolved by commit https://github.com/yiisoft/yii2-app-advanced/commit/7f58ca1f53aab55bb15f8e8feb11aac4af4991f2#diff-3cc128d31956e25113f3d136ae06fdd0