Codeception: Suggestion: Generate namespaced Cest files by default

[ThomasLandauer]

I'm guessing that the majority of framework users wants their tests to be namespaced properly. Right?

Documentation is poor (not to say: inexistant), with https://codeception.com/docs/08-Customization#Namespaces being the only reference I found.

So here's what I did, and I'm suggesting to make this the default behavior of codecept g:cest functional First, to save people from having to copy-paste it into their Cest files later on:

1. Add this to composer.json (this might already be implemented in recent versions):
   json "autoload-dev": { "psr-4": { "Tests\\": "tests/" } },
2. Add this to codeception.yml:
   yaml namespace: Tests
3. Add this on top of every (functional) Cest file:
   php namespace Tests\functional; use Tests\FunctionalTester;

I'm not 100% sure if this is the best way to do it, so please doublecheck.

This would also solve https://github.com/Codeception/Codeception/issues/4874

[aleksblendwerk]

I don't mean to hijack your issue but I want to achieve proper PSR-4 style namespaces and directories as well. So I have done the same as you described above and tried to do some more.

Sorry, long read ahead and let me know if this should be a different issue but essentially I am trying to reach the same goal: more standard-like namespaces and directory names.

So first, I changed the support path in codeception.ymllike this:

paths:
    support: tests/Support

I also created my suite with the name Acceptance and changed the files generated by

vendor/bin/codecept generate:suite Acceptance
vendor/bin/codecept generate:cest Acceptance LoadLabelsCest

a bit, so the structure and namespaces now look like this:

tests/
    _output/
    _data/
    Acceptance.suite.yml
    Acceptance/
        LoadLabelsCest.php # namespace: Tests\Acceptance
    Support/
        AcceptanceTester.php # namespace: Tests\Support
        Helper/
            Acceptance.php # namespace: Tests\Support\Helper
        _generated
            AcceptanceTesterActions.php # (wrong) namespace: Tests\_generated

So now I am facing two problems:

First Problem

The generated AcceptanceTesterActions.php in tests/Support/_generated has the namespace Tests\_generated when I actually would want it to be Tests\Support\_generated. I fixed this (in a rather hacky way) by changing the autoload-dev section in composer.json to:

"autoload-dev": {
        "psr-4": {
            "Tests\\": "tests/",
            "Tests\\_generated\\": "tests/Support/_generated/"
        }
}

Now I also modified AcceptanceTester.php accordingly:

<?php

namespace Tests\Support;

use Codeception\Actor;
use Tests\_generated\AcceptanceTesterActions;

class AcceptanceTester extends Actor
{
    use AcceptanceTesterActions;

    [...]
}

But it would be nice to be able to specify the namespace, maybe even the location of the generated TesterActions file.

Second problem

Now when I run vendor/bin/codecept run, it fails with this error: [Error] Class 'Tests\AcceptanceTester' not found, as the complete class name is Tests\Support\AcceptanceTester in my case.

I fixed this (in an even more hacky way) by adding a bootstrap file in Acceptance.suite.yml, at Tests/Acceptance/bootstrap.php with the following content:

<?php

class_alias('Tests\Support\AcceptanceTester', 'Tests\AcceptanceTester', true);

Finally I explicitly added

use Tests\Support\AcceptanceTester;

to my Cest.

Now I can run the tests and more or less have my desired directory structure but of course it feels very hacky... I think most of this could be solved by being able to manually set the full class names and locations for both AcceptanceTester and AcceptanceTesterActions - or set an option that respects the support directory setting and treats it as part of the generated namespace. All optionally of course.

But the way it is right now, the Codeception directory structure just feels a bit odd compared to the rest of the project and also makes all my code quality check tools cry.

[ThomasLandauer]

I'm certainly no expert here, but I'd say:

Change it back to support: tests/_support in codeception.yml. Given the other directories (_output, _data), _support makes sense to me. Besides, you shouldn't have to do much there, so it really doesn't matter.

[aleksblendwerk]

As opposed to the support directory, the output and data directories do not contain classes, so I am all right with them being written with the leading underscore.

But to me it's weird when I have got a class with namespace Tests\Helper that resides in tests/_support/Helper. So there's the leading Tests in the namespace and then there's _support in the filename/path which doesn't show up in the namespace at all.

[ThomasLandauer]

So there's the leading Tests in the namespace and then there's _support in the filename/path which doesn't show up in the namespace at all.

Yeah, that's indeed weird ;-) I think this is due to Codeception internals: Upon each run, the content of tests/_support/Helper/Functional.php is copied into tests/_support/_generated/FunctionalTesterActions.php and then taken from there. I don't know the reason for doing it this way. => If this bothers you, please open a separate issue :-)

[tacman]

Found this thread while researching the deprecation error for Symfony 5.1. Is that something that is either already fixed (in future tests) or on the to-do list? It's not a show-stopper, but it doesn't generate a bunch of warnings that get in the way of other things. Thx.

[simonberger]

@tacman I guess the deprecation messages you face are those from coming from composer, right?
@aleksblendwerk good analysis. Especially the namespace of the *TesterActions class stops all clean tries to fix the namespaces. Maybe enable us to configure the namespace of the generated files is a fast way to fix this?

[ghost]

Adding:

    "autoload-dev": {
        "classmap": [ "tests/_build/support" ]
    },

to composer.json at least disable deprecation messages...

[Aerendir]

This is just for Google as I didn't found this issue searching on it:

Deprecation Notice: Class App\Tests\ExampleTest located in ./tests/unit/ExampleTest.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class App\Tests\Entity\EmailTest located in ./tests/unit/Entity/EmailTest.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class App\Tests\AcceptanceTester located in ./tests/_support/AcceptanceTester.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class App\Tests\FunctionalTester located in ./tests/_support/FunctionalTester.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class App\Tests\UnitTester located in ./tests/_support/UnitTester.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class App\Tests\Helper\Functional located in ./tests/_support/Helper/Functional.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class App\Tests\Helper\Unit located in ./tests/_support/Helper/Unit.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201
Deprecation Notice: Class App\Tests\Helper\Acceptance located in ./tests/_support/Helper/Acceptance.php does not comply with psr-4 autoloading standard. It will not autoload anymore in Composer v2.0. in phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:201

[simonberger]

Is there an interest by the team to fix this problem?
I would be willing to work on a solution if a PR will be processed.

[ThomasLandauer]

Ping @Naktibalda & @DavertMik :-)

[ghost]

just like @simonberger , I offer to help fix this, in my opinion this single problem makes Codeception seem totally outdated, and this should be tagged as the most critical fix to implement. I wont re-ping but symbolically I join @ThomasLandauer

[Aerendir]

I offer to help fix this, in my opinion this single problem makes Codeception seem totally outdated, and this should be tagged as the most critical fix to implement.

Quote in totally.

[aleksblendwerk]

If this gets an overhaul, I think it would be good to completely get rid of directories and namespaces starting with _, also including _generated. Not sure what the new name or location should be in a proper structure but the underscore stuff really sticks out.

I assume this is what you guys have in mind as well, just wanted to make sure and state it explicitly.

[simonberger]

If this gets an overhaul, I think it would be good to completely get rid of directories and namespaces starting with _, also including _generated. Not sure what the new name or location should be in a proper structure but the underscore stuff really sticks out.

Yes its is, but _support can already be fully replaced by Support (or whatever). It just shouldn't be the default and suggested structure anymore.
The generated class in the _generated folder/namespace is the main issue we need to resolve.

[ghost]

Even though I don't like the name of the folders starting with _
This character has a clear function of not mixing the Acceptance, Unit, Functional, Api directories with everything else:

tests: /tests
    _build:     /_build
    acceptance: /Acceptance
    api:        /Api
    functional: /Functional
    unit:       /Unit

So this should remain an exception to the rule. I just suggest a single _build folder.
In this way,

paths:
    support: tests/_build/Support
    data:    tests/_build/Data
    envs:    tests/_build/Envs
    output:  tests/_build/Output

And regarding namespaces...:

tests:
    support:    'Support'   # App\Tests\_build\Support 
        actors:     'Actors'    # instead of _support root
        actions:    'Actions'   # this was _generated
        helper:     'Helper'
        page:       'Page'
        step:       'Step'      # it generates App\Tests\_build\Support\Step
        utils:      'Utils'

Although this does not eliminate the root problem, it is tolerable and less chaotic, plus it does not involve completely relearning the Codeception directory structure. of course it is just a suggestion and I may be wrong. There is still room for improvement.

For example, an alternative might be:

App\Tests\_support\Actors
App\Tests\_support\Actions
App\Tests\_support\Helper
...

with a folder structure like:

tests/_support/_data
tests/_support/_output
tests/_support/Actors
tests/_support/Actions
tests/_support/Helper
...

[simonberger]

The goal should be in first place to realize correct PSR-4 namespaces.
Personally I have no big problems with the current names beside that they really are looking ugly in a namespace. _support and _generated are not conflicting PSR-4. They should all changeable by the user in any way.

@TavoNiievez I see your intention but it makes no sense to hide user written classes like support/helper as well as _data behind a build folder.

For me names are secondary and shouldn't set back a solution for the namespace issue.
But I won't invest time before the team of this library gave a little comment to this problem.

[Aerendir]

I think we are going out of scope.

In this moment the priority is to make the classes psr4 compliant.

In the psr4 the current names are totally good.

The unique problem is the namespace of the classes generated.

The namespace is the only thing that has to be changed and created accordingly to the folder structure.

So a namespace like ‘..._generated...’ is correct.

And is sufficient in the meantime.

The naming of folder and classes should be discussed elsewhere, I think.

Just for reference, this is the link to the psr4: https://www.php-fig.org/psr/psr-4/

[Aerendir]

Anyway, as a side note, I agree with who say that the underscore has a purpose: I think it is useful.

[aleksblendwerk]

Changeable sounds good to me!

[simonberger]

I already had a closer look and actually it seems possible to have clean namespaces without a hack.
Just put the namespace to your _support folder in your*.suite.yml files like this.

actor: AcceptanceTester
namespace: tests\_support

Now I have to use this namespace for my AcceptanceTester class. The generated AcceptanceTesterActions will also be
created with the correct namespace tests\_support\_generated.

To me it is not really clear what the namespaces in the config files really should mean and control but at the end it works as intended for me and I found no drawback. It is not possible to change the _generated but this class has to be included just once.

[ghost]

@simonberger so every time i run vendor/bin/codecept clean and vendor/bin/codecept build, do i have to manually change the include of Actions?

[simonberger]

I just mean changing the use of AcceptanceTesterActions trait in AcceptanceTester not the AcceptanceTesterActions class itself.

[Aerendir]

Just put the namespace to your _support folder in your*.suite.yml files like this.

@simonberger , ok: this should be set by default by Codeception.

When I install it the first time, I expect it is almost perfect, for sure I expect it doesn't throw any error, exception, issue in any place/tool.

[DavertMik]

Thanks @simonberger for this fix!

So my thoughts here:

• I agree we should switch default politics and provide a default namespace for Cest tests.
• It is pretty hard to find a good strategy for all cases, because tests are really different from support classes
• We don't treat support classes the same way as test classes. There are very rare cases that test classes must be autoloaded, so naming and composer-loading strategy for them was different.
• However, tests and support classes under the same namespace should be named by the same manner
• _support namespace is really ugly, that's why we avoid that name part when autoloading...

So the only option I see now is to get rid of _ in folder names. _ marked a special behavior for those folders. And in case of _generated a special usage of this class in a corresponding namespace.

Well, the solution I see now is new default structure like this one:

tests/
  Acceptance
  Functional
  Unit
  TestData
  TestSupport
  TestOutput

What I don't like about this structure that folder names won't be ordered in a correct way. But prefixes should help to find the which are actual suites and which are special directories.
I see the only option is unifying things and creating a common directory structure for default setups.

We really not married to those _ and _generated, everything can be changed via configs.
So the case here is to create new config defaults for next Codeception version.

As I said - the only way we can go is to unify folder/namespace names. But the directory structure is the subject of discussion.

[Aerendir]

So the case here is to create new config defaults for next Codeception version.

@DavertMik , I think this should be the priority in this moment: easy to do, fast to release, no discussions needed.

Changing the folder structure and their names, instead, requires way more time.

As I already said, the purpose of the underscore "_" is clear and very useful: I think that, also if ugly, it should be maintained as it contribute to make clear that in that folders there are files with a special purpose, internal or generated... something that has to stand out.

This is something used also in code by other libraries: is a practice that, also if may not be a "best" one, is a very used one and to which PHP dev are used to.

[simonberger]

@DavertMik I like the new names

@Aerendir I second your priorities. But _support is not really such a path that stands out as this includes user written code.
_support can already be replaced by something else without code changes.
So I think it could be a good idea to also adjust the defaults of the commands generating the tests in the first go.
Regarding _generated this could be easily made configurable and its default stick to _generated to be backward compatible. Everyone who does not like it is able to change it.
In the next major version there could be a decision to change this default name or not.

[eXorus]

About the new folder structure described by @DavertMik

tests/
  Acceptance
  Functional
  Unit
  TestData
  TestSupport
  TestOutput

I'm 👌 with Acceptance, Functional and Unit.
But I don't like TestData and TestOutput. For me I prefer to have them like today _data, _output. And more particularly because they are non php folders.

About TestSupport it's more complex to handle. I write a lot of codes in this folder so I don't understand why it's a subdirectory of support. Maybe a more flatten approach, but I know it's too big for this issue:

tests/
  Data
  Helpers
  Output
  Pages
  Steps
  Suites
    Acceptance
    Functional
    Unit
  Support (only for the generated stuff)

I'm not an expert so maybe my comment is totally irrelevant but I trust you to make the right choice.

I really enjoyed this discussion to adapt a more modern structure for codeception. Thanks all.

[DavertMik]

@eXorus I really like the structure, and I would really prefer it for enterptise setup. But it is too much for basic projects, also tests are few level deeper than support files so that can be really confusing at start.

However I am not so happy with my folder structure as well 😪

[ThomasLandauer]

Let me try to organize the discussion a bit, and summarize what we have so far. I see two main questions:

1. When to do what?

• Changing the directory structure is a BC break, so it can only be done in the next major version (i.e. v5.0).
• Question: Is it possible to introduce namespaces without a BC break?
  
  
  
  • If yes: Go for it, and do in it the next minor version (i.e. v4.2). Reason: Since everybody agrees that this is important/urgent, it's better to have ugly namespaces (till v5.0) than no namespaces at all ;-)
  
  
  • If no: Obviously do it in v5.0 :-)
  
  

2. Which directory structure

Everybody agrees that a namespace with an underscore is ugly and this should be avoided: namespace App\Tests\_support\Helper;

Maybe we should start by looking at the existing directories and discuss them one by one:

• _data: Just some images etc., so it won't get a namespace, right? So I'd say we should keep it as is (with the underscore), for the reasons given ("sticks out", "character has a clear function", "underscore has a purpose", "PHP devs are used to").
• _output: Keep it as is, for the same reason.
• _support: This is the one making problems ;-) Why? Because it contains both internal stuff (_generated), and user-written classes. So I think it would be a step forward if we separate those two aspects:
  
  
  
  • User-written classes: We're only talking about 6 files, right? (AcceptanceTester.php, FunctionalTester.php, UnitTester.php, Helper/Acceptance.php, Helper/Functional.php, Helper/Unit.php) My suggestion: The top-level directory (/tests) doesn't contain much. So let's move those 6 files up there! (Renaming Helper/Acceptance.php to AcceptanceHelper.php, etc.)
  
  
  • Internal stuff (_generated): Just 3 files, right? (AcceptanceTesterActions.php, FunctionalTesterActions.php, UnitTesterActions.php) My suggestion: Up to the top-level directory as well! Question: May a class name start with an underscore? If yes, rename them to _generatedAcceptanceTesterActions.php. If no, GeneratedAcceptanceTesterActions.php. Both should be fine to indicate that people shouldn't touch them ;-)
  
  

One more point: Since namespaces are usually all capitalized, I suggest to change the first letter of the 3 suite folders to uppercase: Acceptance, Functional, Unit

So in total I'm suggesting (folders enclosed in []):

[_data]
[_output]
[Acceptance]
[Functional]
[Unit]
_generatedAcceptanceTesterActions.php
_generatedFunctionalTesterActions.php
_generatedUnitTesterActions.php
acceptance.suite.yml
AcceptanceHelper.php
AcceptanceTester.php
bootstrap.php  --  maybe
functional.suite.yml
FunctionalHelper.php
FunctionalTester.php
unit.suite.yml
UnitHelper.php
UnitTester.php

BTW: While we're at it, I'm also suggesting to rename "Acceptance" to "Browser Tests", and "Functional" to "Framework Tests", see https://github.com/Codeception/Codeception/issues/5970

[Aerendir]

@ThomasLandauer

If yes: Go for it, and do in it the next minor version (i.e. v4.2). Reason: Since everybody agrees that this is important/urgent, it's better to have ugly namespaces (till v5.0) than no namespaces at all ;-)

Perfectly explained!

[DavertMik]

I will try to sum up the different proposals. I agree with the criticism of my structure - too many Test* Folders.
However, I don't like the idea of putting all the Helpers/Testers into tests folder, as I think we should keep that directory clean and focused on tests.

Ok, how about this:

tests/
  _output => yeah we got used to this folder so much, it doesn't need to belong to namespace
  Acceptance
  Functional
  TestSupport/
    Data/ => ok, let's  put all data things here, because Data can also contain classes
    Generated/ => we still need generated classes
    Helper/
  Unit/

I'm still not so confident about the name "TestSupport" but I can't think of something better.
I think it should still be a "support" folder, as it was a good idea to keep the support code separated from the main code.

[aleksblendwerk]

@DavertMik:

Looks quite good to me. Maybe just Support instead of TestSupport then?

[DavertMik]

@aleksblendwerk I want to have it somehow different from suite folders, so there should be a special prefix...

[Aerendir]

I think that it should be prefixed by _ to make the folder standout and make anyone, also the ones not used to Codeception, understanding that the folder has a special meaning/purpose. 😅

[ThomasLandauer]

However, I don't like the idea of putting all the Helpers/Testers into tests folder, as I think we should keep that directory clean and focused on tests.

Do you have a particular reason for this?

Let me mention another aspect for newbies: If they see a file called "Functional.php", buried two levels down (TestSupport/Helper/), they certainly won't think that this is somehow important. However, if they see it right away, on the top level, they might ask themselves "What is this and what can I do with it?" - which is exactly what we want ;-)
Or (in short): The more you hide, the less they'll use it!

[DavertMik]

I think that it should be prefixed by _ to make the folder standout and make anyone, also the ones not used to Codeception, understanding that the folder has a special meaning/purpose.

But this leads us to _ namespace, which we try to avoid

Well, this Functional file is absolutely not important until you write a bunch of tests and you need to refactor them.
So I think the more confusing is to put more files into the root directory, so users would need to search for actual tests.

[ThomasLandauer]

But this leads us to _ namespace, which we try to avoid

Exactly. We have two conflicting interests here, without any progress throughout this entire thread. And I think that in fact it just is not possible to find a solution that covers both of the following interests (however, feel free to continue searching... :-)

• Nice namespace (i.e. no underscore)
• Outstanding directory (i.e. with underscore)

That's why I'm suggesting: Use the top-level folder, and make the filename stand out, not the directory name.

Well, this Functional file is absolutely not important

To be honest, I don't know what FunctionalTester.php is ;-) What I said about discoverability was regarding FunctionalHelper.php.

so users would need to search for actual tests

Well, in the first place, they don't search for tests, but they need to write and save them :-) And organizing the three tests suits in three folders by that name makes perfect sense to anybody. Besides, it wouldn't change - the path stays /tests/Functional.

In other words: In my project's top-level folder, I have 15 files. Stuff like symfony.lock, .gitignore, phpstan.neon, codeception.yml, phpunit.xml.dist. Most of them I open rarely - if ever. However, I always find my Controllers :-)

[terabytesoftw]

I will try to sum up the different proposals. I agree with the criticism of my structure - too many Test* Folders.
However, I don't like the idea of putting all the Helpers/Testers into folder, as I think we should keep that directory clean and focused on tests.tests

Ok, how about this:

tests/
  _output => yeah we got used to this folder so much, it doesn't need to belong to namespace
  Acceptance
  Functional
  TestSupport/
    Data/ => ok, let's  put all data things here, because Data can also contain classes
    Generated/ => we still need generated classes
    Helper/
  Unit/

I'm still not so confident about the name "TestSupport" but I can't think of something better.
I think it should still be a "support" folder, as it was a good idea to keep the support code separated from the main code.

hi, thanks for all the work, there is some progress already composer2 has come out, somehow this problem should be solved

[DavertMik]

Composer was released.
I am are going to implement the proposed changes as well and release a next major version of Codeception soon

[Naktibalda]

@DavertMik You promised to implement this change, please get it done this month.

[DavertMik]

Yes, thank you for reminder!