Phpunit: Confusing failure message when assertFileNotEquals() is used on two empty files

[TomasVotruba]

| Q | A
| --------------------| ---------------
| PHPUnit version | 6.2.2
| PHP version | 7.1.6
| Installation Method | Composer

$this->assertFileEquals(__DIR__ . '/empty-file.html', __DIR__ . '/second-empty-file.html');

Works fine.
But...

$this->assertFileNotEquals(__DIR__ . '/empty-file.html', __DIR__ . '/second-empty-file.html');

shows

Failed asserting that '' is not equal to <string:>.

Not sure what that means. At the moment I use workaround:

$this->assertFalse(file_get_contents($normalPostLocation) === file_get_contents($ampPostLocation));

[SpacePossum]

If both files are empty the assertNotEquals should not pass right?
So the issue here is to make the (default) fail message better?

[TomasVotruba]

It's not about empty files. This happen for same-content files as well.

I don't now what is the issue is.

[SpacePossum]

$this->assertNotEquals('', '');
Failed asserting that '' is not equal to <string:>.

$this->assertNotEquals('a', 'a');
Failed asserting that 'a' is not equal to <string:a>.

$this->assertNotEquals([], []);
Failed asserting that Array &0 () is not equal to Array &0 ().

I might be wrong here but I think there is no issue,
other than the fail message Failed asserting that '' is not equal to <string:>. might be a bit cryptic, however as it states '' equals <string: '' > so the assert fails correctly.

A nice improvement might be to fail as;

Failed asserting that content of file "/var/foo/a.txt" is not equal to content of file "/var/foo/b.txt".

[TomasVotruba]

This is WTF to me. How do I know what "" is? I didn't event notice the "a" there.

I would expect this message:

$this->assertNotEquals('a', 'a');
Failed asserting that 'a' is not equal to 'a'.

To be consistent with the rest of PHPUnit messages.

$this->assertContains('a', $value);
// Failed $value ... contains 'a'.  # pseudo message, not sure what exactly is there

And not:

// Failed $value ... contains '<string:a>'.  # pseudo message, not sure what exactly is there

A nice improvement might be to fail as;

Failed asserting that content of file "/var/foo/a.txt" is not equal to content of file "/var/foo/b.txt".

Yes. With their content possible, I'd have to look for the real differences manually to the files otherwise.

[SpacePossum]

ah now I see :) :+1:
lets see if this can be improved

[sebastianbergmann]

I just looked at this (sorry for not looking at it sooner) and I am confused as to what this is about.

[SpacePossum]

There is nothing wrong.
When comparing two empty files with assertFileNotEquals the result is:

Failed asserting that '' is not equal to <string:>.

which is fine, but confusing for some users.

[sebastianbergmann]

Can we make it less confusing?

[TomasVotruba]

Awesome, thank you all