Imagesharp: Extend CI test execution to macOS

[antonfirsov]

Initial attempt failed, we need to fix the following issues:
https://github.com/SixLabors/ImageSharp/pull/1302#issuecomment-668183518

Disabling the two (likely) bugs with [ActiveIssue] might be also fine.

[rohansp]

@antonfirsov I am interested in contributing to this project.Can I help?

[JimBobSquarePants]

@rohansp Absolutely! That would be awesome.

Basically the process would be to build and run the tests on your Mac and then raise a PR adding the new CI option to the workflow like #1302 Any questions you have on any failing tests can be raised either here or in a draft PR and we can work together to ensure everything passes.

[antonfirsov]

@rohansp we validate our output by comparing the resulting image to a reference output under tests/Images/External/ReferenceOutput (-- comes from a submodule). The comparison is tolerant. The majority of the failures are because of a small difference. These could be fixed by relaxing the tolerance threshold a bit, preferably in a selective way.

There are a few failures with a high difference which may indicate a bug. By running the tests locally, there is a way to observe the difference visually to identify the nature of the bug. Actual output is saved under tests/Images/ActualOutput when running in debug.

Any help is welcome.

[rohansp]

@JimBobSquarePants Small problem.I have a Windows 10 machine. MacOS is not installed on my machine.Sorry, didn't realize earlier.

[AlexNDRmac]

@antonfirsov I can help you with cross-platform code coverage and codecov reports

[antonfirsov]

@AlexNDRmac this issue is not about metrics or reports, the problem is that at the moment we are not able to do stable test runs on macOS at all. I believe this is easy to fix, see the opening post and https://github.com/SixLabors/ImageSharp/issues/1387#issuecomment-712183926 for details.

If you think you want to help with this, I'm happy to answer any questions!

[AlexNDRmac]

I see the problem with tests on Mac, I'll try to sort out with it. I'll ping you when I'll be ready to ask smth :)

[JimBobSquarePants]

Love it. Thanks!

[AlexNDRmac]

@antonfirsov I've fixed error with libgdiplus... but I see some any problem with tests, like allocation error. Maybe you can recommend more stable branch than master for isolated testing macOS CI with libgdiplus issue?

[JimBobSquarePants]

Master is stable. Nothing gets in there unless all tests pass.