Plots.jl: Visual regression tests failing consistently on Travis CI Linux with GR

[juliohm]

Hi all,

For some reason, my plot recipes are failing on Linux with the latest release of Plots.jl. I guess the issue is specific to GR because it shows an error message again with the GKS terminal failing to load:

https://travis-ci.org/juliohm/GeoStats.jl/jobs/272192863#L641-L655

Could you please confirm it? I tried running the tests locally on my laptop multiple times. The tests pass on Mac OS as well in Travis CI.

[mkborregaard]

Our tests work on linux so it should be possible to fix the GKS error. Try and look at our travis setup.

[daschw]

Travis errors because the images you are comparing differ. Did you update the images in Geostats/test/data?

[juliohm]

@daschw I did, I updated the images, and I do this every now and then given that the plots are constantly changing look.

[juliohm]

I think the issue is not on my package or Travis CI configuration, but it is related to the GR backend.

@jheinen it seems that https://github.com/jheinen/GR.jl/issues/46 is manifesting again in a different form? Could you please take a look when you have time?

[jheinen]

You should set the output device in runtests.j; - on Travis, you don't have access to interactive devices (X11 on U*nix, CoreGraphics on macOS or GDI on Windows):

Setting the GKSwstype environment to 100 (= no output) should fix the problem, e.g.:

ENV["GKSwstype"] = "100"

CI Tests can only be done using vector graphics or image output devices (SVG, PDF, PNG, ...)

Please refer to the corresponding section in Plots:

@testset "GR" begin
    ENV["GKSwstype"] = "100"
    @test gr() == Plots.GRBackend()
    @test backend() == Plots.GRBackend()

    image_comparison_facts(:gr, eps=img_eps)
end

[juliohm]

I will give it a try and close the issue right after. Awesome explanation! :slightly_smiling_face:

[juliohm]

The tests are passing in Linux now, thanks! Highlighting what @jheinen has said, the only necessary change is the addition of a single line in the top of your runtests.jl script:

ENV["GKSwstype"] = "100"