Generator-jhipster: Switch back from Chromium to PhantomJS?

[jdubois]

We switched from PhantomJS to Chromium in #6377 by @PierreBesson

This is of course the future, but at the moment I have at least 2 feedbacks from clients where it doesn't work:

• At one client, using Windows 7, yarn test did not run at all with Chromium
• At another client, using either Red Hat or Ubuntu, we have missing libraries that make Chromium fail

As we had none of those issues with PhantomJS (it works for both clients), I think it's too early to have Chromium, and we should switch back to PhantomJS.

Thoughts, ideas, comments?

[PierreBesson]

I personnaly think that keeping PhantomJS is not viable in the long term. The current best practice is definitely to use a headless browser. Current options are Chrome (>= v59) adn Firefox (>= v56).

I'm aware some people have issues with it but afaik they have not opened issues to report and document those. So for me there are currently no open bugs on Chromium headless.

• About the Windows 7 issue : I think this can be fixed in the short term. (see: https://github.com/karma-runner/karma-chrome-launcher/issues/154). Also we can always setup Karma to use the real Chrome or Firefox. We just need to document this.
• About compatibility issues with some linux distributions, I'm not sure what the problem is.
  [EDIT] Note that we are not using the latest version of puppeteer, we should do more tests.

[jdubois]

Yes I know people didn't send any bug reports: I guess they don't care about Karma tests. Or, as it is easy to change, they just do the change and don't tell us: this is quite common for "easy" issues, people solve the problem for themselves and don't report anything back to us.

Let's keep this open a couple of days and get some feedback. If nobody comments, let's close this.

[applecool]

@PierreBesson Were you able to run the karma tests using chrome headless on linux? If yes, could you please share your karma config. I am trying to fix this issue from a very long time and I am unable to get it work. The log says, starting chrome headless and it just stays there until the browseractivitytimeout is passed and then disconnects without executing any tests. I didnt had this problem on Mac. It just works seamlessly on OSX. Any of your suggestions would make my problem much clearer. Thank you.

[deepu105]

BTW this also introduced the side effect that the Protractor tests are also run in the headless mode now. Some people like to see them running as they may using screen capture tools etc on them

[jdubois]

Yes @deepu105 I also had that problem -> now if you run Protractor you miss the whole "wow" effect that people love. And that's important if you want them to use the tool.

[mraible]

+1 for adding the "wow" back in!

[PierreBesson]

@applecool Yes I had this working on Linux. Can you give more details about your setup ?

@deepu105 I also liked the wow effect but it is accessory. Also puppeteer let's you take screenshot from chrome headless (new chrome feature). Then I think it's still possible to use a non headless browser for protractor tests.

[applecool]

@PierreBesson This is my config.

module.exports = function (config) {
    config.set({
        basePath: 'build/',
        frameworks: ['jasmine'],
        plugins: [
            require('karma-jasmine'),
            require('karma-chrome-launcher'),
        ],
        files: ['tests.js'],
        autoWatch: false,
        browsers: ['ChromeHeadless'],
        singleRun: true,
        browserNoActivityTimeout: 120000,
        urlRoot: '/build/'
    })
}

[gmarziou]

We also had issues on Linux Centos 7 for our Jenkins build where installer did not work and we had to install gtk3 libs and others

[deepu105]

We had issues in our Linux Jenkins serup with chrome as well due to missing
libs

On 21 Oct 2017 11:52 am, "Gaël Marziou" [e-mail] wrote:

We also had issues on Linux Centos 7 for our Jenkins build where installer
did not work and we had to install gtk3 libs and others

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jhipster/generator-jhipster/issues/6567#issuecomment-338379532,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABDlFxAvo8l3cN3bLzNO6ua1wgvXvgssks5sub7ggaJpZM4QAbVG
.

[PierreBesson]

@applecool This doesn't look like code generated by a recent JHipster version. Can you open a stackoverflow post with full yo-rc.json and details of what you are trying to do.

Ok so far we have the following issues with Chrome Headless :

• Not able to run Windows 7. I confirmed this yesterday with a colleague. Still didn't find a way tout fix it. @gmarziou do you have working setup (Headless or not) ? This is the issue I'm most concerned with but I have not been able to understand what doesn't work. Surely we can fix this with documentation on how to connect karma to the user's Chrome install.
• Dependencies issues in some CI servers. At least it worked for my team on a Ubuntu 16.04 server. Similarly we need to document how to fallback to locally installed chrome.

Reverting to PhantomJS is still an option but I think having a real browser is much better.

[hdurix]

I also had this kind of troubles using Chromium Headless with Continuous Integration (Gitlab CI + Docker image openjdk:8). The Linux distribution was missing some libraries for launching Chromium Headless (for instance libXss).

I spent some time trying to resolve the issue and finally went back to PhantomJS as I didn't have too much time to spend on.

[jdubois]

I'm sorry @PierreBesson but for me that's a lot of people and a lot of projects having issues. We just can't provide something that doesn't work out of the box - I know we're "hipsters" and do all the latest cool techs, but here this is just failing for too many people, on too many different systems.

[gmarziou]

@PierreBesson I just checked with my colleague and he put this on hold as he could not get it to work on CentOS after few hours. Ironically, it works fine on Windows 7

[PierreBesson]

@hdurix, Did your karma config had the --no-sandbox flag, I think this would fix the issue.
The openjdk docker image is based on debian, the related issue is this one: https://github.com/GoogleChrome/puppeteer/issues/290
@gmarziou What was your setup on Windows. Did you use git bash ?

I found the canonical documentation to troubleshoot pupeteer: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md
According to this docs, it should work fine on linux systems if you install some dependencies.

@jdubois Agree, if we are not able to find satisfying fixes before the next release let's roll back. I'm will submit a PR to upgrade puppeteer and try it on different OSes.

[jdubois]

Thanks @PierreBesson -> yes let's try to upgrade, and provide better documentation

[hdurix]

@PierreBesson here an example you could run to reproduce the error:

docker run -v ~/workspace/moni-sync:/moni-sync -it openjdk:8 moni-sync/node_modules/puppeteer/.local-chromium/linux-497674/chrome-linux/chrome

Or you the flags:

docker run -v ~/workspace/moni-sync:/moni-sync -it openjdk:8 moni-sync/node_modules/puppeteer/.local-chromium/linux-497674/chrome-linux/chrome --no-sandbox --disable-setuid-sandbox

Both lead to the following error:

moni-sync/node_modules/puppeteer/.local-chromium/linux-497674/chrome-linux/chrome: error while loading shared libraries: libXss.so.1: cannot open shared object file: No such file or directory

I can do some other tests if you want.

[PierreBesson]

@hdurix Did you try to install the libs described in troubleshooting.md ?

[hdurix]

@PierreBesson I'll try this as soon as I've got some time (probably tomorrow).

[pascalgrimaud]

I spent a couple of hours tonight on this. In fact, we currently have the same issue in our jhipster/jhipster image.

To resolve, I needed to follow these instructions: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md#running-puppeteer-in-docker

Then, here my karma.conf.js configuration:

    browsers: ['ChromiumHeadless'],
        customLaunchers: {
            'ChromiumHeadless': {
                base: 'Chrome',
                flags: [
                    '--no-sandbox',
                    '--headless',
                    '--disable-gpu',
                    // Without a remote debugging port, Google Chrome exits immediately.
                    '--remote-debugging-port=9222'
                ],
                debug: true
            }
        },

[PierreBesson]

@pascalgrimaud Yes but you would need yo install chrome in your docker image and not using the chromium binary provided by pupetteer.

[hamark]

Hi all,

I had the same problem on my continuous integration system (Jenkins on a centos7), I finally made it work. To do so, I have :

• Upgraded the linux kernel (https://www.tecmint.com/install-upgrade-kernel-version-in-centos-7/)
• Installed chrome
• Installed some fonts

$ yum -y install google-chrome-stable.x86_64
$ yum -y install liberation-mono-fonts  liberation-narrow-fonts liberation-sans-fonts  liberation-serif-fonts

[pascalgrimaud]

Although the PR by @PierreBesson is merged, there are still some issues which isn't resolved, so I'm reopening this

[pascalgrimaud]

Here all the issues I encounter, when launching yarn test, which means in production profile:

• inside our image Docker, but it is resolved here https://github.com/jhipster/generator-jhipster/pull/6623

• the CI-CD to GitLab with .gitlab-ci.yml doesn't work anymore, as it used the image openjdk:8

• tested in our internal Jenkins at Ippon, and it failed with these logs:

[INFO] 04 11 2017 00:28:14.793:INFO [karma]: Karma v1.7.1 server started at http://0.0.0.0:9876/
[INFO] 04 11 2017 00:28:14.795:INFO [launcher]: Launching browser ChromiumHeadlessNoSandbox with unlimited concurrency
[INFO] 04 11 2017 00:28:15.186:INFO [launcher]: Starting browser ChromiumHeadless
[INFO] 04 11 2017 00:28:17.004:ERROR [launcher]: Cannot start ChromiumHeadless
[INFO]  /data/jenkins/jobs/pgrimaud-jhipster-app/workspace/node_modules/puppeteer/.local-chromium/linux-508693/chrome-linux/chrome: error while loading shared libraries: libXss.so.1: cannot open shared object file: No such file or directory

It's too much issue for no real value immediatly, excepting PhantomJS will be deprecated or is not maintained anymore.

As @jdubois, I would suggest to switch back to PhantomJS.
Then, work step by step in a branch to replace PhantomJS by Chromium, check and test if it works for Docker, GitLab, Jenkins, CentOS, Windows7 etc.

[jdubois]

Hi all, so again this is not solved (and I'm suffering from Puppeteer at the moment as I'm on the train and this takes a huge amount of bandwith!!!).

• Let's switch back to PhantomJS -> @PierreBesson do you agree? This is just temporary, but we need something working out of the box
• I also want to "wow" effect back, I currently have a bug I'm trying to solve, and the headless mode is just horrible
• We also need to make this work in Sonar (see #6581 ), our quality gate has been broken for far too long, this is not good at all

-> does everyone agree? who would want to take some of those tasks?

[Tcharl]

No way to fix every Puppeteer Issues? Working for a temporary does not look to be the best one

[PierreBesson]

I have written some docs on how to fix common chrome headless issues. It also has instruction on how to set up phantomjs. Most issues are actually environment issues where you need some dependencies like libxss to install chrome.

About the wow effect, you should be able to bring it back simply by switching karma from ChromeHeadless to Chrome.

Then if we decide to switch back to Phantomjs I would understand as it impacts a lot of setup.

[pascalgrimaud]

@jdubois : I can work on reverting to PhantomJS (unless you already started it)

Then, with @PierreBesson we can start a branch to prepare the migration to ChromeHeadless, and tested all CICD

[jdubois]

Yes @pascalgrimaud go ahead!

[pascalgrimaud]

Soon day work...
I'll try to do it before the end of day

[jdubois]

BTW I have an entity with a ZonedDateTime: the test is failing today with Chromium and I'm pretty sure this was working before - I hope it will still fail with PhantomJS :-)

[thomastardy]

Just for information if anybody tries to run the current version with ChromiumHeadless on CentOS 7, you have to install the packages chromium-libs and gtk3. This solved the problem for us.

[BitFlipp3r]

Has #6675 been reverted? I'm using v5 beta and get the same chromium error:

25 05 2018 13:56:42.892:ERROR [launcher]: ChromiumHeadless stdout: 
25 05 2018 13:56:42.892:ERROR [launcher]: ChromiumHeadless stderr: /builds/PentestManager/pentest-manager-gateway/node_modules/puppeteer/.local-chromium/linux-536395/chrome-linux/chrome: error while loading shared libraries: libXss.so.1: cannot open shared object file: No such file or directory

[pascalgrimaud]

Yes @BitFlipp3r : a lot of things has changed. Here the history:

• in 4.x, we try to migrate from PhantomJS to puppetter
• after some issues, we decided to switch back to PhantomJS
• in v5 beta, we decided to try puppetter, as PhantomJS is deprecated now
• but currently, in master branch, we migrate to Jest and it seems far better!

[Tarekajaj]

@pascalgrimaud is there a workaround right now for developer using v5 beta?
we are facing the same error as @BitFlipp3r when using Gitlab-CI

[pascalgrimaud]

@Tarekajaj : you can have a look at the official doc: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

But here some solutions:

• migrate back to PhantomJS, it's really easy and fast, but not a good solution as it's deprecated
• migrate to Jest, as we did in our current master branch, it's by far, the best solution
• try to fix your build environment
  
  
  
  • if you use Docker container in Gitlab CI, you can try to use jhipster/jhipster:v5.0.0-beta.1 instead of openjdk:8
  
  
  • this image contains all required libs for Puppeteer: https://github.com/jhipster/generator-jhipster/blob/v5.0.0-beta.1/Dockerfile#L24-L61