Protractor: Add closeBrowserOnExit option

[kayhadrin]

As discussed previously in #1425, now that Protractor is able to always close all browser instances when ending its process, it would be good to have the option to avoid this for debugging purposes.

i.e. Imagine having a closeBrowserOnExit setting that, when it is false, will let Protractor leave any remaining browser instance open after finishing its test run.
That setting default value would be true though.

It would be useful because I think that the developer sometimes wants to access the browser "as-is" to inspect the page or logs and find out why a test framework (jasmine, cucumber, etc...) failed to close it automatically.

Just food for thought...

[sjelin]

Hi kayhadrin,

What would the use case for this feature be? We're already developing browser.pause(), which could be used to inspect the state of a browser directly after failure. It's not totally clear how helpful just leaving the browser open would be, since the state of the browser when the failure occurred would likely be overwritten by future tests.

-Sammy

[kayhadrin]

Hi Sammy,

Here's my use case:

Let's say I am testing something like $('#target') and this element does not exist for some reason.
So it should fail the unit test.

at present, Protractor will throw an exception that will entirely stop the test run - even if there are other un-executed tests.
(That's what I've seen using CoffeeScript + CucumberJs, see #1394 for more info.)

To debug this, I've tried to use browser.pause() and while it does allow to pause the test, it crashes Protractor immediately when I open the web inspector of Chrome.

And now, since Protractor has now the ability to auto-close every browser instance before leaving, it makes it even more difficult to debug.

When this kind of scenario occurs, I usually have to go through a great deal of work to find the cause of a test issue.

As for the uncertainty of knowing the state of the browser when Protractor leaves, it's not really an issue for me. I can manipulate the test framework (CucumberJs, or any other for that matter) to run only the broken test scenario so that I know that the browser is in the right state.

Cheers,

David

[sjelin]

Hi David,

Sorry to take so long to get back to you. It seems like the core issue here is that browser.pause() isn't working. Could you give us some information about that?

-Sammy

[kayhadrin]

Hi Sammy,

I'll give you more details about this once I'm back from holidays.

Cheers,

David

On Wednesday, November 19, 2014, Sammy Jelin [email protected]
wrote:

Hi David,

Sorry to take so long to get back to you. It seems like the core issue
here is that browser.pause() isn't working. Could you give us some
information about that?

-Sammy

—
Reply to this email directly or view it on GitHub
https://github.com/angular/protractor/issues/1433#issuecomment-63524830.

[raptor235]

Hey guys just to add to this, browser.pause() works but only when calling before a assertion fail, once a fail happens the browser.pause() which comes after never gets called so the browser window just closest and you're let with only the terminal window.

[sjelin]

I don't fully understand why you can't just add a browser.pause() before the failed assertion

[raptor235]

You can... but when you have a a test with 10 assertions you then have to narrow down where the error happens and re-run the test... it would be nice to have the option for the test not to kill the driver so you can explore the test failure right after the assertion has failed and see what caused the failure...

[sjelin]

Doesn't the output tell you pretty clearly which assertion failed?

[raptor235]

It does but then you have to get back to the same state of the app in the browser to debug what happened... if the browser paused when the failure happened and not just closed we can then go in and inspect the page to find out what the state of the app is... are elements missing, do selectors need to be updated etc etc...

[scriby]

Here's my use case:

• We run our UI tests in parallel
• If one of the tests errors, it should leave the window in the state it was when the error occurred, and continue on testing in a new window
• A developer can review the failures and look at the state of the browser window to debug any failed tests

I'm having trouble getting it to work exactly like I want.

I got it to open a new window and continue running tests in it by using browser.forkNewDriverInstance(). The only problem is that when protractor finishes running tests it closes all the windows, so there's no time to investigate. The other issue with this is that when running tests in parallel, they don't report any output until the whole process is finished, so I can't just "hang" the process until they are done.

The other thing I tried is to call browser.pause() followed by process.exit(1). This successfully keeps the browser open after the process quits, but I can't run any more tests in the process after calling browser.pause(). So whatever tests followed an errored test will not get run.

Note that this is just for development machines. The build server would just take a screenshot of the current state and not try to keep the window open.

[ppisljar]

I am quite surprised that not more people are looking. I would tought its something everyone would want, but from responses here i see the developers still don't see a valid use case behind it. So i guess there must be another way of doing this ?

my current scenario:

• i run the tests, lets say one test failed
• the browser would close ... i need to inpect the console to see what went wrong
• i can see the assertion that failed as well as full stack trace which should give me a good idea about whats wrong
• but then ... to actually solve it ... i open up my test that failed, and then i open up a new chrome window and i try to get my application in that exact same state (so this might involve quite a lot of clicking ... )
• i use web inspector to find out whats wrong with my selectors

so the last two steps would be much easier, if i could do what scriby suggested. but i guess i am doing this wrong, becouse if everyone would be doing it that way i guess we would already have a solution for it.

so my actual question is ... what would be the correct way to go about this ?
and while i am already asking ... is there any chrome/firefox plugin to make writting the tests easier ?
( like recording what i am clicking and typing in ... and helping me with choosing the best selector ) ?

and ... what is the best way to view the status of tests ? currently i am using protractor-jasmine2-screenshot-reporter ... but i am wondering if anything better exists ?

(sorry for putting two unrelated questions here)

[jedrichards]

I develop on OSX with an editor and shell in full screen mode. Whenever I invoke Protractor OSX moves my desktop away from my dev layout to the newly opened browser instance. I then have to cmd-tab back. By the 200th time this gets pretty old! Admittedly this is more of a developer experience thing than a technical issue, but being able to have Protractor just reuse the same browser all the time would help.

[juliemr]

Note that you can now reuse a browser with the seleniumSessionId option: https://github.com/angular/protractor/blob/master/lib/config.ts#L69

I'm going to leave this feature open though, because I think it's a legitimate user story (wanting to leave it open wherever the error occurred that we'll look into more).

[juliemr]

See also https://github.com/angular/protractor/issues/2568

[otoomey]

I want to throw my two cents in here.

I was writing a test, and suddenly chrome just closed while I was running it and protractor spat this out:

Failed: unknown error: cannot focus element
      (Session info: chrome=53.0.2785.143)
      (Driver info: chromedriver=2.25.426924 (649f9b868f6783ec9de71c123212b908bf3b232e),platform=Linux 4.4.0-47-generic x86_64) (WARNING: The server did not provide any stacktrace information)
    Command duration or timeout: 38 milliseconds
    Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
    System info: host: 'dune', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-47-generic', java.version: '1.8.0_111'
    Driver info: org.openqa.selenium.chrome.ChromeDriver
    Capabilities [{applicationCacheEnabled=false, rotatable=false, mobileEmulationEnabled=false, networkConnectionEnabled=false, chrome={chromedriverVersion=2.25.426924 (649f9b868f6783ec9de71c123212b908bf3b232e), userDataDir=/tmp/.org.chromium.Chromium.7sk4aP}, takesHeapSnapshot=true, pageLoadStrategy=normal, databaseEnabled=false, handlesAlerts=true, hasTouchScreen=false, version=53.0.2785.143, platform=LINUX, browserConnectionEnabled=false, nativeEvents=true, acceptSslCerts=true, locationContextEnabled=true, webStorageEnabled=true, browserName=chrome, takesScreenshot=true, javascriptEnabled=true, cssSelectorsEnabled=true}]
    Session ID: 096e49e783409d12aaf9f55929d697cf

This is not very useful, especially since if I have a long test and I'm not paying attention to chrome, I have no idea why it crashed because the error message gives no hint as to where it died and I can't look in the browser because protractor closed it. I COULD go through every single line in the it() and add browser.pause() statements, but it's incredibly frustrating seeing that it would be so much easier to fix if the browser didn't close every time.

Protractor is supposed to make devs lives easier, but damn is it painful to write tests for.

[slykar]

There's a thing that is driving us crazy. Our e2e tests fail randomly. You can't use pause() in this case.
Using debug allover the place is not a clean option.

There should be an option that stops protractor on failure, so you can debug the problem.
Right now it's not possible at all in our case and it's really frustrating, causing our tests to have almost no value at all.

[kmizinova]

I have the same situation as topic starter. I used to develop on java + selenium webdriver and had to call driver.quit() in @after method. So when I needed to interact with browser I just commented out that line. I started to work in team that uses protractor for their tests and I'm in situation when I need to find faulty locator in some page that is hard to get to manually (you have to go through a long process of data inputs etc). I'm new to protractor and quite surprised that it doesn't have "closeBrowserOnExit" suggested above

[SyedMoin]

hi kayhadrin.. can you share the updates for your issue please.I'm not able to keep the browser open if it fails due to element visibility.. I want to know how to keep the browser open after test fails or completes thanks