Capybara: Capybara::Node::Element::text() does not return newline characters

[brainhurricane]

I have a <pre> element in which lines of text are separated only by newlines, for example:

<pre id="log">2013-02-04 10:03:20 INFO: Successful login for Employee 32 (Boyzen Berry)
2013-02-01 09:54:18 INFO: Successful login for Employee 32 (Boyzen Berry)
2013-01-31 16:26:13 INFO: Successful login for Employee 1 (Jane Staffer)
2013-01-30 18:09:35 INFO: Successful login for Employee 32 (Boyzen Berry)
2013-01-30 13:40:47 INFO: Successful login for Employee 6 (Fred Astaire)
2013-01-29 17:42:18 INFO: Successful login for Employee 1 (Jane Staffer)
2013-01-29 17:42:12 WARN: An unauthenticated user attempted to access the page /staff/applicants
2013-01-29 17:38:25 INFO: Successful login for Employee 1 (Jane Staffer)</pre>

I am using Cucumber to try to verify that the correct number of lines of text are appearing in the <pre> element. Using save_and_open_page and inspecting the HTML of the resulting file, I can see that there are indeed newline characters in the text of the element, however:

page.find("pre#log").text.gsub( "\r", "\n" ).split( "\n" ).count.should == 8

... fails with the following:

expected: 8
got: 1 (using ==) (RSpec::Expectations::ExpectationNotMetError)

Looking at the actual return value of page.find("pre#log").text shows me this:

[2] pry(#<Cucumber::Rails::World>)> t = page.find( "pre#log" ).text
=> "2013-02-04 10:03:20 INFO: Successful login for Employee 32 (Boyzen Berry) 2013-02-01 09:54:18 INFO: Successful login for Employee 32 (Boyzen Berry) 2013-01-31 16:26:13 INFO: Successful login for Employee 1 (Jane Staffer) 2013-01-30 18:09:35 INFO: Successful login for Employee 32 (Boyzen Berry) 2013-01-30 13:40:47 INFO: Successful login for Employee 6 (Fred Astaire) 2013-01-29 17:42:18 INFO: Successful login for Employee 1 (Jane Staffer) 2013-01-29 17:42:12 WARN: An unauthenticated user attempted to access the page /staff/applicants 2013-01-29 17:38:25 INFO: Successful login for Employee 1 (Jane Staffer)"
[3] pry(#<Cucumber::Rails::World>)> /[[:cntrl:]]+/.match( t )
=> nil

So it appears that, even though Capybara's internal representation of the page includes newlines or CRs (as seen via save_and_open_page), the text() method is filtering them out. This is preventing us from testing this functionality until/unless we find a workaround.

[abotalov]

Capybara normlizes whitespace (i.e. changes newlines etc.) to spaces. It's done as type of whitespace usually doesn't matter in HTML.
If you use Selenum, you can use this workaround:
page.find("pre#log").native.text.gsub( "\r", "\n" ).split( "\n" ).count.should == 8

[abotalov]

@jnicklas Do you think it's worth to change behavior of text if it's <pre> element where type of whitespace matters.

[brainhurricane]

Currently we're using capybara-webkit as the driver. Unfortunately page.find("pre#log").native is of class String with this driver, which isn't terribly helpful.

I've tried using selenium, but it has issues with the login screen of our application, and fails for no apparent reason on earlier steps.

I'll see if I can use some combination of send() and instance_eval() to dig the native text out of the Webkit node.

[brainhurricane]

Finally got this to work with the following:
page.find("pre#log").instance_eval( '@base' ).invoke( 'text' ).gsub( "\r", "\n" ).split( "\n" ).count.should == 8

[beggerss]

@brainhurricane @abotalov, did you guys find a workaround that works with capybara-webkit? I'm having the same issue

[brainhurricane]

@beggerss Using capybara-webkit, I found I could use page.find( '#log' ).instance_eval( '@base' ).invoke( 'text' ) to get the raw text of the #log element, instead of page.find( '#log' ).text --- does this work for you?

[beggerss]

@brainhurricane I'm trying to download a PDF within <pre> tags. I've been using page.find(:xpath, '/html/body/pre').instance_eval('@base').invoke('text') and it looks properly formatted, however my PDF viewer says the doc is damaged, and it is about 1.5x larger a file than the original, viewable PDF

[brainhurricane]

@beggerss If the data coming out of that command is correct, then I suspect that your issue may not be with Capybara, but with how you're embedding the PDF.

AFAIK, browsers do not support PDF data embedded in <pre> tags – you would have to use an <object> or <embed> tag. There's some discussion on Stack Overflow about which is better.

[beggerss]

@brainhurricane, I'm actually using capybara for webscraping instead of site testing, so it's not under my control. Thanks for your help!

[brainhurricane]

@beggerss Good luck and godspeed!

[michaelfagan]

For others who stumble here...

With capybara-webkit driver, as mentioned, newlines are collapsed into spaces as per how browsers render pages.

In my case I'm testing a CSV export, and so newlines are important to me.

The trick is to use source instead of page as both are available but source does not have normalized whitespace. I found this via the capybara documenation here: http://rubydoc.info/github/jnicklas/capybara/master/Capybara/Session

[whomwah]

@michaelfagan :+1: