Amphtml: Feature Request: No way to check AMP Validation State using Vanilla HTML+JS
What's the issue?
_Feature request:_
_Gherkin:_
As a developer writing end to end tests
When I go to an AMP page with #development=1 appended to the URL
I expect that I can check the validation state of the current page using a single line of synchronous JS in the console
_More info_
I think it makes the most sense for there to be a data attribute appended to the <html amp> element in this fashion <html amp amp-valid> and for failure <html amp amp-invalid>, but only when in development mode.
That way the developer can reliably check that attribute in order to assert the validation state of the AMP page without any GET requests to third parties, XHR, or Node.
I envision using something like
return document.documentElement.hasAttribute('amp-valid') in my Ghost Inspector tests or in my Selenium/Capybara feature tests. I'd just have to account for the 1-2 seconds it takes for the validator to run before asserting.
How do we reproduce the issue?
N/A
What browsers are affected?
N/A
Which AMP version is affected?
Latest 1494446565961
johnnyshankman
All 14 comments
Why aren't you just using the validator?
jridgewell
on 16 May 2017
@jridgewell not possible from something like Ghost Inspector, or Ruby/Selinum based feature testing. I have an already-renderd AMP page that I've asserted has correct content and want to lastly check that it is amp-valid.
We're not testing against pre-rendered fixtures, we're using this for production smoke testing real articles in using Selenium to keep tabs on our CI.
Use case: Every hour Ghost Inspector hits production AMP article and appends #development=1, checks the response for 200, checks for some content, then finally checks that the document is amp-valid using this new feature.
johnnyshankman
on 17 May 2017
Can you use Cloudflare's validation API?
jridgewell
on 18 May 2017
@jridgewell Yes, I could write a separate test suite that solely hits Cloudflare's validation API endpoint and then parses the JSON to check for validity (although not in Ghost Inspector, it'd have to be done using a different tool).
This seems like a redundant/unnecessary step to me though, since I'm already running E2E feature tests against fully-rendered AMP pages in Selenium/Capybara (or alternatively Ghost Inspector).
Wouldn't it be more efficient to take my same E2E tests with fully rendered AMP pages _but_ run them against pages in #development=1 mode _and then_ run Capybara's evaluate_script to assert something like window.AMP.pageIsValid becoming true when the time comes?
That way I get clean E2E feature tests which test for correct content _and_ amp-validity without having to do any asynchronous API calls to 3rd party services.
Here's the Capybara method I'm hoping to leverage in this scenario and here's a similar Ghost Inspector feature that would also do the job.
Definitely open to discussion though! Let me know of your concerns etc, I'm all ears.
johnnyshankman
on 18 May 2017
Another reason why using Cloudflare isn't a perfect solution:
We host our CI/pre-prod builds on a private server requiring username+password credentials. We'd have to expose our credentials into the URL using https://amp.cloudflare.com/q/name:[email protected] syntax in order to let Cloudflare through our authentication and validate the page. Not ideal.
@jridgewell
johnnyshankman
on 18 May 2017
/to @powdercloud. Ideally, the validator included with #development=1 would have just have a validate that works like the NPM module.
jridgewell
on 18 May 2017
I looked into it for a bit. The status quo is that when #development=1 is appended, this code runs:
https://github.com/ampproject/amphtml/blob/master/src/validator-integration.js
Specifically, there's a call to amp.validator.validateUrlAndLog.
And it looks like once this is all done and I'm in the browser's console (I use Chrome), I can put
amp.validator.validateUrlAndLog(window.location.href);
to trigger it again. The method validateUrlAndLog itself is in here:
https://github.com/ampproject/amphtml/blob/master/validator/engine/validator-in-browser.js
The main thing that may surprise at first is that it does a HTTP get request to fetch the page a second time. This is necessary because the DOM from which it was loaded is heavily modified by the AMP runtime once the validator is there, so we really need to get the contents of the page again.
If amp.validator.validateUrlAndLog were to also store a result object, would this be sufficient?
Then you could access this with amp.validator.validationResult. It would have all the information, like the status, the errors, parameters for the errors, etc.
The way to graft this together is for amp.validator.validateUrlAndLog to return this object, and for the runtime to store it in this variable. It's a slight bit hacky but makes it so we don't have to add this global variable to engine.js.
Opinions on this most welcome. I'm not familiar with Ghost Inspector etc., so it's hard for me to say whether this is the simplest possible solution - but maybe better than annotating the DOM. Note that with the solution outlined here you'd still end up with an extra HTTP fetch per document you're checking - that's just how #development=1 works.
powdercloud
on 19 May 2017
@powdercloud I noticed the same method and found the same confusion, why is there no function to revalidate the current page and return the state synchronously?
validateUrlAndLog is okay, but not nearly as useful because it is async/promise-based (due to the underlying GET request). Things like ghost inspector don't work well with promises are async.
Storing the return value in a global variable would be better than the current implementation because our front end tests can always sleep and then check that variable. I definitely like the simplicity of that idea, and it's just as easy as my original idea of amp-valid as a class on the HTML.
What would be most ideal is validateCurrentDocumentAndLog. It would validate the current page and synchronously return true or false, but I know that's much more complex.
johnnyshankman
on 1 Jun 2017
This might be a way.
1) Load the AMP page. E.g. load this one, with development=1 so that the validator gets loaded:
https://www.ampproject.org/docs/tutorials/login_requiring#development=1
2) In the console, run this:
var request = new XMLHttpRequest(); request.open('GET', location.href, false); request.send(null); var validationResult = amp.validator.validateString(request.responseText);
validationResult has the status field (in this case 'PASS') and also the errors if there are any.
Also:
- On my (pretty big) screen this fits into one line ...
- Chrome prints a deprecation warning for making a synchronous request (the last parameter, false, to request.open). Also documented here: https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/Synchronous_and_Asynchronous_Requests
- It would be worthwhile checking the other fields of request as well, e.g. request.status and make sure it's 200 before proceeding.
- This scheme fetches the page three times. But perhaps for testing this is a non-issue. If it bothers you, you could make a server side handler on your end which script includes the validator.js and then has a
Related issues
choumx
路
3Comments
edhollinghurst
路
3Comments
aghassemi
路
3Comments
torch2424
路
3Comments
gmajoulet
路
3Comments