Powershell: Tests should not rely on external websites which lead to random failures
Today https://httpbin.org/ is down the whole day.
It caused two daily builds to fail with 43 tests for Invoke-WebRequest and Invoke-RestMethod.
https://ci.appveyor.com/project/PowerShell/powershell-f975h/build/6.0.0-alpha.10-26
https://ci.appveyor.com/project/PowerShell/powershell-f975h/build/6.0.0-alpha.11-27
We should replace httpbin with eaver our own hosted version or localhost server, i.e.
https://github.com/Runscope/httpbin
localhost server will introduce unnecessary testing dependencies, not sure which one would be a better solution.
cc @daxian-dbw @Francisco-Gamino
vors
All 10 comments
I prefer to host the service in cloud ourselves. I got it hosted at Heroku, but still not clear if the free plan would be sufficient.
I tried to get it running on Azure using MSDN subscription, but failed and I didn't get the time to resolve it.
daxian-dbw
on 19 Oct 2016
Move tests to use httplistener where we have more control and also suppress progress for the tests
SteveL-MSFT
on 22 May 2017
Also review for other external websites being used beyond httpbin.org
SteveL-MSFT
on 22 Jun 2017
@TravisEz13 I plan to move all of the HTTPS tests to WebListener (#4719).
For HTTP (no-S) tests that are currently pointed to httpbin I plan to pick the one (possibly /get), create the same functionality in WebListener, and move all the tests that use it to WebListener. Before I begin, though, I'd like to just confirm that this is the right path to take. I figured I would start moving one httpbin target URL at a time and then work on moving the HttpListener tests one URL at a time. The end goal being that all HTTP/HTTPS tests will run against WebListener and HttpListener can be retired.
markekraus
on 31 Aug 2017
@markekraus sounds right to me. Thanks!
SteveL-MSFT
on 31 Aug 2017
@SteveL-MSFT How should I do the PR's for these? Should I create a One PR for adding the WebListener functionality and another for switching the tests it replaces? Or just lump the WebListener functionality in with the test changes in a single PR for each URL replacement?
markekraus
on 1 Sep 2017
@markekraus I think it would be better to have both the WebListener changes and test code changes together for each URL replacement so the reviewers can understand what's happening on both the client side and server side
SteveL-MSFT
on 1 Sep 2017
after #5665 is merged, there is only 4 tests which rely on http://http.lee.io/method, all other tests rely on WebListener or HttpListener.
markekraus
on 10 Dec 2017
@markekraus Thank you for all your hard work!
And everyone else that helped review these.
TravisEz13
on 20 Dec 2017
@markekraus Many thanks! :beers:
iSazonov
on 20 Dec 2017
Related issues
JohnLBevan
路
3Comments
pcgeek86
路
3Comments
mklement0
路
3Comments
pcgeek86
路
3Comments
ajensenwaud
路
3Comments
Most helpful comment
@TravisEz13 I plan to move all of the HTTPS tests to WebListener (#4719).
For HTTP (no-S) tests that are currently pointed to httpbin I plan to pick the one (possibly /get), create the same functionality in WebListener, and move all the tests that use it to WebListener. Before I begin, though, I'd like to just confirm that this is the right path to take. I figured I would start moving one httpbin target URL at a time and then work on moving the HttpListener tests one URL at a time. The end goal being that all HTTP/HTTPS tests will run against WebListener and HttpListener can be retired.