Frameworkbenchmarks: aspnetcore: setup-plaintext.sh: line 2: $'\r': command not found
OS (Please include kernel version)
Windows 10 1709 + Docker for Windows
Expected Behavior
Test run finishes without errors
Actual Behavior
Test run aborts with errors
Steps to reproduce behavior
docker network create tfb
docker run -it --network=tfb -v /var/run/docker.sock:/var/run/docker.sock --mount type=bind,source=/d/workspaces/FrameworkBenchmarks,target=/FrameworkBenchmarks techempower/tfb --mode verify --test aspnetcore
Other details and logs
I tried to run tests with docker (on Docker for Windows). It worked fine with the gemini test, but when I run aspnetcore I consistently get this error:
...
techempower/aspnetcore-base:0.1: A command will be run during the install process that will improve project restore speed and enable offline access. It will take up to a minute to complete.
techempower/aspnetcore-base:0.1: Processing triggers for libc-bin (2.23-0ubuntu10) ...
techempower/aspnetcore-base:0.1: Removing intermediate container c6b1faa4320b
techempower/aspnetcore-base:0.1: ---> 02d9b794b866
techempower/aspnetcore-base:0.1: Step 11/11 : ENV PATH="/root/.dotnet:${PATH}"
techempower/aspnetcore-base:0.1: ---> Running in 8c4614ff1ee9
techempower/aspnetcore-base:0.1: Removing intermediate container 8c4614ff1ee9
techempower/aspnetcore-base:0.1: ---> 26ea07c0e5f5
techempower/aspnetcore-base:0.1: [Warning] One or more build-args [TFB_DATABASE MAX_CONCURRENCY] were not consumed
techempower/aspnetcore-base:0.1: Successfully built 26ea07c0e5f5
techempower/aspnetcore-base:0.1: Successfully tagged techempower/aspnetcore-base:0.1
aspnetcore: Step 1/2 : FROM techempower/aspnetcore-base:0.1
aspnetcore: ---> 26ea07c0e5f5
aspnetcore: Step 2/2 : CMD bash setup-plaintext.sh
aspnetcore: ---> Running in c019dee034a1
aspnetcore: Removing intermediate container c019dee034a1
aspnetcore: ---> b2560ba16070
aspnetcore: [Warning] One or more build-args [TFB_DATABASE MAX_CONCURRENCY] were not consumed
aspnetcore: Successfully built b2560ba16070
aspnetcore: Successfully tagged techempower/tfb.test.aspnetcore:latest
aspnetcore: setup-plaintext.sh: line 2: $'\r': command not found
aspnetcore: setup-plaintext.sh: line 25: syntax error: unexpected end of file
aspnetcore: ERROR: Expected techempower/tfb.test.aspnetcore to be running container
aspnetcore: ERROR: One or more expected docker container exited early
================================================================================
Verification Summary
--------------------------------------------------------------------------------
| aspnetcore
| NO RESULTS (Did framework launch?)
================================================================================
Time to complete: 66 seconds
Results are saved in /FrameworkBenchmarks/results/20180406211444
discostu105
All 20 comments
This is likely due to creating the file on Windows which will use crlf for line endings when linux systems are expecting just lf.
Try setting the line endings to just lf and retest please.
EDIT: Excellent bug report, btw. Thanks for taking the time.
msmith-techempower
on 6 Apr 2018
I just tried that for setup-plaintext.sh and run-linux.sh and it worked. I guess this needs to be fixed in all .sh files? Wondering how this works for others?
discostu105
on 6 Apr 2018
Are you running this from cmd.exe?
msmith-techempower
on 6 Apr 2018
Oh, I bet it has to do with git's setting for crlf
I have set git config --global core.autocrlf input in the past, so when I clone the repository I get lf endings. If you have not set this, then you will clone and convert to crlf and when you commit, it stores them as lf in our repo.
Can you try setting autocrlf to input, clone again, and return with results, please?
msmith-techempower
on 6 Apr 2018
Yes, I'm using cmd.exe
Setting autocrlf to true does not do the trick. It needs to be set to false!
So, after running git config --global core.autocrlf false and cloning the repo again, it works fine! Thanks for your help. I guess this could be worth mentioning as caveat for Docker for Windows people :).
discostu105
on 6 Apr 2018
That's a good point; I'll update the docs right now.
msmith-techempower
on 6 Apr 2018
aspnetcore tests now ran through, but 100% 500 reponses :(. But I guess that deserves another issue :)
discostu105
on 6 Apr 2018
>_< Let's leave this issue open for now as you find more.
For context, I am able to clone the repo with autocrlf set to input and run aspnetcore locally in cmd.exe.
msmith-techempower
on 7 Apr 2018
Ok. I just don't know how to find out more about the 500 errors. What info do you need?
This is the end of the console output:
...
---------------------------------------------------------
Concurrency: 16384 for plaintext
wrk "-H 'Host: tfb-server' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive'" --latency -d 15 -c 16384 --timeout 8 -t 2 http://tfb-server:8080/plaintext -s /pipeline.lua -- 16
---------------------------------------------------------
Running 15s test @ http://tfb-server:8080/plaintext
2 threads and 16384 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 0.00us 0.00us 0.00us -nan%
Req/Sec 0.00 0.00 0.00 -nan%
Latency Distribution
50% 0.00us
75% 0.00us
90% 0.00us
99% 0.00us
0 requests in 20.59s, 0.00B read
Requests/sec: 0.00
Transfer/sec: 0.00B
STARTTIME 1523051275
ENDTIME 1523051296
Benchmark results:
{'results': [{'5xx': 85601,
'endTime': 1523051239,
'startTime': 1523051224,
'totalRequests': 85601},
{'5xx': 109374,
'connect': 0,
'endTime': 1523051256,
'read': 2576,
'startTime': 1523051241,
'timeout': 0,
'totalRequests': 109374,
'write': 0},
{'5xx': 106186,
'connect': 0,
'endTime': 1523051273,
'read': 8954,
'startTime': 1523051258,
'timeout': 0,
'totalRequests': 106186,
'write': 0},
{'endTime': 1523051296,
'startTime': 1523051275,
'totalRequests': 0}]}
Complete
================================================================================
Parsing Results ...
================================================================================
fatal: Not a git repository (or any of the parent directories): .git
Cannot count lines of code for ASP.NET Core - no 'source_code' file
================================================================================
Verification Summary
--------------------------------------------------------------------------------
| aspnetcore
| plaintext : PASS
================================================================================
Time to complete: 147 seconds
Results are saved in /FrameworkBenchmarks/results/20180406214555
That looks broken to me. I ran this locally and it benchmarked fine.
Try this:
tfb --mode debug --test aspnetcore
This will put you into debug mode against which you can run tests.
First try to curl a url:
docker run --network=tfb techempower/tfb.wrk curl http://tfb-server:8080/plaintext
If that doesn't work, hopefully it would give you some sort of evidence as to why.
msmith-techempower
on 7 Apr 2018
I ran
docker run -it --network=tfb -v /var/run/docker.sock:/var/run/docker.sock --mount type=bind,source=/d/workspaces/FrameworkBenchmarks,target=/FrameworkBenchmarks techempower/tfb --mode debug --test aspnetcore
then this in another console:
docker run --network=tfb techempower/tfb.wrk curl http://tfb-server:8080/plaintext
and got this output:
D:\workspaces\techempower.git>docker run --network=tfb techempower/tfb.wrk curl http://tfb-server:8080/plaintext
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13 100 13 0 0 2551 0 --:--:-- --:--:-- --:--:-- 2600Hello, World!
Looks good IMHO. ran the full test again with
docker run -it --network=tfb -v /var/run/docker.sock:/var/run/docker.sock --mount type=bind,source=/d/workspaces/FrameworkBenchmarks,target=/FrameworkBenchmarks techempower/tfb --mode benchmark --test aspnetcore
but I still have 100% 500ds.
discostu105
on 7 Apr 2018
That is extremely odd. Your first test actually returned the plaintext response as expected, but the suite couldn't get it working.
msmith-techempower
on 9 Apr 2018
Could you copy all the output from the test and upload it as a gist for review? I'll take a look.
tfb --test aspnetcore > test.log (or however you redirect stdout on windows)
msmith-techempower
on 9 Apr 2018
Sure. Here you go.
I ran:
docker run -it --network=tfb -v /var/run/docker.sock:/var/run/docker.sock --mount type=bind,source=/d/workspaces/FrameworkBenchmarks,target=/FrameworkBenchmarks techempower/tfb --mode benchmark --test aspnetcore > test.log
Here is test.log as well as the results directory: https://www.dropbox.com/sh/vi4om6rciz9hg2b/AACJjhPRfCZt0xNdRJkCOoOna?dl=0
discostu105
on 9 Apr 2018
I am able to reproduce the 'all 5xxs' bug and am investigating locally, and will post back with anything I find.
msmith-techempower
on 9 Apr 2018
I think this is a bug. Opening a PR.
msmith-techempower
on 9 Apr 2018
I have opened #3522 to address this issue.
I was able to replicate the 5xxs bug on my local, and this has resolved it (as well as some other issues we were experiencing with benchmarking on Citrine like api_hour - ping @michaelhixson)
msmith-techempower
on 9 Apr 2018
@discostu105 You can test this out immediately, if you want. I have published the updated techempower/tfb.wrk image and you can try benchmarking aspnetcore immediately to validate my fix. Just pull latest and it should work in the toolset automatically docker pull techempower/tfb.wrk
msmith-techempower
on 9 Apr 2018
@msmith-techempower just tried and it works! Great, thanks for quick help! :)
discostu105
on 9 Apr 2018
@discostu105 No, thank you. This was a legitimate bug that may have had implications reaching far and wide, and it is very likely it would have taken us a long time to find it without your help.
msmith-techempower
on 9 Apr 2018
Related issues
Kaliumhexacyanoferrat
路
4Comments
joanhey
路
6Comments
emilypi
路
8Comments
joanhey
路
4Comments
MatheusRV
路
3Comments
Most helpful comment
@msmith-techempower just tried and it works! Great, thanks for quick help! :)