http.get hang no response

[cwchiu]

enviroment

• windows 10
• dart 2.2.1
• http 0.12.0+2

code

import 'package:http/http.dart' as http;

void test() async {
  final url = 'http://webpac.ksml.edu.tw/maintain/bookDetailAssdataAjax.do?id=1283659&viewDetailType=Details';
  final ret = await http.get(url);// hang here
  print(ret.statusCode);
}
main() {
  test();
}

curl 7.51.0

curl -v "http://webpac.ksml.edu.tw/maintain/bookDetailAssdataAjax.do?id=1283659&viewDetailType=Details"

output

D:\test>curl -v "http://webpac.ksml.edu.tw/maintain/bookDetailAssdataAjax.do?id=1283659&viewDetailType=Details"
* STATE: INIT => CONNECT handle 0x6000638c0; line 1407 (connection #-5000)
* Added connection 0. The cache now contains 1 members
*   Trying 163.32.124.11...
* TCP_NODELAY set
* STATE: CONNECT => WAITCONNECT handle 0x6000638c0; line 1460 (connection #0)
* Connected to webpac.ksml.edu.tw (163.32.124.11) port 80 (#0)
* STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x6000638c0; line 1567 (connection #0)
* Marked for [keep alive]: HTTP default
* STATE: SENDPROTOCONNECT => DO handle 0x6000638c0; line 1585 (connection #0)
> GET /maintain/bookDetailAssdataAjax.do?id=1283659&viewDetailType=Details HTTP/1.1
> Host: webpac.ksml.edu.tw
> User-Agent: curl/7.51.0
> Accept: */*
>
* STATE: DO => DO_DONE handle 0x6000638c0; line 1664 (connection #0)
* STATE: DO_DONE => WAITPERFORM handle 0x6000638c0; line 1791 (connection #0)
* STATE: WAITPERFORM => PERFORM handle 0x6000638c0; line 1801 (connection #0)
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 200 OK
< Cache-Control: no-store
< Pragma: no-cache
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nonsniff
< X-XSS-Protection: 1;mode=block
< Set-Cookie: webpacid=F9C8385C8FF05CF9A0B3A974EFF7C599; Domain=webpac.ksml.edu.tw; Path=/; HttpOnly
< Content-Type: text/html;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Sun, 21 Apr 2019 14:26:36 GMT
< Set-Cookie: webpacslb-163.32.124.11_80=ALAAAAAK; Expires=Mon, 22-Apr-2019 14:37:48 GMT; Path=/
< Set-Cookie: citrix_ns_id=IKO/jMmv32uc4DegdIH41YcfvEAA000; Domain=.ksml.edu.tw; Path=/; HttpOnly
<






























<table>
        <tr>
                <td style="word-break: keep-all; white-space: nowrap;">作者：</td>

                <td>瘋人院院長著</td>

        </tr>
</table>

<table>
        <tr>
                <td style="word-break: keep-all; white-space: nowrap;">出版社：</td>

                <td>旗標</td>

        </tr>
</table>

<table>
        <tr>
                <td style="word-break: keep-all; white-space: nowrap;">出版地：</td>

                <td>臺北市</td>

        </tr>
</table>

<table>
        <tr>
                <td style="word-break: keep-all; white-space: nowrap;">出版年：</td>

                <td>2011[民100]</td>

        </tr>
</table>

<table>
        <tr>
                <td style="word-break: keep-all; white-space: nowrap;">規格：</td>

                <td>1冊:彩圖;21公分</td>

        </tr>
</table>

<table>
        <tr>
                <td style="word-break: keep-all; white-space: nowrap;">語文：</td>

                <td>中文</td>

        </tr>
</table>

<table>
        <tr>
                <td style="word-break: keep-all; white-space: nowrap;">ISBN：</td>

                <td>9789574429226</td>

        </tr>
</table>

<table>
        <tr>
                <td style="word-break: keep-all; white-space: nowrap;">主題：</td>

                <td>行動電話 ； 電腦軟體</td>

        </tr>
</table>

<table>
        <tr>
                <td style="word-break: keep-all; white-space: nowrap;">作者：</td>

                <td>瘋人院院長</td>

        </tr>
</table>



* STATE: PERFORM => DONE handle 0x6000638c0; line 1965 (connection #0)
* multi_done
* Curl_http_done: called premature == 0
* Connection #0 to host webpac.ksml.edu.tw left intact

problem description

i try to trace code http-0.12.0+2/lib/src/response.dart

  static Future<Response> fromStream(StreamedResponse response) {       
    // cache-control: no-store
    // set-cookie: webpacid=EADC10D4246FDFCED8996D430BC7496B; Domain=webpac.ksml.edu.tw; Path=/; HttpOnly,webpacslb-163.32.124.11_80=ALAAAAAK; Expires=Mon, 22-Apr-2019 14:50:13 GMT; Path=/,citrix_ns_id=ldqMyiKuDY14cyx05VY/8j8YwpMA020; Domain=.ksml.edu.tw; Path=/; HttpOnly
    // transfer-encoding: chunked
    // date: Sun, 21 Apr 2019 14:39:01 GMT
    // content-encoding: gzip
    // content-type: text/html;charset=UTF-8
    // pragma: no-cache
    // x-frame-options: SAMEORIGIN
    // x-xss-protection: 1;mode=block
    // x-content-type-options: nonsniff
    // expires: Thu, 01 Jan 1970 00:00:00 GMT
    response.headers.forEach((k,v){
      print('$k: $v');
    }); 

    // here is Ok
    return response.stream.toBytes().then((body) {
      // cannot run here. how to resolve?
      return new Response.bytes(body, response.statusCode,
          request: response.request,
          headers: response.headers,
          isRedirect: response.isRedirect,
          persistentConnection: response.persistentConnection,
          reasonPhrase: response.reasonPhrase);
    });
  }

response.stream.toBytes() cannot pass to next step. How to fix it?

[jonasfj]

I can reproduce this under Linux using the same server as referenced here.

I see that curls says the connection was left intact, we have Transfer-Encoding: chunked... maybe there is a bug in the client... or the server, it's not obvious.

A) Can we write a standalone server in Dart that will reproduce this?
B) Maybe we should try some other HTTP clients, less robust than curl, to see if it's a server bug (doubtful)..

note. this could likely be something in the dart-sdk.

[natebosch]

note. this could likely be something in the dart-sdk.

If we can repro this with dart:io we should move the issue to the SDK.

[jonasfj]

@sortie, you might be interested...

[sortie]

import 'package:http/http.dart' as http;

main() async {
  final url = 'http://webpac.ksml.edu.tw/maintain/bookDetailAssdataAjax.do?id=1283659&viewDetailType=Details';
  final ret = await http.get(url);// hang here
  print(ret.statusCode);
}

doesn't work for me. However,

import 'dart:io';

main() async {
  final url = 'http://webpac.ksml.edu.tw/maintain/bookDetailAssdataAjax.do?id=1283659&viewDetailType=Details';
  final client = new HttpClient();
  final request = await client.getUrl(Uri.parse(url));
  final response = await request.close();
  print(response.statusCode);
}

works for me. I can confirm with strace that they send the same request:

GET /maintain/bookDetailAssdataAjax.do?id=1283659&viewDetailType=Details HTTP/1.1
user-agent: Dart/2.2 (dart:io)
accept-encoding: gzip
content-length: 0
host: webpac.ksml.edu.tw

So I would guess this is an issue in how package:http parses the response?

HTTP/1.1 200 OK
Cache-Control: no-store
Pragma: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nonsniff
X-XSS-Protection: 1;mode=block
Set-Cookie: webpacid=70CAF29A520ED02518534B914108B782; Domain=webpac.ksml.edu.tw; Path=/; HttpOnly
Content-Type: text/html;charset=UTF-8
Date: Tue, 30 Apr 2019 13:28:41 GMT
Set-Cookie: webpacslb-163.32.124.11_80=AMAAAAAK; Expires=Wed, 01-May-2019 13:40:02 GMT; Path=/
Set-Cookie: citrix_ns_id=GXy8DoZfToC3W052U9jDbzjlK/MA010; Domain=.ksml.edu.tw; Path=/; HttpOnly
Content-Encoding: gzip
Transfer-Encoding: chunked

[danish-khan-I]

No response from server on release APK .i.e works fine on debug
tried on different PC's still same.

var _news;
Future<String> getDataFromApi() async {
     http.Response response =  await http.get(
       Uri.encodeFull('http://m.jansamparklife.com/api/all'),
       headers: {
         "accept": "Application/json"
       }
       );
       this.setState((){
         _news = json.decode(response.body);
       });
       print(_news);
    return "success";   

  }

 @override
  void initState() {
    this.getDataFromApi();
    super.initState();
  } 



md5-e36a5a03389f37ba0e35bc7873208b75



C:\Users\danish>flutter doctor -v
[√] Flutter (Channel stable, v1.2.1, on Microsoft Windows [Version 10.0.17134.706], locale en-US)
    • Flutter version 1.2.1 at C:\flutter
    • Framework revision 8661d8aecd (3 months ago), 2019-02-14 19:19:53 -0800
    • Engine revision 3757390fa4
    • Dart version 2.1.2 (build 2.1.2-dev.0.0 0a7dcf17eb)

[!] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at C:\android
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • ANDROID_HOME = C:\android
    • Java binary at: C:\Program Files\Android\AndroidStudio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
    ! Some Android licenses not accepted.  To resolve this, run: flutter doctor --android-licenses

[√] Android Studio (version 3.3)
    • Android Studio at C:\Program Files\Android\AndroidStudio
    • Flutter plugin version 33.4.1
    • Dart plugin version 182.5215
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)

[√] VS Code (version 1.33.1)
    • VS Code at C:\Users\danish\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 2.26.0

[!] Connected device
    ! No devices available

! Doctor found issues in 2 categories.

[jmall]

Same here: No response from server on release APK, but works fine on debug
Here is my release: https://play.google.com/store/apps/details?id=com.shibes.meir.shibe

[natebosch]

No response from server on release APK, but works fine on debug

You have a problem with your manifest or similar. Someone from the flutter repo would be better able to help you solve this than us.

Existing request for better docs from flutter about this: https://github.com/flutter/flutter/issues/29688

[natebosch]

Closing since this seems related to flutter emulator, device, or permission issues.

[sortie]

@natebosch I was able to reproduce this with the standard Dart VM as mentioned above, so I don't think this is flutter related. Reopen?

[Sahba93]

I have the same problem. No matter what package I use i.e http, Client , Chopper, Dio, sometimes GET requests/responses hang. Both on debug and release mode. whenever I close the request it hangs less times but overall from ten times I call a get request there will be at least 3 times that the code hangs and doesn't even ends with the preset timeout exception.

[natebosch]

@Sahba93

No matter what package I use

This is a good indication we won't be able to take any action in this package to solve it. If you are using flutter please follow up with the flutter team with questions about your emulator or device at https://github.com/flutter/flutter/issues/29688.

[PonthepYJ]

Please check _Access-Control-Allow-Origin = _ in response header.for example in python
resp = Response(json.dumps(data))
resp.headers['Access-Control-Allow-Origin'] = ''
resp.headers['Content-Type'] = 'application/json; charset=utf-8'

This hang happens when run flutter on web. When your flutter app does not come from the same server of http.get url.

[natebosch]

I checked the example that we were unable to repro with dart:io . The root cause of the difference in behavior is that the response stream is never done. We can reproduce this with dart:io using the following:

import 'dart:io';

main() async {
  final url = 'http://webpac.ksml.edu.tw/maintain/bookDetailAssdataAjax.do?id=1283659&viewDetailType=Details';
  final client = new HttpClient();
  final request = await client.getUrl(Uri.parse(url));
  final response = await request.close();
  await response.drain(); // hangs here
  print('Done');
}

I believe the difference we are seeing is not a bug in this package. Rather it is a demonstration of the differences in the API between dart:io and package:http. The IO version gives you a Stream, which in this case never closes, but gives you access to the statusCode before the stream closes. The package:http version gives a Future that doesn't complete until the body is available, and here the body is never available.

To get access to similar behavior asdart:io you can use the httpClient.send API which returns a StreamedResponse and will let you read statusCode before the body is complete. The pattern is a bit ugly because it also requires listening on the result and then canceling it, but I'm not sure that it'd be worth doing something to get around that...

import 'package:http/http.dart' as http;

main() async {
  final url =
      'http://webpac.ksml.edu.tw/maintain/bookDetailAssdataAjax.do?id=1283659&viewDetailType=Details';
  final client = http.Client();
  final request = http.Request('GET', Uri.parse(url));
  final ret = await client.send(request);
  print(ret.statusCode);
  ret.stream.listen((_) {}).cancel(); // Ugly, but allows the VM to shut down
}

If there is a bug here, it must be in the SDK. I'm not sure what the expected behavior of the response stream should be, it may be correct for it to never close, or it may be better for it to either error out, or to close anyway.

[zichangg]

I'll close the issue. This is the same issue as https://github.com/dart-lang/sdk/issues/40132, where parser encountered errors but not report correctly.
I've submitted my change. It is probably contained in next release.