Sdk: Sometimes response can not be resolved.

Created on 13 Jan 2020 · 23Comments · Source: dart-lang/sdk

environment

OS: PopOs!
dart: 2.7.0
http: ^0.12.0+4

Problem

I've stumbled upon a small bug, where if the response is too big (or contains some characters http does not like, at least that's my guess) can't finish loading.

Here is the fully working code to the issue:

import 'dart:io';

import 'package:http/http.dart' as http;
import 'package:shelf/shelf.dart';
import 'package:shelf_router/shelf_router.dart';
import 'package:shelf/shelf_io.dart' as io;

const _hostname = 'localhost';

class SampleRouter {
  Handler get handler {
    final router = Router();

    router.get('/one', (Request req) async {
      final url =
          'http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Ema';
      var res = await http.Client().get(url);
      return Response.ok(res.body,
          headers: {HttpHeaders.contentTypeHeader: 'text/plain'});
    });

    router.get('/two', (Request req) async {
      final url = 'http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Em';
      var res = await http.Client().get(url);
      return Response.ok(res.body,
          headers: {HttpHeaders.contentTypeHeader: 'text/plain'});
    });

    return router.handler;
  }
}

void main() async {
  final service = SampleRouter();
  final server = await io.serve(service.handler, _hostname, 8080);
  print('Server running on localhost:${server.port}');
}

If you run the server by executing:

dart sample.dart

You'll have two entrypoints to the server

localhost:8080/one

which will work totally fine. And

localhost:8080/two

which will not finish loading (At least on my machine)
If you open both websites directly which i am trying to get the body from, you'll see that both are loading.

Thanks for your help :)

curl

http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Em

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
  <head>
   <meta http-equiv="Content-Language" content="de">
   <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
   <meta http-equiv="CACHE-CONTROL" content="no-cache">
   <meta http-equiv="PRAGMA" content="no-cache">   
   <title>Abfallkalender von Entsorgung-kommunal</title>
   <link rel="stylesheet" type="text/css"  href="http://213.168.213.236/bremereb/css/main.css">
  </head> 
<style>
.BAKStr1 {
    font-family: Arial, Helvetica, sans-serif; 
    font-size: 13px; 
    font-weight:bold;
    height: 21px;
    background-color:#CCCCCC;
}
.BAKStr2 {
    font-family: Arial, Helvetica, sans-serif; 
   font-size: 13px; 
   font-weight:bold;
    height: 21px;
}
.bakS {
   font-family: Arial, Helvetica, sans-serif;
   font-size: 12px;
   font-weight: bold;
   background-color: #CCCCCC;
   border: 1px outset #CCCCCC;
}
</style>
<script language="JavaScript"><!--
var cActive = '#fefeb8';
var cInActive = '#CCCCCC';
var cInActiveStreetA = '#CCCCCC';
var cInActiveStreetB = '#ffffff';

function MouseOver(obj)
{ obj.style.backgroundColor = cActive; }

function MouseOut(obj,i){ 
  if (i%2==1) obj.style.backgroundColor = cInActiveStreetA; 
  else obj.style.backgroundColor = cInActiveStreetB;
}


function gotoStart(){
 window.location.href="index.jsp";
}

function gotoStreet(streetName){
 window.location.href='hausnummer.jsp?strasse='+streetName;
}
//--></script>
<body bgproperties="fixed" topmargin="8" leftmargin="8" bgcolor="#ffffff"onload="">
<table border="0" cellpadding="0" cellspacing="0" width="600">
<tbody><tr>
<td align="left" bgcolor="#CCCCCC" valign="top">

<td background="strasse-Dateien/bgr_main.jpg" valign="top">
<table border="0" cellpadding="2" width="540">
<tbody><tr>
<td valign="top" width="3"></td>
<td valign="top">

  <!-- BEGIN: Strassen gefunden:-->

<h3>Bitte w&auml;hlen Sie Ihre Stra&szlig;e aus.</h3>
<table border="0" cellspacing="1" width="530" bgcolor="#FFFFFF">
<tr>
 <td valign="top">
 <table border="0" cellspacing="1" width="265">

  <tr>
   <td id="a1" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,1);" onClick="gotoStreet('Emanuel-Backhaus-Stra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a1'))" 
     onblur="MouseOut(window.document.getElementById('a1'))"
     href=hausnummer.jsp?strasse=Emanuel-Backhaus-Stra%DFe>Emanuel-Backhaus-Stra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a2" class='BAKStr2' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,2);" onClick="gotoStreet('Emanuelstra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a2'))" 
     onblur="MouseOut(window.document.getElementById('a2'))"
     href=hausnummer.jsp?strasse=Emanuelstra%DFe>Emanuelstra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a3" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,3);" onClick="gotoStreet('Embsener+Stra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a3'))" 
     onblur="MouseOut(window.document.getElementById('a3'))"
     href=hausnummer.jsp?strasse=Embsener+Stra%DFe>Embsener Stra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a4" class='BAKStr2' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,4);" onClick="gotoStreet('Emder+Stra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a4'))" 
     onblur="MouseOut(window.document.getElementById('a4'))"
     href=hausnummer.jsp?strasse=Emder+Stra%DFe>Emder Stra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a5" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,5);" onClick="gotoStreet('Emil-Richter-Stra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a5'))" 
     onblur="MouseOut(window.document.getElementById('a5'))"
     href=hausnummer.jsp?strasse=Emil-Richter-Stra%DFe>Emil-Richter-Stra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a6" class='BAKStr2' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,6);" onClick="gotoStreet('Emil-Sommer-Stra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a6'))" 
     onblur="MouseOut(window.document.getElementById('a6'))"
     href=hausnummer.jsp?strasse=Emil-Sommer-Stra%DFe>Emil-Sommer-Stra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a7" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,7);" onClick="gotoStreet('Emil-Trinkler-Stra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a7'))" 
     onblur="MouseOut(window.document.getElementById('a7'))"
     href=hausnummer.jsp?strasse=Emil-Trinkler-Stra%DFe>Emil-Trinkler-Stra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a8" class='BAKStr2' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,8);" onClick="gotoStreet('Emil-von-Behring-Stra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a8'))" 
     onblur="MouseOut(window.document.getElementById('a8'))"
     href=hausnummer.jsp?strasse=Emil-von-Behring-Stra%DFe>Emil-von-Behring-Stra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a9" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,9);" onClick="gotoStreet('Emil-Waldmann-Stra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a9'))" 
     onblur="MouseOut(window.document.getElementById('a9'))"
     href=hausnummer.jsp?strasse=Emil-Waldmann-Stra%DFe>Emil-Waldmann-Stra�e</a>
   </td>
  </tr>

 </table>
 </td>
 <td valign="top">
 <table border="0" cellspacing="1" width="265">

  <tr>
   <td id="a10" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,1);" onClick="gotoStreet('Emilienweg+%28KG+In+den+Hufen%29')">
    <a
     onfocus="MouseOver(window.document.getElementById('a10'))" 
     onblur="MouseOut(window.document.getElementById('a10'))"
     href=hausnummer.jsp?strasse=Emilienweg+%28KG+In+den+Hufen%29>Emilienweg (KG In den Hufen)</a>
   </td>
  </tr>

  <tr>
   <td id="a11" class='BAKStr2' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,2);" onClick="gotoStreet('Emmaberg')">
    <a
     onfocus="MouseOver(window.document.getElementById('a11'))" 
     onblur="MouseOut(window.document.getElementById('a11'))"
     href=hausnummer.jsp?strasse=Emmaberg>Emmaberg</a>
   </td>
  </tr>

  <tr>
   <td id="a12" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,3);" onClick="gotoStreet('Emmalene-Bulling-Stra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a12'))" 
     onblur="MouseOut(window.document.getElementById('a12'))"
     href=hausnummer.jsp?strasse=Emmalene-Bulling-Stra%DFe>Emmalene-Bulling-Stra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a13" class='BAKStr2' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,4);" onClick="gotoStreet('Emmastra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a13'))" 
     onblur="MouseOut(window.document.getElementById('a13'))"
     href=hausnummer.jsp?strasse=Emmastra%DFe>Emmastra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a14" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,5);" onClick="gotoStreet('Emmentaler+Stra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a14'))" 
     onblur="MouseOut(window.document.getElementById('a14'))"
     href=hausnummer.jsp?strasse=Emmentaler+Stra%DFe>Emmentaler Stra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a15" class='BAKStr2' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,6);" onClick="gotoStreet('Emscherweg')">
    <a
     onfocus="MouseOver(window.document.getElementById('a15'))" 
     onblur="MouseOut(window.document.getElementById('a15'))"
     href=hausnummer.jsp?strasse=Emscherweg>Emscherweg</a>
   </td>
  </tr>

  <tr>
   <td id="a16" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,7);" onClick="gotoStreet('Emslandstra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a16'))" 
     onblur="MouseOut(window.document.getElementById('a16'))"
     href=hausnummer.jsp?strasse=Emslandstra%DFe>Emslandstra�e</a>
   </td>
  </tr>

  <tr>
   <td id="a17" class='BAKStr2' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,8);" onClick="gotoStreet('Emteweg')">
    <a
     onfocus="MouseOver(window.document.getElementById('a17'))" 
     onblur="MouseOut(window.document.getElementById('a17'))"
     href=hausnummer.jsp?strasse=Emteweg>Emteweg</a>
   </td>
  </tr>

  <tr>
   <td id="a18" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,9);" onClick="gotoStreet('Emtinghauser+Weg')">
    <a
     onfocus="MouseOver(window.document.getElementById('a18'))" 
     onblur="MouseOut(window.document.getElementById('a18'))"
     href=hausnummer.jsp?strasse=Emtinghauser+Weg>Emtinghauser Weg</a>
   </td>
  </tr>

 </table>
 </td>
</table>

<p>&nbsp;
</td></tr></tbody></table></td></tr></tbody></table>
  <!-- END: Strassen gefunden:-->

</body>
</html>

http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Ema

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
  <head>
   <meta http-equiv="Content-Language" content="de">
   <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
   <meta http-equiv="CACHE-CONTROL" content="no-cache">
   <meta http-equiv="PRAGMA" content="no-cache">   
   <title>Abfallkalender von Entsorgung-kommunal</title>
   <link rel="stylesheet" type="text/css"  href="http://213.168.213.236/bremereb/css/main.css">
  </head> 
<style>
.BAKStr1 {
    font-family: Arial, Helvetica, sans-serif; 
    font-size: 13px; 
    font-weight:bold;
    height: 21px;
    background-color:#CCCCCC;
}
.BAKStr2 {
    font-family: Arial, Helvetica, sans-serif; 
   font-size: 13px; 
   font-weight:bold;
    height: 21px;
}
.bakS {
   font-family: Arial, Helvetica, sans-serif;
   font-size: 12px;
   font-weight: bold;
   background-color: #CCCCCC;
   border: 1px outset #CCCCCC;
}
</style>
<script language="JavaScript"><!--
var cActive = '#fefeb8';
var cInActive = '#CCCCCC';
var cInActiveStreetA = '#CCCCCC';
var cInActiveStreetB = '#ffffff';

function MouseOver(obj)
{ obj.style.backgroundColor = cActive; }

function MouseOut(obj,i){ 
  if (i%2==1) obj.style.backgroundColor = cInActiveStreetA; 
  else obj.style.backgroundColor = cInActiveStreetB;
}


function gotoStart(){
 window.location.href="index.jsp";
}

function gotoStreet(streetName){
 window.location.href='hausnummer.jsp?strasse='+streetName;
}
//--></script>
<body bgproperties="fixed" topmargin="8" leftmargin="8" bgcolor="#ffffff"onload="">
<table border="0" cellpadding="0" cellspacing="0" width="600">
<tbody><tr>
<td align="left" bgcolor="#CCCCCC" valign="top">

<td background="strasse-Dateien/bgr_main.jpg" valign="top">
<table border="0" cellpadding="2" width="540">
<tbody><tr>
<td valign="top" width="3"></td>
<td valign="top">

  <!-- BEGIN: Strassen gefunden:-->

<h3>Bitte w&auml;hlen Sie Ihre Stra&szlig;e aus.</h3>
<table border="0" cellspacing="1" width="530" bgcolor="#FFFFFF">
<tr>
 <td valign="top">
 <table border="0" cellspacing="1" width="265">

  <tr>
   <td id="a1" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,1);" onClick="gotoStreet('Emanuel-Backhaus-Stra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a1'))" 
     onblur="MouseOut(window.document.getElementById('a1'))"
     href=hausnummer.jsp?strasse=Emanuel-Backhaus-Stra%DFe>Emanuel-Backhaus-Stra�e</a>
   </td>
  </tr>

 </table>
 </td>
 <td valign="top">
 <table border="0" cellspacing="1" width="265">

  <tr>
   <td id="a2" class='BAKStr1' onMouseOver="MouseOver(this);" onMouseOut="MouseOut(this,1);" onClick="gotoStreet('Emanuelstra%DFe')">
    <a
     onfocus="MouseOver(window.document.getElementById('a2'))" 
     onblur="MouseOut(window.document.getElementById('a2'))"
     href=hausnummer.jsp?strasse=Emanuelstra%DFe>Emanuelstra�e</a>
   </td>
  </tr>

 </table>
 </td>
</table>

<p>&nbsp;
</td></tr></tbody></table></td></tr></tbody></table>
  <!-- END: Strassen gefunden:-->

</body>
</html>

area-library library-io

Source

kiesman99

👍1

Most helpful comment

@kapace I suggest to try to fix on the server end. Apparently, it doesn't follow RFCs. If that's not possible, you probably have to change the source code of HttpParser in the SDK.

Here is the place you need to change:
https://github.com/dart-lang/sdk/blob/67618b05426bee330eaba7a5e9ce55a7ecc0c77b/sdk/lib/_http/http_parser.dart#L775

Something like:

if (_index < _buffer!.length) {
  byte = _buffer![index + 1];
  // Check for invalid characters.
  // Skip if needed.
}

After the parser parses the chunk length, does a manual check for invalid characters.

I strongly discourage modifying the parser unless this is the only way to mitigate the problem.

zichangg on 19 Jun 2020

❤2

All 23 comments

It looks like the response stream is never closing. I think this is the same thing as what is happening in https://github.com/dart-lang/http/issues/270#issuecomment-539226402

I'm not sure if this is a bug in the SDK or not. cc @sortie

natebosch on 14 Jan 2020

Here is a repro using dart:io for this:

import 'dart:io';

void main() async {
  var client = HttpClient();
  final url =
      Uri.parse('http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Em');
  var request = await client.getUrl(url);
  var response = await request.close();
  print('Got response');
  await response.drain();
  print('Done');
  client.close();
}

We never reach 'Done' and the VM stays running. If I change the URI to strasse=Ema as in the above example we print 'Done' and exit.

natebosch on 14 Jan 2020

👍1

@natebosch Thank you for your reply.
Is there anything i can do right now to get this working? Maybe an alternative library that is more stable or something like that?

kiesman99 on 14 Jan 2020

Maybe I should mention that the website I am trying to fetch has an iso 8559-1 encoding.
I don't really know if this is important, but i wanted to mention this.

kiesman99 on 14 Jan 2020

@natebosch
I've discovered that neither

http://webpac.ksml.edu.tw/maintain/bookDetailAssdataAjax.do?id=1283659&viewDetailType=Details
(from https://github.com/dart-lang/http/issues/270)

nor

http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Em

has the attribute 'Content-Length' in the request header.

http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Ema

has this attribute and can be resolved. I am assuming that this is the issue here.

kiesman99 on 14 Jan 2020

Via

curl http://213.168.213.236/bremereb/bify/strasse.jsp\?strasse=Ema | wc -c

I get a size of 3040 which is correct regarding the content length emitted by the header.

Using the same method on strasse=Em

curl http://213.168.213.236/bremereb/bify/strasse.jsp\?strasse=Em | wc -c

I get a size of 8952.

Currently, I am trying to figure out if there is a possibility to inject the expected size of 8952 into the request. If this won't fail I am assuming http can't resolve unkown lengths.

Using this snippet

void main() async {
  var client = HttpClient();
  final url =
      Uri.parse('http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Em');
  client
      .getUrl(url)
      .then((HttpClientRequest request) {
    // Optionally set up headers...
    // Optionally write to the request object...
    // Then call close.
    return request.close();
  }).then((HttpClientResponse response) {
    // Process the response.
    return response.listen((el) {
      print(el.length);
    });
  }); 
}

I get all bytestreams that could be resolved:

1089
1388
5552
155

which sums up to 8184. That tells me that the last 768 bytes couldn't be fetched.

kiesman99 on 14 Jan 2020

Moving to the SDK repo since this can't be solve in the HTTP package.

natebosch on 14 Jan 2020

/cc @ZichangG

a-siva on 14 Jan 2020

@kiesman99
The hanging is a bug that parser didn't notify the exception to the right listener. I'll work on a fix to populate exceptions. But I'm not quite sure for the size difference. It is very likely to be something with encoding scheme.

zichangg on 25 Jan 2020

👍1

@ZichangG
Thank you first of all :)

If I can help in any way, let me know.

Regarding your guess to the encoding:
The Website I am trying to fetch has a iso 8559-1 encoding. Is there anything i have to tell the HttpClient manually? This information is also provided by the websites header.

Thank you!

kiesman99 on 25 Jan 2020

For size of response, here is an answer:

Read the response data, as provided by the library, as small chunks (e.g. by chunks of at most 8192 bytes, a very traditional value). Don't bother thinking about "chunks" as seen on the wire; this is the job of the HTTP library, not yours.

This explained why response is truncated.
The receiver buffer does have correct length. But chunk size received by parser is 0x1FF8(which is 8184) compared to the real size 0x22F8(8952 in decimal).

zichangg on 27 Jan 2020

👍1

Thanks @ZichangG

kiesman99 on 4 Feb 2020

@ZichangG Is there any possibility, that i can use your fix right away, or do i have to wait until the next release?

Thank you in advance!

kiesman99 on 4 Feb 2020

You would have to wait until next release.

Actually there is an easy work around. You can apply following lines of my change. It should solve the problem. nnbd sdk and non_nnbd sdk are smiliar. Depending on which sdk you are using, you need to pick those lines and apply locally.

sdk_nnbd/http_parser.dart:
Line 317-323

      if (_state >= _State.CHUNK_SIZE_STARTING_CR && _state <= _State.BODY) {
        _state = _State.FAILURE;
        _reportBodyError(e, s);
      } else {
        _state = _State.FAILURE;
        _reportError(e, s); // don't use _reportHttpError, If you didn't copy the rest of parts.
      }

line 573:

if (byte < 0x30 || byte > 0x39 || _statusCodeLength > 3) {

line 1074-1081

  void _reportBodyError(error, [stackTrace]) {
    _socketSubscription?.cancel();
    _state = _State.FAILURE;
    _bodyController.addError(error, stackTrace);
    // In case of drain(), error event will close the stream.
    _bodyController?.close();
  }

zichangg on 4 Feb 2020

👍1

which version fix this?

taojoe on 17 Mar 2020

You can click into the commit shown above. Versions are displayed below the description of PR.
Dev.14.0 - Dev.7.0 all contains the fix.

zichangg on 17 Mar 2020

Hey there @ZichangG.
Thanks again for investigating the issue.

Did you try to get a response from the following url mentioned in the issue:
http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Em

When I am trying to get a response I get some errors and I don't know how to resolve them.

void main() async {
  var client = HttpClient();
  final url =
      Uri.parse('http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Em');
  var response = await client.getUrl(url).then((HttpClientRequest request) {

    return request.close();
  });
  print('Got response');
  print('Response length: ${response.contentLength}');
  await response.transform(Latin1Decoder()).listen(print);
  print('Done');
  client.close();
}

Here I get the following error message:

Unhandled exception:
HttpException: Failed to parse HTTP, 32 is expected to be a Hex digit, uri = http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Em
#0      _HttpIncoming.listen.<anonymous closure> (dart:_http/http_impl.dart:164:7)
#1      _invokeErrorHandler (dart:async/async_error.dart:19:29)
#2      _HandleErrorStream._handleError (dart:async/stream_pipe.dart:288:9)
#3      _ForwardingStreamSubscription._handleError (dart:async/stream_pipe.dart:170:13)
#4      _RootZone.runBinaryGuarded (dart:async/zone.dart:1328:10)
#5      _BufferingStreamSubscription._sendError.sendError (dart:async/stream_impl.dart:358:15)
#6      _BufferingStreamSubscription._sendError (dart:async/stream_impl.dart:376:16)
#7      _BufferingStreamSubscription._addError (dart:async/stream_impl.dart:275:7)
#8      _SyncStreamControllerDispatch._sendError (dart:async/stream_controller.dart:777:19)
#9      _StreamController._addError (dart:async/stream_controller.dart:657:7)
#10     _StreamController.addError (dart:async/stream_controller.dart:609:5)
#11     _HttpParser._reportBodyError (dart:_http/http_parser.dart:1068:21)
#12     _HttpParser._parse (dart:_http/http_parser.dart:324:9)
#13     _HttpParser._onData (dart:_http/http_parser.dart:813:5)
#14     _RootZone.runUnaryGuarded (dart:async/zone.dart:1316:10)
#15     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#16     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:266:7)
#17     _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:773:19)
#18     _StreamController._add (dart:async/stream_controller.dart:649:7)
#19     _StreamController.add (dart:async/stream_controller.dart:595:5)
#20     _Socket._onData (dart:io-patch/socket_patch.dart:1873:41)
#21     _RootZone.runUnaryGuarded (dart:async/zone.dart:1316:10)
#22     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#23     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:266:7)
#24     _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:773:19)
#25     _StreamController._add (dart:async/stream_controller.dart:649:7)
#26     _StreamController.add (dart:async/stream_controller.dart:595:5)
#27     new _RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:1418:33)
#28     _NativeSocket.issueReadEvent.issue (dart:io-patch/socket_patch.dart:929:14)
#29     _microtaskLoop (dart:async/schedule_microtask.dart:43:21)
#30     _startMicrotaskLoop (dart:async/schedule_microtask.dart:52:5)
#31     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
#32     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:169:5)

And using the following snippet:

void main() async {
  var res = await http.get('http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Em');
  print(res.body);
}

I get:

Unhandled exception:
type '(HttpException) => Null' is not a subtype of type '(dynamic) => dynamic'
#0      _invokeErrorHandler (dart:async/async_error.dart:18:23)
#1      _HandleErrorStream._handleError (dart:async/stream_pipe.dart:288:9)
#2      _ForwardingStreamSubscription._handleError (dart:async/stream_pipe.dart:170:13)
#3      _RootZone.runBinaryGuarded (dart:async/zone.dart:1328:10)
#4      _BufferingStreamSubscription._sendError.sendError (dart:async/stream_impl.dart:358:15)
#5      _BufferingStreamSubscription._sendError (dart:async/stream_impl.dart:376:16)
#6      _BufferingStreamSubscription._addError (dart:async/stream_impl.dart:275:7)
#7      _ForwardingStreamSubscription._addError (dart:async/stream_pipe.dart:139:11)
#8      _addErrorWithReplacement (dart:async/stream_pipe.dart:190:8)
#9      _HandleErrorStream._handleError (dart:async/stream_pipe.dart:293:11)
#10     _ForwardingStreamSubscription._handleError (dart:async/stream_pipe.dart:170:13)
#11     _RootZone.runBinaryGuarded (dart:async/zone.dart:1328:10)
#12     _BufferingStreamSubscription._sendError.sendError (dart:async/stream_impl.dart:358:15)
#13     _BufferingStreamSubscription._sendError (dart:async/stream_impl.dart:376:16)
#14     _BufferingStreamSubscription._addError (dart:async/stream_impl.dart:275:7)
#15     _SyncStreamControllerDispatch._sendError (dart:async/stream_controller.dart:777:19)
#16     _StreamController._addError (dart:async/stream_controller.dart:657:7)
#17     _StreamController.addError (dart:async/stream_controller.dart:609:5)
#18     _HttpParser._reportBodyError (dart:_http/http_parser.dart:1068:21)
#19     _HttpParser._parse (dart:_http/http_parser.dart:324:9)
#20     _HttpParser._onData (dart:_http/http_parser.dart:813:5)
#21     _RootZone.runUnaryGuarded (dart:async/zone.dart:1316:10)
#22     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#23     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:266:7)
#24     _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:773:19)
#25     _StreamController._add (dart:async/stream_controller.dart:649:7)
#26     _StreamController.add (dart:async/stream_controller.dart:595:5)
#27     _Socket._onData (dart:io-patch/socket_patch.dart:1873:41)
#28     _RootZone.runUnaryGuarded (dart:async/zone.dart:1316:10)
#29     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#30     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:266:7)
#31     _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:773:19)
#32     _StreamController._add (dart:async/stream_controller.dart:649:7)
#33     _StreamController.add (dart:async/stream_controller.dart:595:5)
#34     new _RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:1418:33)
#35     _NativeSocket.issueReadEvent.issue (dart:io-patch/socket_patch.dart:929:14)
#36     _microtaskLoop (dart:async/schedule_microtask.dart:43:21)
#37     _startMicrotaskLoop (dart:async/schedule_microtask.dart:52:5)
#38     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
#39     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:169:5)

I currently use 2.8.0-dev.15.0.

Thank you and have a nice day!

kiesman99 on 18 Mar 2020

Because some characters are not valid utf8 characters. So parser doesn't know how to parse the response.

zichangg on 18 Mar 2020

Even if i just try to get the List of Int that i retrieve from the Stream subscription?

HttpClient client = new HttpClient();
client.getUrl(Uri.parse('http://213.168.213.236/bremereb/bify/strasse.jsp?strasse=Em'))
    .then((HttpClientRequest request) {
      return request.close();
    })
    .then((HttpClientResponse response) {
        response.listen(print);
    });
}

(Thanks for the quick response btw)

kiesman99 on 18 Mar 2020

This is the tricky part.
If you try to get the body of http response, it ideally doesn't need parser to work.
But what happened here is chunked encoding is used, parser get involved to translate the chunk size. The malformed characters mislead the parser to believe those bytes are a chunk size(but it is not). Parser then start to parse those digits and fail with exceptions.

zichangg on 18 Mar 2020

Mh, so as far as I understand i can do nothing about that right?

I don't have control over the web page and how it transmits it's data.

I might try to implement libcurl via ffi, so that i can fetch the body of the webpage. I've already written my application in python and there it does work pretty well. I just wanted to migrate the project to dart, because I like the language very much.

You're doing an awesome job with the dart language if i am allowed to mention that :)

kiesman99 on 18 Mar 2020

Hello,

I'm encountering a very similar problem, but in my case, the server has a bad chunk length which included an invalid character ("<length><space character><cr><lf>"). I upgraded to dart-sdk 2.8 which includes @zichangg fix but this request results in a HttpException now. Is there anyway I could monkey patch the parser to ignore that invalid space character after the length?

Thanks for your hard work!

kapace on 19 Jun 2020

@kapace I suggest to try to fix on the server end. Apparently, it doesn't follow RFCs. If that's not possible, you probably have to change the source code of HttpParser in the SDK.

Here is the place you need to change:
https://github.com/dart-lang/sdk/blob/67618b05426bee330eaba7a5e9ce55a7ecc0c77b/sdk/lib/_http/http_parser.dart#L775

Something like:

if (_index < _buffer!.length) {
  byte = _buffer![index + 1];
  // Check for invalid characters.
  // Skip if needed.
}

After the parser parses the chunk length, does a manual check for invalid characters.

I strongly discourage modifying the parser unless this is the only way to mitigate the problem.

zichangg on 19 Jun 2020

❤2

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Customers would like to generate JS interop bindings

matanlurey · 3Comments

Infix function syntax

DartBot · 3Comments

Supply an easy way to pipe shell processes or accept stdin in Process.runSync

xster · 3Comments

dart:io HttpRequest won't receive data on windows 10 linux subsystem

rinick · 3Comments

Tree-shaking and inlining

ranquild · 3Comments