Filedownloader: Several issues with chunked resources and each inputstream buffer very tiny.

[genkido]

When attempting to download a file which does not report it's filesize, the following happens:
-Callback is called several times over the limit
-Pausing the task works randomly (sometimes it pauses after some time, sometimes it doesn't stop at all)
-Sometimes the download won't report as completed even after it downloaded (progress callback keeps getting called with totalBytes = soFarBytes)

Sample urls:
http://feedproxy.google.com/~r/TeaintheTreetops/podcast/~5/8KgLZ5CRro8/TeaintheTreetops-Ep42-Passenger.mp3
http://jazzpianoschool.com/podlove/file/437/s/feed/c/jpspodcast2/043.mp3

Thanks for making this library!

[Jacksgong]

What version do you using? Is it 0.3.1?

[Jacksgong]

Thanks for your issue.

OK, I just tried URLs you provided in Demo Project SingleTaskTestActivity.java.

1. Callback is called several times over the limit

Yes, The transfer encoding of All URLs you provided is chunked, It means FileDownloader doesn't know its content-length, So we can't calculate the rate to call-back, so for this issue, it's it, Not BUG.

---

For 2 other amazing issues, I will check and finger out problems, It must only occur in the chunked resources.

[Jacksgong]

I finger out the reason.

First of all, you can 'fix' bellow tow issues through FileDownloader.disableAvoidDropFrame().

1. Pausing the task works randomly (sometimes it pauses after some time, sometimes it doesn't stop at all)
2. Sometimes the download won't report as completed even after it downloaded (progress callback keeps getting called with totalBytes = soFarBytes)

---

Why?

Because FileDownloader read a buffer(Size 4096bytes) from the input stream for URLs you provided, FileDownloader provides the size of the buffer is 4096bytes, but the input stream provides is very tiny most of the time(even less than 15bytes) in your case, so it is cause the count of the callback Message is very very huge(even hundreds) and with the AvoidDropFrame Function, all action/call-back is waiting in queue, So with the long waiting, you will feel everything goes wrong.

---

Solution

1. FileDownloader will provide the new interface to BaseDownloadTask to limit call-back through the progress interval bytes.
2. FileDownloader will protect the rate of the callback in some ways.

[genkido]

@Jacksgong I was using 0.3.1 indeed and testing with the Demo project.

Thanks a lot for the amazingly fast response and explanation!

Have a great week!

[Jacksgong]

Enhancement since v0.3.2