Filedownloader: Unable to download text file through Chunked transfer encoding

[sergiomartinez4]

Issue

1. What problem do you get?
I'm not able to download one text file (VTT file). Presumably, it's because the server is using Chunked transfer encoding.

2. Which version of FileDownloader are you using when you produce such problem?
Version 1.7.4

3. How to reproduce such problem?
Just try downloading the following VTT file. Please note that I added the VTT url to Bitly, because the url expires and I can keep updating it for testing purposes: http://bit.ly/2HtV0gP
You might need to get the redirect url first. Please let me know if the url expires.

4. Do you set FileDownloadLog.NEED_LOG=true?
Yes

5. Could you please reproduce this problem and provide all main process and :filedownloader process logcat
See the logcat below.

6. Can you fix it by yourself and request PR, if not, what's problem do you get when you try to fix it
Not at this moment

/////////////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////MAIN PROCESS LOG////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////////////
06-07 17:10:45.678 15132-15132/com.example.bugapp I/zygote64: Late-enabling -Xcheck:jni
06-07 17:10:45.786 15132-15150/com.example.bugapp D/libEGL: loaded /vendor/lib64/egl/libEGL_adreno.so
06-07 17:10:45.792 15132-15150/com.example.bugapp D/libEGL: loaded /vendor/lib64/egl/libGLESv1_CM_adreno.so
06-07 17:10:45.805 15132-15150/com.example.bugapp D/libEGL: loaded /vendor/lib64/egl/libGLESv2_adreno.so
06-07 17:10:45.864 15132-15171/com.example.bugapp D/OpenGLRenderer: HWUI GL Pipeline
06-07 17:10:45.900 15132-15171/com.example.bugapp I/Adreno: QUALCOMM build : 594927b, I916dfac403
Build Date : 10/11/17
OpenGL ES Shader Compiler Version: EV031.21.02.00
Local Branch : mybranch28618966
Remote Branch : quic/gfx-adreno.lnx.6.4.9-rel
Remote Branch : NONE
Reconstruct Branch : NOTHING
06-07 17:10:45.905 15132-15171/com.example.bugapp I/Adreno: PFP: 0x005ff087, ME: 0x005ff063
06-07 17:10:45.910 15132-15171/com.example.bugapp I/zygote64: android::hardware::configstore::V1_0::ISurfaceFlingerConfigs::hasWideColorDisplay retrieved: 1
06-07 17:10:45.910 15132-15171/com.example.bugapp I/OpenGLRenderer: Initialized EGL, version 1.4
06-07 17:10:45.910 15132-15171/com.example.bugapp D/OpenGLRenderer: Swap behavior 2
06-07 17:10:55.359 15132-15301/com.example.bugapp I/ODRM: Starting downloadOneByOne
06-07 17:10:55.373 15132-15301/com.example.bugapp D/FileDownloader.DownloadTask: setPath /storage/emulated/0/Android/data/com.example.bugapp/files/Download
06-07 17:10:55.374 15132-15301/com.example.bugapp D/FileDownloader.DownloadTask: setListener com.example.bugapp.util.DownloadIntentService$MyFileDownloadListener@f4140ac
06-07 17:10:55.374 15132-15301/com.example.bugapp V/FileDownloader.DownloadTaskHunter: call start Url[http://bcboltproduse1-a.akamaihd.net/media/v1/text/vtt/clear/5783296997001/9f8be68c-fd34-4b94-922e-ce97768dd48d/8d06ebef-49db-43fe-921a-c6a38b777a91/text.vtt?akamai_token=exp=1528428442~acl=/media/v1/text/vtt/clear/5783296997001/9f8be68c-fd34-4b94-922e-ce97768dd48d/8d06ebef-49db-43fe-921a-c6a38b777a91/text.vtt~hmac=77a5a4c6cb010b8594b9e55a7f35f3a6e0fda9f7713b752dd81815860882b896], Path[/storage/emulated/0/Android/data/com.example.bugapp/files/Download] Listener[com.example.bugapp.util.DownloadIntentService$MyFileDownloadListener@f4140ac], Tag[null]
06-07 17:10:55.377 15132-15303/com.example.bugapp V/FileDownloader.FileDownloadEventPool: setListener event.service.connect.changed
06-07 17:10:55.378 15132-15301/com.example.bugapp V/FileDownloader.DownloadTaskHunter: the task[-818857745] has been into the launch pool.
06-07 17:10:55.378 15132-15301/com.example.bugapp I/ODRM: Checking that atomic integer is ZERO
06-07 17:10:55.380 15132-15303/com.example.bugapp I/FileDownloader.FileDownloadProperties: init properties 0
load properties: http.lenient=FALSE; process.non-separate=FALSE; download.min-progress-step=65536; download.min-progress-time=2000; download.max-network-thread-count=3; file.non-pre-allocation=FALSE; broadcast.completed=FALSE; download.trial-connection-head-method=FALSE
06-07 17:10:55.381 15132-15303/com.example.bugapp D/FileDownloader.LostServiceConnectedHandler: Waiting for connecting with the downloader service... -818857745
06-07 17:10:55.383 15132-15303/com.example.bugapp D/FileDownloader.FileDownloadServiceUIGuard: bindStartByContext PlaybackTestApplication
06-07 17:10:55.398 15132-15303/com.example.bugapp D/FileDownloader.DownloadTaskHunter: free the task -818857745, when the status is 10
06-07 17:10:55.493 15132-15132/com.example.bugapp D/FileDownloader.FileDownloadServiceUIGuard: onServiceConnected ComponentInfo{com.example.bugapp/com.liulishuo.filedownloader.services.FileDownloadService$SeparateProcessService} com.liulishuo.filedownloader.i.IFileDownloadIPCService$Stub$Proxy@a935ff1
06-07 17:10:55.494 15132-15132/com.example.bugapp V/FileDownloader.FileDownloadEventPool: asyncPublishInNewThread event.service.connect.changed
06-07 17:10:55.495 15132-15321/com.example.bugapp V/FileDownloader.FileDownloadEventPool: publish event.service.connect.changed
06-07 17:10:55.496 15132-15321/com.example.bugapp D/FileDownloader.LostServiceConnectedHandler: The downloader service is connected.
06-07 17:10:55.496 15132-15321/com.example.bugapp V/FileDownloader.DownloadTaskHunter: call start Url[http://bcboltproduse1-a.akamaihd.net/media/v1/text/vtt/clear/5783296997001/9f8be68c-fd34-4b94-922e-ce97768dd48d/8d06ebef-49db-43fe-921a-c6a38b777a91/text.vtt?akamai_token=exp=1528428442~acl=/media/v1/text/vtt/clear/5783296997001/9f8be68c-fd34-4b94-922e-ce97768dd48d/8d06ebef-49db-43fe-921a-c6a38b777a91/text.vtt~hmac=77a5a4c6cb010b8594b9e55a7f35f3a6e0fda9f7713b752dd81815860882b896], Path[/storage/emulated/0/Android/data/com.example.bugapp/files/Download] Listener[com.example.bugapp.util.DownloadIntentService$MyFileDownloadListener@f4140ac], Tag[null]
06-07 17:10:55.497 15132-15321/com.example.bugapp V/FileDownloader.DownloadTaskHunter: the task[-818857745] has been into the launch pool.
06-07 17:10:55.498 15132-15322/com.example.bugapp D/FileDownloader.FileDownloadMessenger: notify begin [email protected]@458f944
06-07 17:10:55.498 15132-15322/com.example.bugapp V/FileDownloader.DownloadTaskHunter: filedownloader:lifecycle:start com.liulishuo.filedownloader.DownloadTaskHunter@46d0a2d by 11
06-07 17:10:55.499 15132-15322/com.example.bugapp V/FileDownloader.FileDownloadList: add list in all [email protected]@458f944 11 1
06-07 17:10:55.521 15132-15325/com.example.bugapp D/FileDownloader.MessageSnapshotGate: ~callback -818857745 old[11] new[1] 1
06-07 17:10:55.522 15132-15325/com.example.bugapp D/FileDownloader.FileDownloadMessenger: notify pending [email protected]@458f944
06-07 17:10:55.523 15132-15325/com.example.bugapp D/FileDownloader.MessageSnapshotGate: updateKeepFlow
06-07 17:10:55.524 15132-15325/com.example.bugapp D/FileDownloader.MessageSnapshotGate: ~callback -818857745 old[1] new[6] 1
06-07 17:10:55.524 15132-15325/com.example.bugapp D/FileDownloader.FileDownloadMessenger: notify started [email protected]@458f944
06-07 17:10:55.524 15132-15325/com.example.bugapp D/FileDownloader.MessageSnapshotGate: updateKeepFlow
06-07 17:10:55.729 15132-15325/com.example.bugapp D/FileDownloader.MessageSnapshotGate: ~callback -818857745 old[6] new[2] 1
06-07 17:10:55.730 15132-15325/com.example.bugapp D/FileDownloader.FileDownloadMessenger: notify connected [email protected]@458f944
06-07 17:10:55.731 15132-15325/com.example.bugapp D/FileDownloader.MessageSnapshotGate: updateKeepFlow
06-07 17:10:55.831 15132-15325/com.example.bugapp D/FileDownloader.MessageSnapshotGate: ~callback -818857745 old[2] new[-1] 1
06-07 17:10:55.832 15132-15325/com.example.bugapp V/FileDownloader.FileDownloadList: remove [email protected]@458f944 left -1 0
06-07 17:10:55.833 15132-15325/com.example.bugapp D/FileDownloader.FileDownloadMessenger: notify error [email protected]@458f944 com.liulishuo.filedownloader.exception.FileDownloadGiveUpRetryException: can't know the size of the download file, and its Transfer-Encoding is not Chunked either.
you can ignore such exception by add http.lenient=true to the filedownloader.properties
06-07 17:10:55.833 15132-15325/com.example.bugapp V/FileDownloader.DownloadTaskHunter: filedownloader:lifecycle:over com.liulishuo.filedownloader.DownloadTaskHunter@46d0a2d by -1
06-07 17:10:55.834 15132-15325/com.example.bugapp D/FileDownloader.MessageSnapshotGate: updateKeepFlow

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////FILEDOWNLOADER service LOG//////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
06-07 17:10:55.401 15304-15304/? I/zygote64: Late-enabling -Xcheck:jni
06-07 17:10:55.523 15304-15324/com.example.bugapp:filedownloader D/NetworkSecurityConfig: No Network Security Config specified, using platform default

[rantianhua]

This is a good issue, thanks.

In fact, the log info has a very clear explanation:

com.liulishuo.filedownloader.exception.FileDownloadGiveUpRetryException: can't know the size of the download file, and its Transfer-Encoding is not Chunked either.
you can ignore such exception by add http.lenient=true to the filedownloader.properties

It means that the server response doesn't contain Content-Length and Transfer-Encoding, so FileDownload doesn't know the file length. Also, you can ignore this issue by adding http.lenient=true to filedownloader.properties, and let me know this solves your problem or not.

[sergiomartinez4]

[rantianhua]

[sergiomartinez4]

[rantianhua]

[rantianhua]

[sergiomartinez4]

[rantianhua]

[sergiomartinez4]

[bcdarius]

[BCsl]

[BCsl]

[799837663]

[sergiomartinez4]