Fetch: Add support for groupProgress

[daniellAlgar]

Hi @tonyofrancis!

Maybe I'm missing something here but.. I can see that you have a FetchGroupInfo#groupDownloadProgress(), but I can't see that it is ever used. And I can't find how to access it?

I would expect there to be a fetch.groupProgress(groupId: Int). Is this value/functionality possible to access somewhere?

Best regards,
Daniell Algar

[tonyofrancis]

@daniellAlgar
There are two ways to get group progress and group information. The first one is attaching a FetchGroupListener.

  fetch.addListener(new AbstractFetchGroupListener() {
            @Override
            public void onProgress(int groupId, @NotNull Download download, long etaInMilliSeconds, long downloadedBytesPerSecond, @NotNull FetchGroup fetchGroup) {
                super.onProgress(groupId, download, etaInMilliSeconds, downloadedBytesPerSecond, fetchGroup);
            }

            //.... other methods
        });

The other way is getting a FetchGroup and attaching observers to it.

        fetch.getFetchGroup(groupId, new Func<FetchGroup>() {
            @Override
            public void call(@NotNull FetchGroup fetchGroup) {

                fetchGroup.addFetchGroupObservers(new AbstractFetchGroupObserver() {
                    @Override
                    public void onChanged(@NotNull List<? extends Download> data, @NotNull Download triggerDownload, @NotNull Reason reason) {
                        super.onChanged(data, triggerDownload, reason);
                    }
                });
            }
        });

Note: FetchGroup objects are live objects. If a change happens in the group, the FetchGroup object will always reflect this change without having to query it again. The attached observers will always return the download that caused a change to the group and the reason it changed. Also not that the fetch.getFetchGroup method always returns a fetchgroup object even if that group does not exist. The library is smart enough to create the group and update it if need be.

Sorry for the lack of guides for this library. Pretty much a one man team with contributions from others. Hope this helps.

[daniellAlgar]

@tonyofrancis This is awesome. Thank you very much!

Just a follow up question: AbstractFetchGroupListener weights in on queued (not started) downloads as well I guess? So if we download 10 objects, but limit the concurrency to 1, we will still get the full progress for all 10 objects?

Thanks again!

[tonyofrancis]

@daniellAlgar Each of the group methods on the AbstractFetchGroupListener should return a Fetchgroup object. That Fetchgroup object will always have the updated group information/downloads. Say you insert 10 downloads to Fetch with concurrent limit 1. All downloads belong to the same group. The group information will show 1 being downloaded and the others with a status of queued. etc.

[daniellAlgar]

@tonyofrancis Yes, great 👍 Although I can't seem to get a smooth progress/eta. I have .setDownloadConcurrentLimit(100).. Doesn't help.. fetchGroup.downloads.sumByDouble { it.etaInMilliSeconds.toDouble() }, doesn't help.. Lots of files (images) gets 0 in eta. And I use fetchGroup.groupDownloadProgress.

So instead I'm trying to make Fetch download my largest files first (videos), so that I get a large time to begin with (rather than 0). But I didn't have much luck with that either. It do help somewhat to use Downloader.FileDownloaderType.SEQUENTIAL on my downloader (OkHttp). But it's still quite far from smooth.

Do you have any thoughts on this?

Thanks in advance!

[daniellAlgar]

I ended up doing

val summedBytes = fetchGroup.downloads.sumByDouble { it.downloaded.toDouble() }
val totalBytes = fetchGroup.downloads.sumByDouble { it.total.toDouble() }

val groupProgress = summedBytes / totalBytes

[daniellAlgar]

In your FetchGroupInfo, where you calculate

override val groupDownloadProgress: Int
    get() {
        val progressSum = downloads.sumBy { it.progress }
        return  progressSum / downloads.size
    }

don't you want to calculate it on bytes instead? The above will always give a resolution of downloads.size as best, right?

I guess the underlying problem is that you don't know all the bytes, until you start all downloads? So the above might be the best default calculation. But would it make sense to have a "download all files" concept, where if true, you could return a better progress based on bytes?

(and this is the tiniest of comments but: there's one extra space after return :) )

[tonyofrancis]

@daniellAlgar The sever may return -1 for a download's contentLength. That will cause an inaccurate reporting for the downloads progress. Mhmmmm. What I may be able to do is check if all the downloads have a proper contentLength. If every thing checks out calculate the progress based on bytes if not default to the above code. What you think? May even fix that space issue whiles im at it. lol

[daniellAlgar]

@tonyofrancis sounds like a plan! At least worth trying.

Though it sounds like this approach could cause the progress to go backwards, when jumping between how it's calculated? And that's a no no. So you'll have to add logic for locking it at the greatest so far (as minimum)? Unless one adds a Download to the group, then it may go back in progress once.

[tonyofrancis]

@daniellAlgar valid points. mhmm I will keep this issue open for more brain storming. In the mean time it would be better to calculate group progress manually in your case.

[yohane55]

one additional thing is that when the group downloads is empty (for some reason), it does crash throwing ArithmeticException and to prevent this I had to check the number of downloads before requesting group progress like this
it.progress = if (fetchGroup.downloads.isNotEmpty()) fetchGroup.groupDownloadProgress else 0
but for more cleaner code it would be great if this checking process is don in the Fetch library.

[tonyofrancis]

@yohane55 gonna look into this. Thanks for reporting.

[freehussain]

can you please send complete code to do group download progress, please

[tonyofrancis]

@freehussain have a look at the sample app. That should get you started in the right direction.

[freehussain]

hello,

@tonyofrancis i have looked at simple app, there is no group download notification

[tonyofrancis]

@freehussain here is an example.

class GroupActivity: AppCompatActivity() {

    private val groupId = 12
    private lateinit var fetch: Fetch

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        fetch = Fetch.getDefaultInstance()
        enqueueGroupRequest()
    }

    private fun enqueueGroupRequest() {
        val requests = Data.getFetchRequestWithGroupId(groupId)
        fetch.addListener(groupListener)
        fetch.enqueue(requests, Func { list ->  
            //the list contains pairs of request that were enqueued 
            // and will also return error for requests that were not enqueued.
        })
    }

    private val groupListener: AbstractFetchGroupListener = object: AbstractFetchGroupListener() {


        override fun onAdded(groupId: Int, download: Download, fetchGroup: FetchGroup) {

        }

        override fun onQueued(groupId: Int, download: Download, waitingNetwork: Boolean, fetchGroup: FetchGroup) {

        }

        override fun onStarted(groupId: Int, download: Download, downloadBlocks: List<DownloadBlock>, totalBlocks: Int, fetchGroup: FetchGroup) {

        }

        override fun onCompleted(groupId: Int, download: Download, fetchGroup: FetchGroup) {

        }

        override fun onResumed(groupId: Int, download: Download, fetchGroup: FetchGroup) {

        }

        override fun onPaused(groupId: Int, download: Download, fetchGroup: FetchGroup) {

        }

        override fun onCancelled(groupId: Int, download: Download, fetchGroup: FetchGroup) {

        }

        override fun onError(groupId: Int, download: Download, error: Error, throwable: Throwable?, fetchGroup: FetchGroup) {

        }

        override fun onProgress(groupId: Int, download: Download, etaInMilliSeconds: Long, downloadedBytesPerSecond: Long, fetchGroup: FetchGroup) {

        }
    }

    override fun onDestroy() {
        super.onDestroy()
        fetch.close()
    }

}

[freehussain]

hello,

thank for quick response,

can you please provide use an example how to make it show on notification area with progress indicator?

thank you a-lot for your help

[tonyofrancis]

@freehussain Please check out the FetchNotificationManager interface java documentation on how to create notifications for downloads.

[tonyofrancis]

Fixed. Will be in next release.