Fetch: onComplete Prematurely ... Immediately called

[iBlueDust]

Fetch in itself is not working for me, unfortunately. Immediately after enqueueing, onComplete is immediately called. The destinated file does not exist at all, yet download.getDownloaded() returns a value far more than zero. Creating the file before enqueueing does not help as well.

Note: The destination is in a cache dir. Fetch is static in an Application class and a service does all the enqueueing.

[tonyofrancis]

@iBlueDust can you provide sample code? Simply not enough to just say it does not work.

[iBlueDust]

Commons.fetch.addListener(new FetchListener() {
            @Override
            public void onAdded(@NotNull Download download) {
                UpdateMaps(download);
            }

            @Override
            public void onQueued(@NotNull Download download, boolean b) {
                DownloadService.this.onProgress(0L, 0L, true, DOWNLOADING);
                UpdateMaps(download);
            }

            @Override
            public void onWaitingNetwork(@NotNull Download download) {
                UpdateMaps(download);
            }

            @Override
            public void onCompleted(@NotNull Download download) {
                DownloadComplete(download.getRequest());
                // Function called, file still doesnt exist
            }

            @Override
            public void onError(@NotNull Download download, @NotNull com.tonyodev.fetch2.Error error, @org.jetbrains.annotations.Nullable Throwable throwable) {
                String msg = "Download Error" + (throwable == null ? "" : throwable.getMessage());
                Log.e(TAG, msg);
                onFinish(download.getRequest(), false, DOWNLOADING, new RuntimeException(msg));
            }

            @Override
            public void onDownloadBlockUpdated(@NotNull Download download, @NotNull DownloadBlock downloadBlock, int i) { }

            @Override
            public void onStarted(@NotNull Download download, @NotNull List<? extends DownloadBlock> list, int i) {
                DownloadArgs args = map.get(download.getRequest());
                if (args == null)
                    Log.w(TAG, MISSING_IN_MAP);
                else args.setStatus(Status.DOWNLOADING);
            }

            @Override
            public void onProgress(@NotNull Download download, long l, long l1) {
                DownloadService.this.onProgress(l, l1, false, DOWNLOADING);
            }

            @Override
            public void onPaused(@NotNull Download download) {
                DownloadArgs args = map.get(download.getRequest());
                if (args == null)
                    Log.w(TAG, MISSING_IN_MAP);
                else args.setStatus(Status.PAUSED);
            }

            @Override
            public void onResumed(@NotNull Download download) {
                UpdateMaps(download);
            }

            @Override
            public void onCancelled(@NotNull Download download) {
                DownloadArgs args = map.get(download.getRequest());
                if (args == null) 
                    Log.w(TAG, MISSING_IN_MAP);
                else args.setStatus(Status.CANCELLED);
            }

            @Override
            public void onRemoved(@NotNull Download download) {
                UpdateMaps(download);
            }

            @Override
            public void onDeleted(@NotNull Download download) {
                UpdateMaps(download);
            }
        });

Request request = new Request(url, String.format(Locale.ENGLISH, "%s/%d.%s", Commons.Directories.BIN.getAbsolutePath(), i, format));
        request.setNetworkType(NetworkType.ALL);
        request.setPriority(Priority.HIGH);
        map.put(request, current);
        Commons.fetch.enqueue(request, null, null);

[tonyofrancis]

@iBlueDust without the full url and file path I still cannot test. Please provide url and file path.

[lbuse]

I'm having the same issue while testing everything locally.

I have this node package running on my machine as a file server.

This is the code to run my local server:
http-server D:\android\repo\files -p 80 -c-1

Which gives me this url:
http://192.168.0.195/myfile.zip

On Android I'm trying to save the file in the app's private folder:
File updateDir = getExternalFilesDir(null);

This is the log from Fetch:
LibGlobalFetchLib: Completed download DownloadInfo(id=-1693056914, namespace='LibGlobalFetchLib', url='http://192.168.0.195/myfile.zip', file='/storage/sdcard/Android/data/com.app.myapp.debug/files/myfile.zip', group=0, priority=NORMAL, headers={clientKey=SD78DF93_3947&MVNGHE1WONG}, downloaded=1541071, total=1541071, status=COMPLETED, error=NONE, networkType=ALL, created=1544049087670, tag=null, enqueueAction=UPDATE_ACCORDINGLY, identifier=0, downloadOnEnqueue=true, extras={})

Before Fetch, I was using DownloadManager, but I had some problems to show the progress, so I ended up finding this lib that look much more flexible and useful!

[iBlueDust]

The URL is https://r8---sn-4pgnuhxqp5-jb3r.googlevideo.com/videoplayback?id=o-AED5glku5UOn3NEWZDk55yi_052xzZznFhF02PDFWAOp&mime=audio%2Fmp4&pl=22&mm=31%2C26&mn=sn-4pgnuhxqp5-jb3r%2Csn-i3b7knld&ipbits=0&gcr=id&ip= [your public ip]&requiressl=yes&ms=au%2Conr&mt=1544078894&gir=yes&dur=248.685&ei=g8YIXK2KEqeWz7sPmPWK4A8&initcwndbps=473750&expire=1544100579&mv=m&sparams=clen%2Cdur%2Cei%2Cgcr%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cexpire&clen=3950365&source=youtube&c=WEB&keepalive=yes&itag=140&key=yt6&lmt=1537129430191219&fvip=4&signature=1AE01968BDF3677820DF9CDD25691CB2F6B4CE9F.312FB825549C2B1D00AB4DE45E9A87056606A728

And the path is /storage/sdcard1/Android/data/com.example.app/cache/bin0.m4a

[iBlueDust]

@tonyofrancis , please tell if you were able to recreate the problem. Thx

[tonyofrancis]

@iBlueDust could not get the url to work even in a web browser. Are you sure this link points to the actual file or an html page?

[tonyofrancis]

@lucasbusellato is the file created? What does the file size look like if created?

[iBlueDust]

The url might be generated. Please note that you must insert your public ip in the designated area.

The url was generated by https://github.com/HaarigerHarald/android-youtubeExtractor. Try to directly feed the url from the library. The url might only work for a certain period of time after querying.

Also you will need to get a Youtube API key
http://console.developer.google.com

[lbuse]

@tonyofrancis I've just discovered something, the file will only be actually saved in the folder once, after that, I need to create/rename a subfolder to work again (a bit confusing to explain).

Let me try like this:

1. Download in myAppPath/files/
   File appears
2. Download in myAppPath/files/
   File does not appear
3. Add subfolder, download in myAppPath/files/folder1/
   File appears
4. Download in myAppPath/files/folder1/
   File does not appear

When the file appears, it has the same size as the log presents: 1.505KB.

Additional information:
In all my tests, I haven't received any notification. Right now I've been using setNotificationManager(new DefaultFetchNotificationManager(getApplicationContext())) just for testing.

I'm targeting api 28 and using androidx, tests are happening on android 5.1 and 8.1.

[tonyofrancis]

@iBlueDust @lucasbusellato What do your FetchConfiguration Look like? I also need simple link I can try from or an example project.? You can send me an email with the example project like. Email is on my profile.

[lbuse]

@tonyofrancis here is the class I'm using DownloadWorker.zip and this is how I call it:

OneTimeWorkRequest downloadFile = new OneTimeWorkRequest.Builder(DownloadWorker.class).build(); WorkManager.getInstance().enqueue(downloadFile);

To provide a real link, I switched to a real server where an image is downloaded, and I get the same behavior.
You will see a function that deletes everything inside the application folder before starting the download, this is part of a control mechanism that I intend to implement.

[tonyofrancis]

@lucasbusellato. I just had a look at your code and it looks fine. Are you creating instances of Fetch anywhere else in the app other than the worker?
Also what version of Fetch are you using. The latest is 2.3.3

[tonyofrancis]

@iBlueDust Fetch should be able to download the correct file with its content if the right url is provided. Also verify that you are on the latest version.

[tonyofrancis]

Fetch 2.3.4 has be released.

[lbuse]

Hello @tonyofrancis!

I upgraded to 2.3.4 but I still see the same behavior.
I really don't know why this is happening, so I will try to give you more info and a sample project.

I tried to create a new project just to try Fetch without any possible interference from my current project.

Unfortunately, I got the same result, the behavior I've saw is this:

1. The file is created in the app's directory, but with 0 Kb and no notification;
2. I try a new download and then the file is actually downloaded, because now it has the correct size and I get the default notification;
3. I manually delete the file using Android Studio's Device File Explorer;
4. A new download is executed, but nothing happens, the file is not created and I don't get a notification;
5. I can go back to step nº 1 and repeat this process if I try to save the file in a subdirectory of the app's directory, e.g.: subdir/.

This is the project MyTest.zip
I've tested this project on real device running 8.1 and on a emulator with 5.1.

[eon-yogiwp]

It seems like re-download doesn't work. To be more specific:

• If both URL and destination file matches any previous completed request, then download won't happen (we get onCompleted immediately). This persists even after terminating and restarting the app.
• If only the URL matches any previous request but destination file is different, then download will happen as expected.

It seem to remember completed requests and refuse to redo identical requests. One workaround I found is by calling mFetch.delete(request.id) before re-download. If this is the expected behavior, I think this needs to be stated clearly somewhere. Discovered this after hours of trial and error.

[tonyofrancis]

@eon-yogiwp Thanks for explaining the issue plainly.

I know exactly what is going on. Each request as an EnqueueAction associated with it. The Default EnqueueAction is UPDATE_ACCORDINGLY and it does the following:

  /** If Fetch is already managing an existing request with the same file path do one the following:
     * 1: If existing download is completed, Fetch will call onComplete method on attached Fetch Listeners.
     * 2: If existing download is not completed, resume download normally.
     * 3: If not downloaded, download will proceed normally.
     * Note: If download is existing, Fetch will update the old request/download with the new settings on
     * from the request object.
     * */

The behavior @iBlueDust @lucasbusellato is looking for is most likely EnqueueAction.REPLACE_EXISTING Which will always restart the download.

Request request = new Request(url, file);
request.setEnqueueAction(EnqueueAction.REPLACE_EXISTING);
fetch.enqueue(request, callback, callback);

Also have a look at the other EnqueueActions.
I will also have another look and test EnqueueAction.UPDATE_ACCORDINGLY when downloads are already completed to make sure it works correctly

[lbuse]

@tonyofrancis @eon-yogiwp I just tested your solutions and the problem was partially solved, I got the same result with either of them, to be 100%, I had to add Thread.sleep() right after fetch.enqueue because the execution was finishing before the download was enqueued.

That's why on the first try the file was just created and on the second the download was completed successfully.

[iBlueDust]

request.setEnqueueAction(Enqueue.REPLACE_EXISTING; did it for me.

Sorry for not helping in the later process of debugging this, I had other things in the way apart from forgetting as well.

No more shenanigans, everything works fine as expected now.
Thanks a lot, everyone :D

[sagarsiddhpura]

This also helped me. There is an issue with fetch.

1) download file
2) delete file from file manager
3) request download of file again

Fetch will always call onCompleted as if download is completed. The file is never re-created. It should have a check that if requested file does not exist in filesystem, it should default to REPLACE.

[tonyofrancis]

@lucasbusellato found the bug and working on a fix. will update once I release.

[tonyofrancis]

This bug is now fixed in Fetch version 2.3.5