Fetch: Possibility to customise saving to output

[marcin-adamczewski]

Hi,

It would be nice to have a possibility to customise saving input data to the output.
It would allow for making things like encryption on the fly. So while data are being downloaded
they would also be encrypted and saved to the output.

You could easily achieve this if in FileDownloaderImpl there would be an OutputStream instead of RandomAccessFile and if user would have ability to provide his own OutputStream. For example user could provide CipherOutputStream so he would have the ability to encrypt data being downloaded.

[tonyofrancis]

@marcin-adamczewski Thanks for using Fetch2. Interesting feature request. Looking into this feature and will update this post very soon. Please check back.

[tonyofrancis]

@marcin-adamczewski I just released version 2.0.0-RC17 with this new feature. In order to provide your own OutputStream you need to provide Fetch your own custom Downloader. Example using a custom okHttpDownloader:

    /**
     * Customer downloader that lets you provide your own output streams for downloads.
     * See Downloader.kt documentation for more information on providing your own downloader.
     */
    private static class OkHttpOutputStreamDownloader extends OkHttpDownloader {

        public OkHttpOutputStreamDownloader(@Nullable OkHttpClient okHttpClient) {
            super(okHttpClient);
        }

        @Nullable
        @Override
        public OutputStream getRequestOutputStream(@NotNull Request request, long filePointerOffset) {
            //If overriding this method, see the Downloader.kt documentation on how to properly use this method.
            // If done incorrectly you may override data in files.
            try {
                final FileOutputStream fileOutputStream = new FileOutputStream(request.getFile(), true);
                return new BufferedOutputStream(fileOutputStream);
            } catch (FileNotFoundException e) {
                e.printStackTrace();
                //Cannot find file. Provide fallback.
            }
            return null;
        }
    }

   final OkHttpClient client = new OkHttpClient.Builder().build();
        final Downloader okHttpDownloader = new OkHttpOutputStreamDownloader(client);
        final Logger logger = new FetchTimberLogger();
        final int concurrentLimit = 2;
        final boolean enableLogging = true;
        final Fetch fetch = new Fetch.Builder(this, namespace)
                .setLogger(logger)
                .setDownloader(okHttpDownloader)
                .setDownloadConcurrentLimit(concurrentLimit)
                .enableLogging(enableLogging)
                .enableRetryOnNetworkGain(true)
                .build();

The OutputStream getRequestOutputStream(@NotNull Request request, long filePointerOffset) method on the downloader lets you return an output stream per request/download that Fetch will use to save data for the download. If you return null, Fetch will provide the output stream for you. Use the request object to get the information of a request. The filePointerOffset is the location Fetch will begin writing/appending data to on the stream/file. Thats how Fetch resumes a paused download.

Note: If your output stream does not support appending data, Fetch will write to the beginning of the file. This will cause data corruption for downloads that are paused and resumed.

You did mention that you may like to use CipherOutputStream as the outputStream. You could probably return an outputstream like the following for Fetch to use:

        @Nullable
        @Override
        public OutputStream getRequestOutputStream(@NotNull Request request, long filePointerOffset) {
            //If overriding this method, see the Downloader.kt documentation on how to properly use this method.
            // If done incorrectly you may override data in files.
            try {
                final FileOutputStream fileOutputStream = new FileOutputStream(request.getFile(), true);
                return new CipherOutputStream(fileOutputStream, getCipher());
            } catch (FileNotFoundException e) {
                e.printStackTrace();
                //Cannot find file. Provide fallback.
            } catch (Exception e) {
                e.printStackTrace();
           }
            return null;
        }

Give it a try and awaiting your feedback.

[tonyofrancis]

@marcin-adamczewski Have you been able to test this new feature out?. Would like to hear your feedback.

[marcin-adamczewski]

Hi @tonyofrancis,
I was roughly testing it today and it seems to work well. 😀
I was able to use my encrypting output stream using this code and then decode data with success.
I'll test it more on Monday and let you know if there is any issue.
So far it looks good. Thanks for really quick job.

[harshitwynk]

Hi @tonyofrancis
I have just come across this library for downloading music files.
But unfortunately, there is no getRequestOutputStream() method present in Downloader interface anymore.
Can you please help me to implement CipherOutputStream to encrypt files while downloading?