Picasso: Picasso saving cached data of continuous IPTV streams inside the app

[needz]

The issue is a follow up reported in google/ExoPlayer (https://github.com/google/ExoPlayer/issues/3696).

Whenever I use ExoPlayer to stream IPTV content, the app starts to intensively cache some kind of data on local device storage (monitored by dumpsys diskstats) and it cache this data until it fills all the available storage on device. After the storage is filled, Android shows an alert the the device storage is low and apps will stop working.

It seems that the picasso extension (com.squareup.picasso) I am using is writing some kind of temporary file, but do you have any idea how it could be connected with ExoPlayer? I am using it for small images only, which do not exceed a couple of kilobytes, but the size of the cache is directly related to the stream being shown.

Is there anything that can be done to avoid this issue, because it's quite critical at the moment.

Here's the contents of one of those .tmp files (the smaller one), other file is encrypted content:

http://192.168.0.102:8001/1:0:1F:BC2:25:56:300000:0:0:0:
GET
0
HTTP/1.0 200 OK
6
Connection: Close
Content-Type: video/mpeg
Server: streamserver
X-Android-Selected-Protocol: http/1.0
X-Android-Sent-Millis: 1515708344386
X-Android-Received-Millis: 1515708344393

Below is the change in size during some time:

./cache/picasso-cache:
total 334672
drwx------ 2 u0_a99 u0_a99      4096 2018-01-12 00:05 .
drwxrwx--x 3 u0_a99 u0_a99      4096 2018-01-12 00:03 ..
-rw------- 1 u0_a99 u0_a99       261 2018-01-12 00:05 5198767f7d86a079eace4a3fcdfdbbe6.0.tmp
-rw------- 1 u0_a99 u0_a99 171335680 2018-01-12 00:08 5198767f7d86a079eace4a3fcdfdbbe6.1.tmp

./cache/picasso-cache:
total 364752
drwx------ 2 u0_a99 u0_a99      4096 2018-01-12 00:05 .
drwxrwx--x 3 u0_a99 u0_a99      4096 2018-01-12 00:03 ..
-rw------- 1 u0_a99 u0_a99       261 2018-01-12 00:05 5198767f7d86a079eace4a3fcdfdbbe6.0.tmp
-rw------- 1 u0_a99 u0_a99 186736640 2018-01-12 00:08 5198767f7d86a079eace4a3fcdfdbbe6.1.tmp

./cache/picasso-cache:
total 410624
drwx------ 2 u0_a99 u0_a99      4096 2018-01-12 00:05 .
drwxrwx--x 3 u0_a99 u0_a99      4096 2018-01-12 00:03 ..
-rw------- 1 u0_a99 u0_a99       261 2018-01-12 00:05 5198767f7d86a079eace4a3fcdfdbbe6.0.tmp
-rw------- 1 u0_a99 u0_a99 210223104 2018-01-12 00:08 5198767f7d86a079eace4a3fcdfdbbe6.1.tmp

./cache/picasso-cache:
total 453488
drwx------ 2 u0_a99 u0_a99      4096 2018-01-12 00:05 .
drwxrwx--x 3 u0_a99 u0_a99      4096 2018-01-12 00:03 ..
-rw------- 1 u0_a99 u0_a99       261 2018-01-12 00:05 5198767f7d86a079eace4a3fcdfdbbe6.0.tmp
-rw------- 1 u0_a99 u0_a99 232169472 2018-01-12 00:09 5198767f7d86a079eace4a3fcdfdbbe6.1.tmp

[JakeWharton]

Those are files in the HTTP cache Picasso sets up for the HTTP client. ExoPlayer doesn't use Picasso, so it's likely coming from some other library or SDK you are using.

[needz]

But the cache file only appears and grows when there is a stream coming in using ExoPlayer and the size of it is directly related. I know ExoPlayer doesn't use picasso, but the folder is called picasso-cache.

[JakeWharton]

Well _something_ is asking Picasso to download images in your ExoPlayer configuration.

[needz]

Is there a way to somehow disable caching based on Content-Type: video/mpeg or Server: streamserver, so it doesn't write cache when it detects such content?

[JakeWharton]

You could write an OkHttp interceptor to strip the cache headers on responses with such content types, but I don't think those URLs should be going through Picasso in the first place. That's the problem that I would aim to fix.

[ojw28]

ExoPlayer doesn't have any dependency on Picasso, so I don't think any direct configuration by ExoPlayer could be responsible for this. Is it possible that Picasso is directly or indirectly inserting functionality in to the same network stack ExoPlayer is using, and that this is having unintended consequences? This isn't the first "Picasso broke ExoPlayer" issue we've seen. Another example is https://github.com/google/ExoPlayer/issues/625.

@JakeWharton - Is that a possibility? On the ExoPlayer side, playback may be using OkHttp bundled in the app (the OkHttpClient instance is provided by application code) or Android's HttpUrlConnection API, depending on configuration. I'm particularly curious as to whether Picasso could be inserting functionality that affects all requests made through app bundled OkHttp, and/or whether it could be inserting functionality into a particular OkHttpClient that app code is then giving to ExoPlayer, where it should instead be providing a separate instance. I'm not familiar with how OkHttp works internally, so sorry if these questions don't make sense :).

[ojw28]

This looks suspicious, in the case that OkHttp is not being used:

https://github.com/square/picasso/blob/picasso-parent-2.5.2/picasso/src/main/java/com/squareup/picasso/UrlConnectionDownloader.java#L132

Does that cause a cache to be installed that would affect all requests going through HttpUrlConnection, including those made by ExoPlayer and any other parts of the application? If so, shouldn't Picasso being doing something that only affects its own loads?

[JakeWharton]

Yep, that'll do it. It's impossible to localize a HTTP cache using that API. One of the many reasons that no one should be using HttpURLConnection!

We've completely removed support for HttpURLConnection in Picasso 3 (unreleased on master). I would _strongly_ encourage you to migrate to OkHttp for Picasso 2 using https://github.com/JakeWharton/picasso2-okhttp3-downloader until Picasso 3 is released.

[needz]

@JakeWharton, that's exactly what I am using Picasso 2 with at the moment, so I guess the only thing left is to make ExoPlayer use OkHttp only.

[ojw28]

If you're using the OkHttp3 downloader then I don't understand where the problem is coming from, since the hypothesis above relies on you (or at least Picasso internally) using UrlConnectionDownloader.

[JakeWharton]

Picasso shouldn't install the HttpURLConnection response cache unless
you're using the UrlConnectionDownloader (either explicitly or implicitly).

That being said, using OkHttp with ExoPlayer is likely a good choice as it
enables all kinds of fancy new things that HttpURLConnection doesn't
support (most notable HTTP/2).

On Fri, Jan 12, 2018 at 12:15 PM needz [e-mail] wrote:

@JakeWharton https://github.com/jakewharton, that's exactly what I am
using Picasso 2 with at the moment, so I guess the only thing left is to
make ExoPlayer use OkHttp only.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/square/picasso/issues/1727#issuecomment-357295534,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEEEb3klFTF5yHkIqXnAJdnFhaFHDF9ks5tJ5AagaJpZM4RbjoS
.

[needz]

@ojw28, I am only using OkHttp3 downloader for Picasso, but not for ExoPlayer. Maybe you could provide an example of using it with ExoPlayer, so I can test it and provide feedback if it solves the problem. Please check https://github.com/google/ExoPlayer/issues/3696#issuecomment-357246659 in ExoPlayer issue.

[ojw28]

It doesn't make sense that this would be an issue unless you're using HttpUrlConnection for both Picasso and ExoPlayer. You would need to be using it for Picasso for the cache configuration to occur, and also for ExoPlayer for it to then write into the cache. I suspect you're using HttpUrlConnection by accident somewhere in your Picasso usage.

[needz]

I set Picasso to use okhttp3 downloader like this:

**Activity**
OkHttpClient okHttp3Client = new okhttp3.OkHttpClient();
OkHttp3Downloader okHttp3Downloader = new OkHttp3Downloader(okHttp3Client);

**onCreate**
picasso = new Picasso.Builder(context).downloader(okHttp3Downloader).build();

[needz]

Using OkHttp with ExoPlayer fixes the issue, thanks everyone for support!

[JakeWharton]

You should also call Picasso.setSingletonInstance so that any consumer of the Picasso.with(Context) method uses your configured instance instead of a new one (which might fall back to HttpURLConnection if it doesn't detect OkHttp).