Audio_service: Integration with cached_network_image

[hacker1024]

Feature

I'm using the cached_network_image throughout my app so I''m not constantly downloading media art every time I show it. Additionally, in my audio task, I have a for loop that pre-fetches the art for upcoming songs in the queue.

if (_cacheManager == null) _cacheManager = DefaultCacheManager();
for (Song song in newSongs) _cacheManager.downloadFile(song.getArtUrl(serviceArtSize));

The art still gets downloaded twice for each song, though - once with cached_network_image, and once on the platform side.

It would be great if the platform side could use the images cached from the Dart side.

Proposed implementation

cached_network_image uses flutter_cache_manager for storage - which, in turn, uses a SQLite database to store files. As far as I'm aware, the flutter_cache_manager plugin does not provide ways to access the database from iOS or Android. I've made an issue about it at https://github.com/renefloor/flutter_cache_manager/issues/131.

There are three solutions that I can think of:

1. Use platform channels to retrieve cached images in Dart and send the bytes to the platform side
2. Manually read the SQLite database on the platform side
3. Wait for the flutter_cache_manager developer(s) to respond to my issue, and then use the feature if they implement it.

[hacker1024]

After looking at your bitmap loading code (on Android, I can't read Objective-C/Swift), it seems we can change artBitmapCache into an adapter object that uses one of the above implementations.

[rohansohonee]

Hi @hacker1024 ,

I believe you can use the cached images on the platform side.

@ryanheise provided the solution https://github.com/ryanheise/audio_service/pull/70#issuecomment-505918069

You can retrieve the filePath from the FileInfo object.

import 'dart:io';

///Flutter Cache Manager
///Copyright (c) 2019 Rene Floor
///Released under MIT License.

enum FileSource { NA, Cache, Online }

class FileInfo {
  FileInfo(this.file, this.source, this.validTill, this.originalUrl);

  String originalUrl;
  File file;
  FileSource source;
  DateTime validTill;
}

[ryanheise]

I also ran into this problem recently in my own app because I just pass in the remote URL as the artUri but I think the way to get around it is by implementing caching within your Flutter app and then passing in a file:// uri to your cached file as the artUri.

[hacker1024]

What would happen if the file hadn't downloaded yet? Would the service update the art when it finishes?

[hacker1024]

So basically this? What happens if the file's half-downloaded?

    MediaItem mediaItem = musicProvider.currentMediaItem;
    List<MediaItem> queue = musicProvider.queue;

    Future<String> getArtUri() async {
      // Get the cached file info
      final cachedFileInfo = (await musicProvider.cacheManager?.getFileFromCache(mediaItem.artUri));

      // If it doesn't exist, use the online URL
      if (cachedFileInfo == null) return mediaItem.artUri;

      // Return the local art URI
      return 'file://${cachedFileInfo.file.path}';
    }

    mediaItem = musicProvider.currentMediaItem.copyWith(
      duration: duration == 0 ? null : duration,
      artUri: await getArtUri(),
    );

    queue[0] = mediaItem;

    // Set metadata for the playing media
    AudioServiceBackground.setMediaItem(mediaItem);

    // Set the queue, used by Android Auto and some custom ROMs
    AudioServiceBackground.setQueue(queue);

[hacker1024]

This causes another problem - CachedNetworkImage, used throughout my UI, can't handle file:// URIs - breaking caching again.

Of course, I could handle that with if statements, but it'd be great if we could handle caching in the platform code so we don't have to jump through all these hoops.

[ryanheise]

Currently audio_service will try to load any URL that you give it, and if it fails to load, it remembers it on a blacklist so it doesn't try again. It would probably be better to allow reload attempts, actually.

However, with the current code, you should be able to initially set the media items with a null artUri and when you set an updated media item or queue with the artUri set, it should then begin loading it.

I don't know if I like the idea of having multiple caches. If you're already caching things to file on the Flutter side, then ideally, the cache library being used should provide a way to get the file path of a cached image. Just looking at cached_network_image, it does provide access to the cache which in turn does provide access to the file path, so you ought to be able to use that cache without creating another on the platform side.

[rohansohonee]

Hi @hacker1024

Solution to your problem:

• If the file is not downloaded send null or default artwork image
• Once the file is completed downloading you can again set the mediaItem by calling AudioServiceBackground.setMediaItem(mediaItem)

Also, you should use a different mediaItem object say tempMediaItem to set your AudioServiceBackground.setMediaItem(tempMediaItem)

 mediaItem = musicProvider.currentMediaItem.copyWith(
      duration: duration == 0 ? null : duration,
      artUri: use online URL over here,
    );

tempMediaItem = MediaItem(
              mediaId:  .....,
              duration:  ......,
              .....,
              artUri: 'file://${cachedFileInfo.file.path}'
              );

This ensures your caching will not break on UI side.

[rohansohonee]

@ryanheise

We both commented at nearly the same time LOL

[hacker1024]

I don't know if I like the idea of having multiple caches. If you're already caching things to file on the Flutter side, then ideally, the cache library being used should provide a way to get the file path of a cached image. Just looking at cached_network_image, it does provide access to the cache which in turn does provide access to the file path, so you ought to be able to use that cache without creating another on the platform side.

@ryanheise There'd just be one cache database, but it would be accessed from both sides.

Also, you should use a different mediaItem object say tempMediaItem to set your AudioServiceBackground.setMediaItem(tempMediaItem)

@rohansohonee That won't help, as the UI uses the media item from AudioServiceBackground.setMediaItem.

[hacker1024]

Btw, when using local file URLs, skipping to the next song is much faster. The plugins still blocks the UI thread while loading the notification image.

[hacker1024]

Until #132 drops, I'm using the genre field to store online URLs for my app's UI, which works at least.
https://github.com/EpimetheusMusicPlayer/Epimetheus/commit/ef13c1ad0d6152d8c3796447ce123e3ee713f5f4

I'm keeping this issue open because I'm gonna take a crack at implementing reading the Flutter cache from Android.

[ryanheise]

@hacker1024 If you had #132 then I think you wouldn't need to implement any special cache code on the Android side. Am I missing something?

[rohansohonee]

Implementing bi-directional custom actions can easily solve this issue I think in my opinion

[hacker1024]

@hacker1024 If you had #132 then I think you wouldn't need to implement any special cache code on the Android side. Am I missing something?

It's just more convenient. Rather than checking if the item's cached on the Dart side, the Dart side can just always provide the online URL and let the platform side handle caching. It's a "set and forget" thing.

It also has the byproduct of solving the bug where the UI thread gets blocked when loading images, because the images load in milliseconds.

[ryanheise]

Remember that implementing image loading on the Android side also comes with its own blocking bugs, which is another reason why I thought it would be better to let the Dart side do it. The issue was that all method channels would be blocked while preloading images for some unknown reason (even though this was done in a separate thread), and this sort of thing might happen if you try to do caching on the Android side too.

[hacker1024]

And this sort of thing might happen if you try to do caching on the Android side too.

I assumed it was network related, I'll see how it goes.

[ryanheise]

If the method channel blocking bug still exists, then it would alternatively be possible to implement caching in the plugin, but on the Dart side.

However I am still not convinced about having multiple caches. I see there are some advantages, although I'm inclined to first treat #132 as a higher priority and see how that goes first.

[hacker1024]

I'm not sure what you mean by multiple caches?

[ryanheise]

Sorry, I see you don't mean to have multiple caches, you just want to access the cache from the Android side.

I think @rohansohonee and I have both described a solution that should work without needing to add any new code to audio_service. The misunderstanding is basically about where to use the file:// URL. The idea is that you use the https:// URL when creating your CachedNetworkImage then query its cache to obtain the file path. Upon obtaining the file path, you construct a file:// URL pointing to that file and pass that in as the artUri to audio_service which doesn't need to know anything about caching. Since you are only passing the file:// URL into audio_service, you should not run into the problem you described with file:// URLs.

[hacker1024]

The problem is that the CachedNetworkImage widget doesn't work with file:// URIs, so I guess after #132 is resolved this issue is as well.

[ryanheise]

CachedNetworkImage gives you access to the cache manager which gives you access to the file which gives you access to the file path. Simply prepend file:// in front of it, and you've got a URL that can be passed into audio_service via artUri. Can you clarify which step in this solution is not possible?

[hacker1024]

That's all fine.

The issue is showing the cached album art in my app UI - I can't use the local URI, as the CachedNetworkImage widget needs the online one.

[hacker1024]

https://github.com/renefloor/flutter_cached_network_image/issues/311

[ryanheise]

I wouldn't pass the cached file back into CachedNetworkImage, I think logically the input to CachedNetworkImage should be the https:// URL. So after #132 is resolved, you'll be able to store the https// URL in the media item as an extra and you'll be able to use that in your UI.

[hacker1024]

Yes, that's what I said here.

The problem is that the CachedNetworkImage widget doesn't work with file:// URIs, so I guess after #132 is resolved this issue is as well.

[ryanheise]

Oops, I misread that, so yes I think we agree.

[ryanheise]

Just to update, the latest git commit now integrates flutter_cache_manager. In the process, I've also added "extras" to MediaItem so that the cache file path can be passed to the platform side, but may have other uses, too.

The cache has been tested on iOS and Android.

By the way, this also allows you to managing pre-fetching of artwork yourself. I had an option before called shouldPreloadArtwork but it was not generally useful as it just preloaded "everything" in the queue. Not great with a long queue.

Now instead what you can do is use the cache manager to preload the artwork for the next track while you're on the current track (or however you want preloading to work in your app).

[hacker1024]

That's great, thanks!
If I'm reading that new code correctly, it looks like all I'd need to do is make sure my images are cached with the Flutter cache manager, and put the external artUri in my MediaItem?

[hacker1024]

Is shouldPreloadArtwork gone completely? It looks like this forces developers to implement their own caching logic.

Perhaps we can include some default logic that does what shouldPreloadArtwork did, and caches the whole queue? I'd happily take a crack at it people think it's a good idea.

[ryanheise]

I considered reimplementing it in the Dart side where all of the artwork fetching code has moved, but then I thought it could be an opportunity to rethink preloading. Do we really want to preload the whole queue all at once, or would it make more sense to just have a shorter lookahead?

There's also the fact that in most cases, if the UI is already displaying the queue, it is likely that the images are already in the cache and preloading by this stage would be unnecessary.

So it seems like if there are any further edge cases, they would be rare, and they can be handled manually by the app developer through the new cache API.

I'm happy to be persuaded otherwise. Thoughts?

[hacker1024]

That makes sense. In my case, I'm only adding four songs to the queue at a time, when the second-last song is reached - so I cache the whole queue.

All apps with a "radio" like feature like my Pandora client probably have similar behaviour, but there probably aren't enough people doing this to justify including it in the plugin.

[ryanheise]

I would consider radio apps as one of the more common use cases, so if as you say most radio apps would always preload the whole queue, I think it could be worth adding this back in. It wouldn't be too difficult to add the code into setQueue on the Dart side.

[hacker1024]

I think it depends on how far ahead radio apps populate the queue - mine adds four songs at a time, when there are two left, so there's a maximum of six songs in the queue at any given time.

Other radio apps, however, may add any number of queue entries at a time, so it may be bad to preload the entire queue, as the user may not get that far down it.

I suppose there's no harm in including it, and leaving it off by default?

[ryanheise]

Sounds OK to me, given that the API is already there and I haven't removed the parameter yet. I'll try to implement it before the next release so that we don't have a zombie parameter.

[hacker1024]

If it helps, here's how I cached the icons for a list of radio stations:

@override
void cacheData(List<Station> stations, BaseCacheManager cacheManager) {
  for (Station station in stations) {
    cacheManager
        .downloadFile(
          station.getArtUrl(500),
        )
        .catchError(
          (error) {},
          test: (error) => error is HttpException || error is SocketException,
        );
  }
}

It's the same in my audio task, when I cache song album art:

// Cache the album art
for (Song song in newSongs) {
   _cacheManager.downloadFile(song.getArtUrl(serviceArtSize)).catchError(
      (error) {},
      test: (error) => error is HttpException || error is SocketException,
    );
}

[ryanheise]

I've now implemented artwork preloading and the option is now an optional parameter to setQueue. I've made it so that artwork is loaded serially rather than opening up all network requests simultaneously which I think should be a bit safer for long queues.

[ryanheise]

Now released in 0.7.0.

[hacker1024]

Great! Does this project have a wiki? I'd be happy to write a page on custom caching techniques.

EDIT: I see it does.

[ariefwijaya]

Hi, @ryanheise I think this feature should be able to be disabled. Because this feature cause database locked when we use flutter_cache_manager in foreground simultaneously because they used same database access.

[ryanheise]

@ariefwijaya can you open a new bug report and fill in all required details in the issue template?