Capacitor: Read large files from the filesystem

[diachedelic]

My app must be able to download images/video/audio for offline-perusal, on both iOS and Android. Pending #984, I have used cordova-plugin-file and cordova-promise-fs to store a downloaded blob to the filesystem, which is working nicely.

However, on both iOS and Android, whatever I try I cannot play a local video!

Using .toInternalURL()

<video src="cdvfile://localhost/persistent/data/user/0/com.example.app/cache/video.mp4"></video>

Throws error:

Mixed Content: The page at 'https://localhost:5174/' was loaded over HTTPS, but requested an insecure video 'cdvfile://localhost/persistent/data/user/0/com.example.app/cache/video.mp4'. This request has been blocked; the content must be served over HTTPS.

So cdvfile: is considered an insecure protocol.

Using .toURL()

<video src="file:///data/user/0/com.example.app/files/files/data/user/0/com.example.app/cache/video.mp4"></video>

Throws error:

Not allowed to load local resource: file:///data/user/0/com.example.app/files/files/data/user/0/com.example.app/cache/video.mp4

I realise it's a weird looking path, but that's what I get from .toURL(), and I don't think that's really the problem here.

Both the above examples behaved very similarly on my iOS simulator so I will not bother to post that error log here.

Failed approaches

Content-Security-Policy

I have tried tweaking the Content-Security-Policy meta tag to include the cdvfile: protocol, but according to this post you can only tighten security using that, not loosen it.

Potential solutions

Capacitor support for <access origin="cdvfile://*"/>

It appears Cordova may have a workaround for this issue, perhaps the functionality could be duplicated in capacitor.config.js? However Cordova's solution might be outdated.

Read as data URL

I can read the video as a Base64 string using fs.toDataURL() and then supply a data: URL to my <video> element. This works fine on iOS and Android for small videos, but it will break for larger videos as browsers generally set a maximum size for strings (apparently ~256MB), and also loads the whole video into memory which is a big no-no.

Tweak Swift/Java

According to this, there is a way to allow mixed content on an Android WebView, perhaps there is also a way to get it working on iOS?

Use native video player

The VideoPlayer plugin can play local videos using an Android's native player. I'd like to avoid this as I'm trying to keep compatibility with iOS and the browser.

Streams

If Capacitor's FileSystem plugin could return a ReadableStream of the file, I'm sure a solution could be found. To avoid holding the entire file in memory at once, it could be piped into successive blobs, then stitched together into a larger blob and then turned into a playable URL using URL.createObjectURL(). Chrome at least will write blobs to disk if memory pressure gets too high.

[jcesarmobile]

If you get the file:/// url and change it to capacitor-asset:/// it should work. There is a js function to do it too, better use it as capacitor-asset might change, so better don’t hardcode it. It’s in window.Ionic.WebView.convertFileSrc

[jcesarmobile]

Just noticed you are talking about android. What I told you is still unreleased for Android.
On Android there is a setting to allow mixed content
https://capacitor.ionicframework.com/docs/basics/configuring-your-app

[diachedelic]

Ah that's handy little function! Perhaps it could be part of the FileSystem plugin?

However, I've run into further problems. When I try to play the video on iOS (via capacitor-asset://), I get a MEDIA_ERR_SRC_NOT_SUPPORTED error, and on Android (via https://localhost:5174/_capacitor_/...) I get a DEMUXER_ERROR_COULD_NOT_OPEN, which are both just MediaErrors with code 4.

On Android, I suspect it has something to do the fact that the response has a Content-Type: text/html header:

fetch('https://localhost:5174/_capacitor_/data/user/0/com.example.app/files/files/data/user/0/com.example.app/cache/007881CC-9CDA-406E-A0BD-19F9CDB6E867')
  .then(res => console.log(res.headers.get('content-type'))) // "text/html"

However I haven't been able to use fetch to inspect the iOS request, as I get these errors:

[Error] Cross origin requests are only supported for HTTP.
[Error] Fetch API cannot load capacitor-asset:///Users/.../B6E867 due to access control checks.

[diachedelic]

I realise I was totally mistaken about the Android situation - https://localhost:5174/_capacitor_/... always returns the HTML of the app. So as you mentioned, it doesn't look like Android supports the window.Ionic.WebView.convertFileSrc method. However, if instead I use a cdvfile: URL I get

cdvfile://localhost/persistent/data/user/0/com.example.app/cache/video.mp4
Failed to load resource: net::ERR_UNKNOWN_URL_SCHEME

[diachedelic]

So I still don't know how to play a local video in iOS or Android

[diachedelic]

I am also experiencing problems when trying to write a 20MB video to disk, using cordova-plugin-file (as Capacitor's FileSystem plugin does not seem to support writing binary data yet). It appears the entire file is being converted to a string at once :(

    const fs = CordovaPromiseFS({
      persistent: true,
      storageSize: 0,
    })

    const blob = await fetchResponse.blob()

    await fs.write('video.mp4', blob)

Java exception on my Samsung A5:

V/Capacitor/Plugin: To native (Cordova plugin): callbackId: File61318015, service: File, action: getDirectory, actionArgs: ["cdvfile:\/\/localhost\/persistent\/","videos",{"create":true}]
V/Capacitor/Plugin: To native (Cordova plugin): callbackId: File61318016, service: File, action: getFile, actionArgs: ["cdvfile:\/\/localhost\/persistent\/","videos\/8DBE7D18-6FCB-4005-ADBD-9090730BD15F\/",{"create":true}]
V/Capacitor/Plugin: To native (Cordova plugin): callbackId: File61318017, service: File, action: getFileMetadata, actionArgs: ["cdvfile:\/\/localhost\/persistent\/videos\/8DBE7D18-6FCB-4005-ADBD-9090730BD15F"]
I/zygote64: Background concurrent copying GC freed 102126(4MB) AllocSpace objects, 10(3MB) LOS objects, 47% free, 26MB/50MB, paused 49.382ms total 76.570ms
I/zygote64: Background concurrent copying GC freed 234(64KB) AllocSpace objects, 1(16KB) LOS objects, 23% free, 77MB/101MB, paused 33.661ms total 59.492ms
I/zygote64: Background concurrent copying GC freed 355(87KB) AllocSpace objects, 10(33MB) LOS objects, 23% free, 78MB/102MB, paused 12.077ms total 40.865ms
I/zygote64: Clamp target GC heap from 129MB to 128MB
    Background concurrent copying GC freed 7(32KB) AllocSpace objects, 1(9MB) LOS objects, 17% free, 105MB/128MB, paused 22.315ms total 52.793ms
I/zygote64: Starting a blocking GC Alloc
I/zygote64: Starting a blocking GC Alloc
I/zygote64: Alloc concurrent copying GC freed 3(16KB) AllocSpace objects, 1(18MB) LOS objects, 21% free, 87MB/111MB, paused 236us total 24.679ms
    Starting a blocking GC Alloc
I/zygote64: Alloc concurrent copying GC freed 3(16KB) AllocSpace objects, 0(0B) LOS objects, 21% free, 87MB/111MB, paused 211us total 19.443ms
    Forcing collection of SoftReferences for 72MB allocation
    Starting a blocking GC Alloc
I/zygote64: Alloc concurrent copying GC freed 1550(82KB) AllocSpace objects, 0(0B) LOS objects, 21% free, 87MB/111MB, paused 206us total 24.198ms
W/zygote64: Throwing OutOfMemoryError "Failed to allocate a 75497480 byte allocation with 25165824 free bytes and 40MB until OOM, max allowed footprint 116700504, growth limit 134217728"
I/zygote64: Starting a blocking GC Alloc
I/zygote64: Starting a blocking GC Alloc
I/zygote64: Alloc concurrent copying GC freed 4(31KB) AllocSpace objects, 0(0B) LOS objects, 21% free, 87MB/111MB, paused 206us total 19.467ms
    Starting a blocking GC Alloc
I/zygote64: Alloc concurrent copying GC freed 3(15KB) AllocSpace objects, 0(0B) LOS objects, 21% free, 87MB/111MB, paused 204us total 19.365ms
    Forcing collection of SoftReferences for 72MB allocation
    Starting a blocking GC Alloc
I/zygote64: Alloc concurrent copying GC freed 14(16KB) AllocSpace objects, 0(0B) LOS objects, 21% free, 87MB/111MB, paused 210us total 23.627ms
W/zygote64: Throwing OutOfMemoryError "Failed to allocate a 75497480 byte allocation with 25165824 free bytes and 40MB until OOM, max allowed footprint 116700264, growth limit 134217728"
W/System.err: java.lang.OutOfMemoryError: Failed to allocate a 75497480 byte allocation with 25165824 free bytes and 40MB until OOM, max allowed footprint 116700264, growth limit 134217728
W/System.err:     at java.util.Arrays.copyOf(Arrays.java:3260)
W/System.err:     at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:125)
        at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:660)
        at java.lang.StringBuilder.append(StringBuilder.java:203)
        at org.json.JSONStringer.string(JSONStringer.java:344)
        at org.json.JSONStringer.value(JSONStringer.java:252)
        at org.json.JSONArray.writeTo(JSONArray.java:613)
        at org.json.JSONArray.toString(JSONArray.java:585)
        at java.lang.String.valueOf(String.java:2827)
        at org.json.JSON.toString(JSON.java:90)
W/System.err:     at org.json.JSONObject.getString(JSONObject.java:554)
        at com.getcapacitor.JSObject.getString(JSObject.java:48)
        at com.getcapacitor.JSObject.getString(JSObject.java:43)
        at com.getcapacitor.MessageHandler.postMessage(MessageHandler.java:43)
        at android.os.MessageQueue.nativePollOnce(Native Method)
        at android.os.MessageQueue.next(MessageQueue.java:325)
        at android.os.Looper.loop(Looper.java:142)
        at android.os.HandlerThread.run(HandlerThread.java:65)
V/Capacitor: callback: -1, pluginId: Console, methodName: log, methodData: {"level":"error","message":"{}"}
E/Capacitor/Plugin/Console: {}
V/Capacitor: callback: -1, pluginId: Console, methodName: log, methodData: {"level":"error","message":"{}"}
E/Capacitor/Plugin/Console: {}
E/Capacitor: JavaScript Error: {"type":"js.error","error":{"message":"Uncaught Error: Java exception was raised during method invocation","url":"capacitor-runtime.js","line":2017,"col":38,"errorObject":"{}"}}

[diachedelic]

I have managed to download a 20MB video and persist it to disk! However I still get an out-of-memory error when I try to view it as a data URL.

import base64 from 'base64-js'
import { Plugins, FilesystemDirectory } from '@capacitor/core'

const { Filesystem } = Plugins

const videoPath = 'video.mp4'

async function downloadVideo() {
  // wipe video
  await Filesystem.deleteFile({
    path: videoPath,
    directory: FilesystemDirectory.Documents,
  })

  // download video
  let res = await fetch('https://example.com/video.mp4', {
    mode: 'cors',
    credentials: 'omit',
  })

  if (res.status !== 200) {
    throw new Error(`${res.status}: ${await res.text()}`)
  } else if (res.type === 'opaque') {
    throw new Error('opaque response')
  }

  // write file to disk as it comes in, encoding each chunk to base64
  const reader = res.body.getReader()

  while (true) {
    const { done, value } = await reader.read()

    if (done) {
      break
    }

    const encoded = base64.fromByteArray(value)

    await Filesystem.appendFile({
      path: videoPath,
      data: encoded,
      directory: FilesystemDirectory.Documents,
    })
  }

  // read file as base 64
  const { data } = await Filesystem.readFile({
    path: videoPath,
    directory: FilesystemDirectory.Documents,
  })

  // file has a trailing newline
  const encoded = data.replace('\n', '')

  // set mime type
  const videoURL = `data:video/mp4;base64,${encoded}`

  // display video on page
  document.getElementById('video').src = videoURL
}

[guylando]

@diachedelic
On android you can disable mixed content policy in the webview, see: https://developer.android.com/reference/android/webkit/WebSettings.html#MIXED_CONTENT_ALWAYS_ALLOW

However I still haven't found a way to load non-https js/css files on a remote https page on ios. Doesn't seem that there is a way to bypass the mixed content policy.

Any luck on ios to read local files from remote https page?

[diachedelic]

@guylando actually it appears window.Ionic.WebView.convertFileSrc now does the job for us, at least on Android (I think iOS as well). As you can see from the source, it will not work from a remote page - have you opened a separate issue?

[diachedelic]

For anyone interested, this is our solution on Android (probably works on iOS too).

Installed dependencies:

• cordova-plugin-file
• cordova-plugin-file-transfer
• cordova-plugin-whitelist (Android only)

import CordovaPromiseFS from 'cordova-promise-fs'

let progress = null
const videoPath = 'video.mp4'

const fs = CordovaPromiseFS({
  persistent: true,
  storageSize: 0,
})

async function download(url) {
  await fs.download(
    url,
    videoPath,
    ({ lengthComputable, total, loaded }) => {
      if (lengthComputable) {
        progress = loaded / total
      }
    }
  )
}

async function showVideo() {
  const fileURL = await fs.toURL(videoPath)
  const localURL = window.Ionic.WebView.convertFileSrc(fileURL)

  document.querySelector('video').src = localURL
}

Further Android configuration:

<!-- added to android/src/main/res/xml/config.xml -->
<widget ...>
  <access origin="*" subdomains="true" />
  <allow-navigation href="http://*/*" />
  <allow-navigation href="https://*/*" />
  <allow-intent href="http://*/*" />
  <allow-intent href="https://*/*" />
  <allow-intent href="tel:*" />
  <allow-intent href="sms:*" />
  <allow-intent href="mailto:*" />
  <allow-intent href="geo:*" />
  ...
</widget>

[diachedelic]

@guylando here is a modified convertFileSrc you could try...

  function(url) {
    if (!url) {
      return url;
    }

    return url.replace('file://', 'http://localhost/_capacitor_file_');
  }

[guylando]

this will not work on ios as I described from mixed content problem because getting js/css file using http protocol from https page is not allowed by mixed content policy.

[guylando]

I submitted a PR to the file plugin which solves the mixed content problem on ios: https://github.com/apache/cordova-plugin-file/pull/296
The fixed version is available at: https://github.com/guylando/cordova-plugin-file
If you load a remote site https://xxx.com on the webview then it allows to access local files using the url:
https://xxx.com/cdvfile/bundle/www/cordova.js
instead of
cdvfile://localhost/bundle/www/cordova.js
And by this solves the mixed content problems.

[rushabh-makwana-bacancy]

For anyone interested, this is our solution on Android (probably works on iOS too).

Installed dependencies:

• cordova-plugin-file
• cordova-plugin-file-transfer
• cordova-plugin-whitelist (Android only)

import CordovaPromiseFS from 'cordova-promise-fs'

let progress = null
const videoPath = 'video.mp4'

const fs = CordovaPromiseFS({
  persistent: true,
  storageSize: 0,
})

async function download(url) {
  await fs.download(
    url,
    videoPath,
    ({ lengthComputable, total, loaded }) => {
      if (lengthComputable) {
        progress = loaded / total
      }
    }
  )
}

async function showVideo() {
  const fileURL = await fs.toURL(videoPath)
  const localURL = window.Ionic.WebView.convertFileSrc(fileURL)

  document.querySelector('video').src = localURL
}

Further Android configuration:

<!-- added to android/src/main/res/xml/config.xml -->
<widget ...>
  <access origin="*" subdomains="true" />
  <allow-navigation href="http://*/*" />
  <allow-navigation href="https://*/*" />
  <allow-intent href="http://*/*" />
  <allow-intent href="https://*/*" />
  <allow-intent href="tel:*" />
  <allow-intent href="sms:*" />
  <allow-intent href="mailto:*" />
  <allow-intent href="geo:*" />
  ...
</widget>

@diachedelic I tried to download the video from this https://www.learningcontainer.com/wp-content/uploads/2020/05/sample-mp4-file.mp4 URL the way you mentioned above but I an got an error like below:
FileTransferError: {"code":1,"source":"https://www.learningcontainer.com/wp-content/uploads/2020/05/sample-mp4-file.mp4","target":"sample-mp4-file.mp4","http_status":200,"body":null,"exception":"/sample-mp4-file.mp4 (Read-only file system)"}

Could you please let me know what is going wrong with this and does this solution works for iOS too?

[diachedelic]

@rushabh-makwana-bacancy maybe try using https://github.com/triniwiz/capacitor-downloader instead? I no longer use cordova-promise-fs for this use case