Exoplayer: Offlining video with encryption

[moagrius]

I'm not using the template because I'm not filing a bug report or reporting an issue, bur rather requesting a wiki, sample, documentation, or some form of direction

First, thanks to all the contributors for the time and work they'v invested, this is a great product and I appreciate what's involved in maintaining it.

This isn't a bug report so much of a request for direction. The question seems common: playing non-HLS (offline) encrypted video.


https://github.com/google/ExoPlayer/issues/2420
https://github.com/google/ExoPlayer/issues/1964
https://github.com/google/ExoPlayer/issues/1720
https://github.com/google/ExoPlayer/issues/1586
https://github.com/google/ExoPlayer/issues/1241
https://github.com/google/ExoPlayer/issues/1091

I think a simple MVP would go a long way. Hopefully this wouldn't take too long for someone who knows what they're doing.

In our case, we allow users to download a video to internal or external storage, but need to encrypt it so that it's not distributed.

Last year, using ExoPlayer 1, I was able to do this successfully with the RC4 algorithm and a simple CipherInputStream in a custom DataSource. However, seeking was extremely slow (5, 10, 20 seconds), IIUC because the encrypted stream needed to be read from the beginning of the file.

Fast forward to this month, using ExoPlayer 1 again (initially; I quickly moved to ExoPlayer 2), I was able to modify the bytes in the input and output buffers successfully, so could make the content "non-playable" by flipping bits and grunging, but of course this is not secure at all.

In exploring other strategies, it looks like some modes allow random read access (http://libeasy.alwaysdata.net/network/#server), so I moved to AES/CTR/NoPadding (and ExoPlayer 2), which is the same algorithm used by AesCipherDataSource. All of my subsequent attempts have used this algorithm.

My first attempt was to use the AesCipherDataSource with my encrypted content. I encrypted the file "in-app" (using standard stream methods) and copied it to assets. While the IV was different (since AesFlushingCipher makes its own IV in the constructor), the key was the same, and to my understanding it should have been able to decrypt, but it failed with "None of the available extractors" (which I've come to learn usually means the decryption failed). I was hopeful here and am curious why this failed. I'm confident the encryption didn't fail, because I was able to decrypt and read it with alternate approaches (details follow; although none worked perfectly, I was able to consistently get at least playback), and decrypting the same file immediately afterward and playing it with standard DataSource also worked.

I then tried modifying both AesFlushingCipher and AesCipherDataSource to allow me to pass in my own cipher instance. This got closer - I had playback but seeking will throw an error - IIRC it would always error when seeking to a point previous to last played, and sometimes when seeking otherwise (TBH this was many attempts back and I might be misremembering the specifics of the failure, but I did fail).

Returning to the original strategy of using a CipherInputStream (basically copying AssetDataSource with a CipherInputStream around it), again provided close-but-not-quite results - playback was fine but seeking would sometimes throw errors. If you scrubbed "gently" (gave it plenty of time to "warm up", only moved forward in small increments), it would work reasonably well, but seeking immediately or seeking backwards would almost always throw an error: "Top bit not zero".

FWIW I'm using the same sample video I used for the RC4 version last year. During those investigations, I tried several different files and believe the inconsistency is not because of the file (an 11MB mp4).

Would it be possible to provide a working example of a DataSource reading encrypted local files? I'd really appreciate any insight or direction.

Thanks again.

[ojw28]

I think this is relatively straightforward to achieve in V2. First, create a SimpleCache into which you'll offline media. In this example we'll use NoOpCacheEvictor to ensure nothing's evicted from the cache. Note that in a full solution you should pass an evictor implementation that's smart enough to evict media that the user no longer wants to be offlined.

cache = new SimpleCache(cacheDir, new NoOpCacheEvictor());

You'll already have a DataSource.Factory somewhere that you're injecting into your media source. Let's call this defaultFactory. Make a new DataSource.Factory that generates DataSource instances that can read and write from the cache using the AES encryption components:

new DataSource.Factory() {
      @Override
      public DataSource createDataSource() {
        DataSource dataSource = defaultFactory.createDataSource();
        // Wrap the DataSource from the regular factory. This will read media from the cache when
        // available, with the AesCipherDataSource element in the chain performing decryption. When
        // not available media will be read from the wrapped DataSource and written into the cache,
        // with AesCipherDataSink performing encryption.
        return new CacheDataSource(cache, dataSource,
            new AesCipherDataSource(SECRET_KEY, new FileDataSource()),
            new AesCipherDataSink(SECRET_KEY, new CacheDataSink(cache, Long.MAX_VALUE)), 0, null);
      }
    };

If you use this DataSource.Factory for playback you'll get caching with encryption and without eviction. To offline media without playback, you can just read a file through a DataSource chain like:

public void offlineMediaUri(Uri mediaUri) {
  DataSource dataSource = cachingDataSourceFactory.createDataSource();
  Log.i("Offline", "Starting");
  DataSpec dataSpec = new DataSpec(mediaUri);
  try {
    dataSource.open(dataSpec);
    // Pull the media through the DataSource chain.
    byte[] scratchSpace = new byte[32 * 1024];
    int bytesRead = 0;
    while (bytesRead != C.RESULT_END_OF_INPUT) {
      bytesRead = dataSource.read(scratchSpace, 0, scratchSpace.length);
    }
    Log.i("Offline", "Finished");
  } catch (IOException e) {
    Log.e("Offline", "Failed", e);
  } finally {
    Util.closeQuietly(dataSource);
  }
}

Note that if you don't want caching during playback of non-offlined content, you'll need to use a different DataSource.Factory for playback than for offlining. You can create a CacheDataSource that reads from the cache but doesn't write to it by passing a null sink.

[moagrius]

Thank you very much for the response. I think I misidentified some of the requirements (e.g., we won't be able to cache due to space considerations, and I'll need to be able to directly encrypt the download stream - there will never be a clear file on the device), but I should be able to figure it out from what you've posted. Assuming I can get something working, I'll post back for future searchers.

Thanks again!

[ojw28]

You can disable caching by passing null as the sink for the DataSource.Factory used during playback. The code above already directly encrypts the stream prior to it being written to disk, so I think it does what you want.

[moagrius]

it won't be downloaded during playback - it's a download "operation" - the user has to explicitly request a download (which does not result in playback - e.g., they can select multiple downloads from a list, and have multiple downloads running in parallel.

in theory, shouldn't i be able to create an AesFlusingCipher in decrypt mode, read from a download stream, call updateInPlace on the FileOutputStream, then just use a AesCipherDataSource?

E.g.,

AesFlushingCipher aesFlushingCipher = new AesFlushingCipher(Cipher.ENCRYPT_MODE, key, 0, 0);
originalVideoStream = // stream from a downloader or URL connection w/e
fileOutputStream = new FileOutputStream(encryptedFile);
byte[] bufferToHoldReadData = new byte[1024 * 1024];
int sizeOfDataRead = originalVideoStream.read(bufferToHoldReadData);
while (sizeOfDataRead != -1) {
  aesFlushingCipher.updateInPlace(bufferToHoldReadData, 0, sizeOfDataRead);
  fileOutputStream.write(bufferToHoldReadData, 0, sizeOfDataRead);
  sizeOfDataRead = originalVideoStream.read(bufferToHoldReadData);
}

and on the other side, nothing custom, just:

new DataSource.Factory() {
  @Override
  public DataSource createDataSource() {
      new AesCipherDataSource(key, new FileDataSource()),
  }
};

That was the first thing I thought to try but have never got it working. Is the write action incorrect?

[ojw28]

The code sample provided above (offlineMediaUri) is exactly code for a download operation that is not part of playback.

[moagrius]

right, i'm just wondering if we actually need the CacheDataSource at all - could we not skip a lot of the extra plumbing by using an AesCipherDataSource directly?

[ojw28]

Probably, but it's unclear why you're trying to avoid it. It's not a significant amount of extra code for you. Note also that the approach described does nice things like:

• Support resumption of a partially completed download. If offlineMediaUri fails part way through, calling it again will do this. The already downloaded portion will just be read from the offline cache (which is a no-op), and then the part that's not yet downloaded will be read from the upstream source and written to the offline cache. The DataSource chain handles this automatically. You can just call offlineMediaUri repeatedly until it succeeds.
• Playback of a partial download where the media is read from the offline cache or network as required.

[moagrius]

our download mechanic has a great deal of existing infrastructure, saving offline content using a chunked, resumable download api in it's own service, we supply the file name and location based on user prefs, it's download lists of assets in addition to video files, not all content gets encrypted, downloaded content is tracked in a database and might be moved en masse when settings change, many of these files are very large and many of our users are on legacy devices with little storage space, so an extra copy of the file for caching isn't feasible, etc

really i'm just trying to get a minimum working example so i can really _understand_ what's happening.

i appreciate you taking the time to respond and don't want to take advantage of it; i wonder if you see anything in the file write snippet earlier that would explain why the simple approach i described didn't work - is updateInPlace inappropriate here? would an AesCipherDataSource be able to read a file encrypted _without_ an AesFlushingCipher (e.g., encrypted by some other mechanism outside of or previous to the library), or does it need to be written using an AesFlusingCipher as well?

[ojw28]

our download mechanic has a great deal of existing infrastructure, saving offline content using a chunked, resumable download api in it's own service, we supply the file name and location based on user prefs, downloaded content is tracked in a database and might be moved en masse when settings change, but really i'm just trying to get a minimum working example so i can really understand what's happening.

Most of this feels pretty unrelated to the actual mechanism for downloading the media. There's no reason why you can't take the offlineMediaUri snippet and incorporate it into your service instead of your current code for offlining the media. I doubt there's anything stopping you from moving the directory, either. The problem with going for a simple approach is it'll either lack basic functionality or you'll have to re-implement things like download resumption, which then ends up not being simpler at all.

i appreciate you taking the time to respond and don't want to take advantage of it; i wonder if you see anything in the file write snippet earlier that would explain why the simple approach i described didn't work - is updateInPlace inappropriate here? would an AesCipherDataSource be able to read a file encrypted without an AesFlushingCipher (e.g., encrypted by some other mechanism outside of or previous to the library), or does it need to be written using an AesFlusingCipher as well?

I can't see anything obviously wrong with your code. But I maintain the suggested approach is going to be much easier. If you want to go down some alternate path and it's not working, then you'll need to debug what's wrong yourself.

[moagrius]

got it, thanks for your help

[moagrius]

Here's a follow up for future searchers:

It's absolutely possible to get reliable playback _with seeking_ from an encrypted file, if the algorithm used to encrypt the file does not pad. (You can get playback and seek with a padded algorithm, but it can be very slow (I've seen over 10 seconds lag between a tap and a completed seek) - not recommended.)

The key component is the CipherInputStream - both available and skip are problematic. Wrap your existing InputStream (whether that's in assets, or a local file, whatever) in a CipherInputStream with the modifications mentioned below.

I used ASE/CTR/NoPadding during development.

It's documented that CipherInputStream.available should be overridden, so this was less of a surprise and is fairly straightfoward - assuming you used a no-padding cipher (which is a requirement for the technique I'm describing), you should return the available bytes from the wrapped stream. You can query this directly by saving a reference to the wrapped stream, or just get the total/available bytes up front and return that.

The second point of failure, which will either result in a "Top bit not zero" or "None of the available extractors..." error, is from the skip method. Most of the existing DataSource implementations use skip in the open function body to jump to the specific seek/scrub position (dataSpec.position), and might throw an error if the return from skip was less than the desired position (skip returns the number of bytes actually skipped); which would seem to indicate the stream had reached its end:

https://github.com/google/ExoPlayer/blob/release-v2/library/src/main/java/com/google/android/exoplayer2/upstream/AssetDataSource.java#L81

https://github.com/google/ExoPlayer/blob/release-v2/library/src/main/java/com/google/android/exoplayer2/upstream/FileDataSource.java#L70

However, CipherInputStream.skip was consistently returning 0 or less than position. The workaround is simply to read if that happened, until you've processed the requisite number of bytes, basically:

public long forceSkip(long bytesToSkip) throws IOException {
  long processedBytes = 0;
  while (processedBytes < bytesToSkip) {
    long bytesSkipped = skip(bytesToSkip - processedBytes);
    if (bytesSkipped == 0) {
      if (read() == -1) {
        throw new EOFException();
      }
      bytesSkipped = 1;
    }
    processedBytes += bytesSkipped;
  }
  return processedBytes;
}

This should be enough to get most implementations working. I've got a complete, working DataSource implementation if anyone's interested.

HTH

[mostafaHamed]

Hi moagrius,

I appreciate if you can share your complete example to overcome the problems you have faced

[moagrius]

sure. here's the DataSource:

public final class EncryptedFileDataSource implements DataSource {

  private final TransferListener<? super EncryptedFileDataSource> mTransferListener;
  private StreamingCipherInputStream mInputStream;
  private Uri mUri;
  private long mBytesRemaining;
  private boolean mOpened;
  private Cipher mCipher;

  public EncryptedFileDataSource(Cipher cipher, TransferListener<? super EncryptedFileDataSource> listener) {
    mCipher = cipher;
    mTransferListener = listener;
  }

  @Override
  public long open(DataSpec dataSpec) throws EncryptedFileDataSourceException {
    // if we're open, we shouldn't need to open again, fast-fail
    if (mOpened) {
      return mBytesRemaining;
    }
    // #getUri is part of the contract...
    mUri = dataSpec.uri;
    // put all our throwable work in a single block, wrap the error in a custom Exception
    try {
      setupInputStream();
      skipToPosition(dataSpec);
      computeBytesRemaining(dataSpec);
    } catch (IOException e) {
      throw new EncryptedFileDataSourceException(e);
    }
    // if we made it this far, we're open
    mOpened = true;
    // notify
    if (mTransferListener != null) {
      mTransferListener.onTransferStart(this, dataSpec);
    }
    // report
    return mBytesRemaining;
  }

  private void setupInputStream() throws FileNotFoundException {
    File encryptedFile = new File(mUri.getPath());
    FileInputStream fileInputStream = new FileInputStream(encryptedFile);
    mInputStream = new StreamingCipherInputStream(fileInputStream, mCipher);
  }

  private void skipToPosition(DataSpec dataSpec) throws IOException {
    mInputStream.forceSkip(dataSpec.position);
  }

  private void computeBytesRemaining(DataSpec dataSpec) throws IOException {
    if (dataSpec.length != C.LENGTH_UNSET) {
      mBytesRemaining = dataSpec.length;
    } else {
      mBytesRemaining = mInputStream.available();
      if (mBytesRemaining == Integer.MAX_VALUE) {
        mBytesRemaining = C.LENGTH_UNSET;
      }
    }
  }

  @Override
  public int read(byte[] buffer, int offset, int readLength) throws EncryptedFileDataSourceException {
    // fast-fail if there's 0 quantity requested or we think we've already processed everything
    if (readLength == 0) {
      return 0;
    } else if (mBytesRemaining == 0) {
      return C.RESULT_END_OF_INPUT;
    }
    // constrain the read length and try to read from the cipher input stream
    int bytesToRead = getBytesToRead(readLength);
    int bytesRead;
    try {
      bytesRead = mInputStream.read(buffer, offset, bytesToRead);
    } catch (IOException e) {
      throw new EncryptedFileDataSourceException(e);
    }
    // if we get a -1 that means we failed to read - we're either going to EOF error or broadcast EOF
    if (bytesRead == -1) {
      if (mBytesRemaining != C.LENGTH_UNSET) {
        throw new EncryptedFileDataSourceException(new EOFException());
      }
      return C.RESULT_END_OF_INPUT;
    }
    // we can't decrement bytes remaining if it's just a flag representation (as opposed to a mutable numeric quantity)
    if (mBytesRemaining != C.LENGTH_UNSET) {
      mBytesRemaining -= bytesRead;
    }
    // notify
    if (mTransferListener != null) {
      mTransferListener.onBytesTransferred(this, bytesRead);
    }
    // report
    return bytesRead;
  }

  private int getBytesToRead(int bytesToRead) {
    if (mBytesRemaining == C.LENGTH_UNSET) {
      return bytesToRead;
    }
    return (int) Math.min(mBytesRemaining, bytesToRead);
  }

  @Override
  public Uri getUri() {
    return mUri;
  }

  @Override
  public void close() throws EncryptedFileDataSourceException {
    mUri = null;
    try {
      if (mInputStream != null) {
        mInputStream.close();
      }
    } catch (IOException e) {
      throw new EncryptedFileDataSourceException(e);
    } finally {
      mInputStream = null;
      if (mOpened) {
        mOpened = false;
        if (mTransferListener != null) {
          mTransferListener.onTransferEnd(this);
        }
      }
    }
  }

  public static final class EncryptedFileDataSourceException extends IOException {
    public EncryptedFileDataSourceException(IOException cause) {
      super(cause);
    }
  }

  public static class StreamingCipherInputStream extends CipherInputStream {

    private int mBytesAvailable;

    public StreamingCipherInputStream(InputStream is, Cipher c) {
      super(is, c);
      try {
        mBytesAvailable = is.available();
      } catch (IOException e) {
        // let it be 0
      }
    }

    // if the CipherInputStream has returns 0 from #skip, #read out enough bytes to get where we need to be
    public long forceSkip(long bytesToSkip) throws IOException {
      long processedBytes = 0;
      while (processedBytes < bytesToSkip) {
        long bytesSkipped = skip(bytesToSkip - processedBytes);
        if (bytesSkipped == 0) {
          if (read() == -1) {
            throw new EOFException();
          }
          bytesSkipped = 1;
        }
        processedBytes += bytesSkipped;
      }
      return processedBytes;
    }

    // We need to return the available bytes from the upstream.
    // In this implementation we're front loading it, but it's possible the value might change during the lifetime
    // of this instance, and reference to the stream should be retained and queried for available bytes instead
    @Override
    public int available() throws IOException {
      return mBytesAvailable;
    }
  }

}

if you're in the demo, just make a Factory to produce EncryptedFileDataSource instances, and update the json with some entries that have uri members that point to encrypted files.

Make sure it gets the correct Cipher instance and you're good to go.

HTH, GL

[rafaeladel]

Thanks for sharing your solution.
But I got a problem with seeking through a large file (~600MB), It takes around 30 seconds to seek to the middle of the file. Have you found any solution or work around to minimize this problem ?
I tried AES/CBC/NoPadding, AES/CTR/NoPadding, AES/ECB/NoPadding. And also all their with padding variants.

[moagrius]

@rafaeladel i tested mostly with smaller videos and seemed to get really snappy playback, but after reading your post, i downloaded a larger video (26 mins; I don't know the file size without firing up AS) and I'm getting seek delays of 5 or 6 seconds near the end of the video - and this is on a new Pixel so i can imagine older phones with larger files could easily see the 30 second seek. The fact that seek operations near the start go much faster than seek operations near the end of the video (regardless of distance from last position) makes me think that we're not really getting random access...

I don't have anything to offer, unfortunately. I wonder if @ojw28 might have any insight?

[rafaeladel]

I've read here and there that seeking re-decrypts everything from the beginning till the seek point. I wonder if there's a way to make it starts decryption at the very point of seeking and so forth. If you know what class is responsible for this, maybe it can be achieved.

Waiting for @ojw28 input too.

[moagrius]

i just tried an alternate approach that i'd abandoned in favor of a direct data source, which involved a streaming HTTP server in-app, providing the decrypted content via byte ranges (the approach that the commercial product libeasy http://libeasy.alwaysdata.net/ uses with MediaPlayer), and it's basically the same result (in fact maybe even a tiny bit worse).

using a 26 minute, 160MB mp4 for testing. AES/CTR/NoPadding.

[ojw28]

@rafaeladel - You'd be much better off just following the official advice in my first response to this issue, including use of the library provided AesCipherDataSource and AesCipherDataSink components. There's really no need to reinvent the wheel.

[moagrius]

@ojw28 what if someone has content coming in encrypted, or is otherwise unable to encrypt in-app?

[ojw28]

• If you're serving content using https then it seems unnecessary for it to be further encrypted for transmission. If you're not serving content using https then I would strongly urge you to do so.
• If you're serving content encrypted despite the above using AES/CTR/NoPadding then I think the advice in my first response still more or less applies. You just need to tweak the DataSource chains a little (i.e. remove AesCipherDataSink from the chain used for downloading since the media is already encrypted) to make sure that the right steps occur in the chain.
• If you're serving the content encrypted with something other than AES/CTR/NoPadding, then I guess at that point you need to go down a custom implementation route.
• It's unclear what "otherwise unable to encrypt in-app" refers to here, since I can't think of any valid cases where this statement would hold?

[moagrius]

It's unclear what "otherwise unable to encrypt in-app" refers to here, since I can't think of any valid cases where this statement would hold?

e.g., downloading content from an http server, but not streaming it. e.g., provided by a third-party api, or several encrypted videos in a gzip, or a video delivered from another source (e.g., a publisher), etc

fwiw i have tried to modify the DataSource chains, and work with the AesFlusingCipher class as well, but no luck (definitely because of my lack of knowledge around this domain).

[rafaeladel]

@ojw28 I've tried implementing the first response code. For playing a large file (~600MB) It takes time in decryption for the first time if I seek to the middle of the file for example. Also it stops in the middle of the playback for no reason with no errors.

Is there a way that I can tell it to start the decryption from where I specified the seek ?

[ojw28]

I'm unsure what the problem is that you're seeing, but I don't think it's decrypting the entire file up to the seek position. That's just not what the code does. If you look at the implementation you should be able to verify that it starts decryption from the specified position offset.

[rafaeladel]

@ojw28 Would you mind pointing which file you're talking about ? because I've been searching for that part in the entire library.
Thanks.

[moagrius]

@rafaeladel hopefully @ojw28 and you can get your issue resolved, but in case it helps, i was able to get really nice playback on large videos using exoplayer in conjunction with another third party library called "libeasy", which is an in-app streaming server (http://libeasy.alwaysdata.net/), although i'm confident the fact that's it's a server is irrelevant (you can get roughly the same results as my earlier post using a SocketServer instead of a DataSource).

unfortunately it's a commercial product and is not open source (and in fact is obfuscated), but the playback was just too big of an issue. i'd like to continue to investigate but needed to move on to other work - at this point i must be around 100 hours of research and failed experiments. hopefully i can come back to it when things settle down. really there's no reason that the fact that it's a server should matter at all - i strongly suspect the difference comes down to their FilterOutputStream implementation, but as I said, it's not open.

i also revisited the AesFlushingCipher pattern and tried to modify that but still failed - I can get playback but seeking always crashes with the Top bit not zero. The basic approach was:

1. Create a factory that produces standard Cipher instances (both encryption and decryption) with a predictable key and IV (both are per user, per install).
2. Modify AesFlushingCipher to take a Cipher instance (per 1), and an offset.
3. During download (using google's MediaHttpDownloader, which writes to an OutputStream), wrap it in a FilterOutputStream that calls AesFlushingCipher.updateInPlace just before it calls to the upstream's write method, as is shown in AesDataSink https://github.com/google/ExoPlayer/blob/release-v2/library/src/main/java/com/google/android/exoplayer2/upstream/crypto/AesCipherDataSink.java#L72
4. During playback, using a RandomAccessFile with an AesFlushingCipher, that's provided a decryption Cipher (per 1) and combines the open and read logic from https://github.com/google/ExoPlayer/blob/release-v2/library/src/main/java/com/google/android/exoplayer2/upstream/crypto/AesCipherDataSource.java and https://github.com/google/ExoPlayer/blob/release-v2/library/src/main/java/com/google/android/exoplayer2/upstream/FileDataSource.java

I was hopeful this would work, and is (as far as I can tell), roughly what the CacheDataSource is doing behind the scenes, but again I get playback from zero, but any seek actions crash with Top bit not zero.

for anyone else running into these issues, i really think the key is the class that reads the encrypted content and passes those bytes to exoplayer. using "traditional" CipherInput/OutputStreams works with modifications (https://github.com/google/ExoPlayer/issues/2467#issuecomment-281856611), but seek on large files is still slow. i'm fairly certain that libeasy does not use a flushing cipher (although they may do something similar), and looking at what i can see through the IDE are implementing a custom FilterInputStream with a Cipher, but that implementation definitely seems to provided the snappiest seek.

[rafaeladel]

Thank you @moagrius for your time and effort. I'll take a look at libeasy. I'll also try the approach you specified.

But for what it's worth, I was able to identify the point at which it takes a lot of time seeking to the middle or the end of the video/audio file, It's in forceSkip method inside your EncryptedFileDataSource class, It loops and reads bytes up to the selected one, meaning it has an O(n) complexity, I didn't investigate it yet but you've mentioned that because of CipherInputStream's skip method returning 0.

[moagrius]

@rafaeladel ha yeah i had that same thought and left myself a reminder in the middle of the night. i also think we might need to round to block size... and yeah the base implementation of CipherInputStream does decrypt from 0 when skip is called, AFAICT; maybe since we have a streaming mode we don't need to do that... i'll try to find some time to mess with this further... thanks for following up

[moagrius]

@rafaeladel try this update and let me know what you get. for the skip, we're using the native skip for the upstream stream, then forcing the cipher to update but without reading. it needs to update to exactly the same point.

i tried it with a 700MB, 85 minute video, and am getting less than a second delay all the way to the end. let me know what results you get.

it could probably use some EOF checks, min against bytesRemaining, etc, but that can be added later if the performance is good.

public class StreamingCipherInputStream extends CipherInputStream {

  private InputStream mUpstream;
  private Cipher mCipher;

  public StreamingCipherInputStream(InputStream is, Cipher c) {
    super(is, c);
    mUpstream = is;
    mCipher = c;
  }

  @Override
  public int read(byte[] b, int off, int len) throws IOException {
    return super.read(b, off, len);
  }

  public long forceSkip(long bytesToSkip) throws IOException {
    mUpstream.skip(bytesToSkip);
    long skipped = 0;
    // Integer.MAX_VALUE is basically 2GB, so it's _possible_ we might have a video that size
    // but we can't make a buffer that big...  repeat until satisfied
    int skipSize = (int) Math.min(bytesToSkip, 1024 * 1024);  // lets go 1MB at a time
    byte[] skipBuffer = new byte[skipSize];
    while (skipped < bytesToSkip) {
      // later iterations may bring skip size down, and we want to update exactly
      long remaining = bytesToSkip - skipped;
      if (remaining < skipSize) {
        skipSize = (int) remaining;
        skipBuffer = new byte[skipSize];
      }
      try {
        skipped += mCipher.update(skipBuffer, 0, skipSize, skipBuffer);
      } catch (ShortBufferException e) {
        // should never happen, we're sending the same input as output
        break;
      }
    }
    return skipped;
  }

  // We need to return the available bytes from the upstream.
  // In this implementation we're front loading it, but it's possible the value might change during the lifetime
  // of this instance, and reference to the stream should be retained and queried for available bytes instead
  @Override
  public int available() throws IOException {
    return mUpstream.available();
  }
}

fwiw, there's a really efficient way to do this by precomputing the exact offset you'll need, but i don't 100% understand the logic behind it so am really hesitant to include it. but you can basically take the approach from http://stackoverflow.com/a/23744623/6585616, which was modified by another user with a simple demo here http://stackoverflow.com/a/39974172/6585616 - if i'm reading that right the skip buffer is always going to be smaller than the block size (16 bytes), which is awesome - he's computing that using the IV somehow - i tried to break it down but wasn't able to keep up - especially with the magnitude of the bigint - i may revisit but the class posted above is working well here - i'm anxious to hear if you get good results as well.

[moagrius]

just tried on older samsung tablet, not as bad but still slow... going to try the iv offset thing...

[bbincybbaby]

@moagrius can you please share your contact ? mailid or something

[moagrius]

sorry for the spamming this thread - i finally ended up using the cipher skip shortcut i mentioned earlier, which is super slick IMO but i can't get my head around the biginteger/iv stuff... the pertinent changes are based entirely off work by stackoverflow user Maarten Bodewes and i not only take no credit but freely admit i can't follow the logic between the 2 inline comments in forceSkip.

that said, this was tested with a number of really large videos (600, 700, 800 mb, 1-2 hours in length), as well as some more "normal" files (double digit MB and minutes), and seek is pretty much instantaneous. physical devices used include Pixel on Nougat, Nexus 6 on Marshmallow, Samsung Tab on Lollipop, Samsung Tag on Marshmallow, Xiaomi Mi Pad (didn't ask OS version).

clearly you're going to have to pass more information to the StreamingCipherInputStream than previously; just fill in what's missing.

last note on this: no idea why, but using a SocketServer and a ContentProvider (and _not_ using the libmedia library), just serving byte ranges via HTTP using a _local, in-app_ instance of SocketServer, not only was playback pretty much identical, but you can throw pretty much any reader at it - I must have tried 4 or 5 different versions of hastily thrown-together stream readers (including one with a RandomAccessFile) - all of which worked with really snappy seek with the SocketServer, and all failed with direct-to-exoplayer DataSource with either "Top bit not zero" or "None of the available extractors" - for the record, i'm not blaming ExoPlayer, and i have no idea why piping the bytes through a server would make such a big difference. also, i only tested this approach with videos up to 160MB, and didn't pursue it because it seems even more magical than this shortcut below.

public static class StreamingCipherInputStream extends CipherInputStream {

  private static final int AES_BLOCK_SIZE = 16;

  private InputStream mUpstream;
  private Cipher mCipher;
  private SecretKeySpec mSecretKeySpec;
  private IvParameterSpec mIvParameterSpec;

  public StreamingCipherInputStream(InputStream inputStream, Cipher cipher, SecretKeySpec secretKeySpec, IvParameterSpec ivParameterSpec) {
    super(inputStream, cipher);
    mUpstream = inputStream;
    mCipher = cipher;
    mSecretKeySpec = secretKeySpec;
    mIvParameterSpec = ivParameterSpec;
  }

  @Override
  public int read(byte[] b, int off, int len) throws IOException {
    return super.read(b, off, len);
  }

  public long forceSkip(long bytesToSkip) throws IOException {
    long skipped = mUpstream.skip(bytesToSkip);
    try {
      int skip = (int) (bytesToSkip % AES_BLOCK_SIZE);
      long blockOffset = bytesToSkip - skip;
      long numberOfBlocks = blockOffset / AES_BLOCK_SIZE;
      // from here to the next inline comment, i don't understand
      BigInteger ivForOffsetAsBigInteger = new BigInteger(1, mIvParameterSpec.getIV()).add(BigInteger.valueOf(numberOfBlocks));
      byte[] ivForOffsetByteArray = ivForOffsetAsBigInteger.toByteArray();
      IvParameterSpec computedIvParameterSpecForOffset;
      if (ivForOffsetByteArray.length < AES_BLOCK_SIZE) {
        byte[] resizedIvForOffsetByteArray = new byte[AES_BLOCK_SIZE];
        System.arraycopy(ivForOffsetByteArray, 0, resizedIvForOffsetByteArray, AES_BLOCK_SIZE - ivForOffsetByteArray.length, ivForOffsetByteArray.length);
        computedIvParameterSpecForOffset = new IvParameterSpec(resizedIvForOffsetByteArray);
      } else {
        computedIvParameterSpecForOffset = new IvParameterSpec(ivForOffsetByteArray, ivForOffsetByteArray.length - AES_BLOCK_SIZE, AES_BLOCK_SIZE);
      }
      mCipher.init(Cipher.ENCRYPT_MODE, mSecretKeySpec, computedIvParameterSpecForOffset);
      byte[] skipBuffer = new byte[skip];
      // i get that we need to update, but i don't get how we're able to take the shortcut from here to the previous comment
      mCipher.update(skipBuffer, 0, skip, skipBuffer);
      Arrays.fill(skipBuffer, (byte) 0);
    } catch (Exception e) {
      return 0;
    }
    return skipped;
  }

  // We need to return the available bytes from the upstream.
  // In this implementation we're front loading it, but it's possible the value might change during the lifetime
  // of this instance, and reference to the stream should be retained and queried for available bytes instead
  @Override
  public int available() throws IOException {
    return mUpstream.available();
  }

}

[moagrius]

@bbincybbaby you can just use this thread :) what's up?

[rafaeladel]

@moagrius That's great ! using mCipher.update() apparently is a lot faster than read() , since we're not actually reading as far as I know. The seeking time has decreased a lot.
As for the last code snippet you put, I don't really follow how it works either, I'll try to track it slowly and see what it does.

Thank you so much for your time and effort. That helped a lot addressing a lot of issues we're facing in our workplace.

[Slim1991]

@moagrius I'm very admire your work. Can you give me your example project?

[moagrius]

@rafaeladel great, glad it was helpful. fwiw i'd definitely suggest using that last snippet - it's virtually instantaneous, no delay at all even with 800MB, 90 minute encrypted video seeking to any position.

@Slim1991 i'll try to to put a demo up in the next week or so

[Slim1991]

@moagrius I'm planning to play AES-128 encrypted HLS, you will save so much time for me. Just by ExoPlayer's Document can't help me but you can. You are so good!

[moagrius]

@Slim1991 i _think_ there's already functionality built into ExoPlayer for streaming HLS video (note my comments about how things are much easier with a server); what we're doing is around playback of locally encrypted files, or remote encrypted files that aren't being served via HLS

[Slim1991]

@moagrius Yeah. I know. I'm still working on that. Normal HLS is easy to play with ExoPlayer but encrypted HLS is harder, no example found and no document found. Your work just give me some suggestions to do that.

[moagrius]

yeah you should be able to use the same techniques shown here - just replace the FileInputStream with whatever Stream you're getting from your networking layer

[Slim1991]

That's right!
Integrating the decryption and knowing how to do that are important to me.
I hope I receive your project soon to do that. Thanks

[devm2024]

@moagrius
I have tried your code, it works for fine for small files, 15MB, but I am getting EOF exception for large files 25MBs. Not sure what to do, probbaly I am doing something wrong.
Here is my initialization of Exoplayer:

`File file=FileBackend.getEncryptedVideoFile(video_name);
Uri uri= Uri.fromFile(file);

    dataSourceFactory= new MyDataSourcefactory(activity, Util.getUserAgent(this, "ExoPlayerVideoActivity"),uri,listener,true);

    ExtractorsFactory extractorsFactory = new DefaultExtractorsFactory();
    Uri video_uri = Uri.parse(file.toString());

    videoSource = new ExtractorMediaSource(video_uri, dataSourceFactory, extractorsFactory, null, null);`

My Custom datasource create source :

`@Override
public DataSource createDataSource() {

    Cipher cipher = somethnig;
    SecretKeySpec keySpec=something;
    IvParameterSpec ivSpec=something;
    return new NewEncryptedDataSource(cipher,listener, keySpec,ivSpec);
}`

Please help, I am using direct data source and not the nanohttpd.

[moagrius]

@devm2024 hm, i've tried with lots of very large videos (600MB+) and have not seen that.

are you using the most recent snippet for your stream reader? the one from https://github.com/google/ExoPlayer/issues/2467#issuecomment-283471153

also this technique only works for AES/CTR/NoPadding - is that your algorithm?

are you making sure to send the same key _and_ iv spec to the cipher to decrypt as you use to encrypt?

[devm2024]

Hi @moagrius :
Yes I have tried both recent snippet and the previous one. My cipher is this

Cipher cipher = Cipher.getInstance("AES/CTR/NoPadding", "BC");
keys are same. I am able to play smaller videos, getting EOF in larger ones.
Can you please provide a demo project. It would be really helpful.
Thanks.

[Slim1991]

@devm2024 your init vector should has 16 bytes. "BC" is not enough

[devm2024]

@Slim1991
I am not sure but "BC" is the provider name.
My keys are in this form:
SecretKeySpec keySpec = new SecretKeySpec("16-char key".getBytes(), "AES"); IvParameterSpec ivSpec = new IvParameterSpec("16-char-key".getBytes());

Let me know if I am wrong and suggest the replacement of "BC".

[moagrius]

"BC" is "Bouncy Castle" and is the provider not the IV. I didn't use it but IIUC it shouldn't matter.

The only other thing I can think to suggest is to create new Cipher instances for each operation (don't re-use).

I'll try to get a demo up this weekend, it definitely won't happen today or tomorrow though (work and school). When I do put it up, it definitely won't have large videos included, you'll need to supply those (probably clean, unencrypted).

[devm2024]

Hi @moagrius tried with creating new cipher instances, didn't worked. I suppose there is something wrong in the way I initialize exoplayer. If would be really helpful if you just provide me the code snippet of your exoplayer initialization.

[moagrius]

in my actual project the player initialization is very complicated and tied into a lot of other pieces of the production app, so i can't paste it simply. that said, when i was experimenting i just used the demo project, and then another project with the "simple setup". i don't think the initialization is the problem if you get playback but slow seek. like i said, i can probably put up a project but it's going to be this weekend at the earliest

[moagrius]

Here's the most simple example I could come up with: https://github.com/moagrius/EncryptedExoPlayerDemo

It's got a download and encrypt function built in, or you can supply your own but you'll need to modify the keys and iv (which are generated using SecureRandom right now). If you want to use the download and encrypt, you'll need to supply the URL of a remote clean (NOT encrypted) mp4 file. You can of course supply the URL of a remote _encrypted_ file, just remove the CipherOutputStream stuff.

I think this is as bare-bones as I can get it - there's not much else to remove.

I tested with a few small videos I found on example sites, and with a private file about 150MB, 30 minutes. All worked great, virtually no delay in seek.

You'll get more accurate results testing on a real device.

HTH, GL

[rafaeladel]

@moagrius Thanks for the repo.
PS: you may want to change the package name from com.test.exoplayer2 to something else other than test because it confused me at the beginning.

[rafaeladel]

@moagrius Have you tried using your EncryptedFileDataSource.java to download an audio stream and decode it on the fly ? I tried to modify the method setupInputStream() as in this gist.
I tried it and It always gives me this known error:

com.google.android.exoplayer2.source.UnrecognizedInputFormatException: None of the available extractors (MatroskaExtractor, FragmentedMp4Extractor, Mp4Extractor, Mp3Extractor, AdtsExtractor, Ac3Extractor, TsExtractor, FlvExtractor, OggExtractor, PsExtractor, WavExtractor) could read the stream.

[moagrius]

I have not tried that - I've only been working with local files. That error probably just means it's not decrypting properly.

1. Are you sure you're using the same Cipher on both the encrypt and decrypt? The key _and_ the IV need to be the same byte array, not sure about the Provider.
2. Try supplying a Cipher factor, and getting a new Cipher for each open.

What happens if you download the file manually and try to play it? I suspect you'll get the same errors, but am curious...

[rafaeladel]

@moagrius I tried downloading the file to a temp file and playing it .. It works perfectly.
I don't know what the problem is, but I think the StreamingCipherInputStream is not downloading the file at all when you call read() or something.

[moagrius]

hm nothing's obviously out of order... maybe get rid of the BufferedInputStream and wrap the connection directly in the StreamingCipherInputStreamReader... also strange because everything i tried previously over a server (granted, mine were local) worked really well; it was much harder to get the direct data source to work with fast seek than it was when using a local server...

i might have some time next week to poke around but really have no suggestions, other than the above, at the moment.

[starstar3306]

thx moagrius..

it is really good source.

but i still need your help.

you use AES CTR but my AES method is AES CBC

i cant seek movie.

can you help me?

please. have a good day

[RajatSingh]

Hi Moagrius,

I am trying to download video from this url "http://storage.googleapis.com/vrview/examples/video/hls/congo.m3u8 ".

But getting below error while playing the video: Please help

com.test.exoplayer2 E/ExoPlayerImplInternal: Source error.
com.google.android.exoplayer2.source.UnrecognizedInputFormatException: None of the available extractors (MatroskaExtractor, FragmentedMp4Extractor, Mp4Extractor, Mp3Extractor, AdtsExtractor, Ac3Extractor, TsExtractor, FlvExtractor, OggExtractor, PsExtractor, WavExtractor) could read the stream.
at com.google.android.exoplayer2.source.ExtractorMediaPeriod$ExtractorHolder.selectExtractor(ExtractorMediaPeriod.java:705)
at com.google.android.exoplayer2.source.ExtractorMediaPeriod$ExtractingLoadable.load(ExtractorMediaPeriod.java:628)
at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:295)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:423)
at java.util.concurrent.FutureTask.run(FutureTask.java:237)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
at java.lang.Thread.run(Thread.java:818)

[andrewlewis]

@RajatSingh That link points to an HLS stream, not a media container file. See Why do some streams fail with UnrecognizedInputFormatException?.

[RajatSingh]

@andrewlewis What about this url?

http://vod.nexg.tv/video/v/amz3/brd115/17/05/ent/Shahrukh_And_Salman_Top_10_Richest_Actors_Of_Bollywood.smil/chunklist_w894125283_b80000.m3u8?sf=mon

I just want to download video from this url to our local device in encrypted form and play it using this sample. Please help. Thanks in advance.

[DiGra]

@moagrius , Hi. I have question about your code. I tried it with "AES/CTR/NoPadding" it's works perfectly for all formats. But my question is what should I do with "PBEWithMD5AndDES" ? My issue is that cipher.update works with .mp3 files, but not with .m4a files. I can't understand why. So question is can I somehow to update cipher for .m4a or not?

[moagrius]

let me first disclaim that i am no expert in cryptography or video playback. that said, the trick with the "wrap around" skip is specific to AES/CTR, but if you take that out it might work. I don't know anything about PBEWithMD5AndDES - google says it's a block cipher (predictable size) but I don't know if it's a streaming cipher... if you go to the post right before the last one (with the special skip technique), that'd presumably work for any streaming cipher. AFAIK the extension (mp3 vs m4a) doesn't matter, just how it's encrypted. Another cheap workaround it to use a local HTTP server - there are lots of examples of that laying around the googles...

[moagrius]

@DiGra ^

[DiGra]

@moagrius Yes, thanks. There is only one way out, read the file in larger blocks than the player asks (what we can decrypt), and give him only the parts that he needs. I do not really want to go into the logic of the InputStream for read()/ skip(). In theory, this can be done, but so far I do not have time for this, there are more priority tasks for now.

[coffeeforyou]

Just wanted to say thanks. Amazing code, works perfectly, easy to adjust and saved me lots of time.

[moagrius]

Just wanted to say thanks. Amazing code, works perfectly, easy to adjust and saved me lots of time.

HTH!

[DiGra]

Hi guys. If someone have situation like me (using PBEWithMD5AndDES instead of normal AES/CTR for this tasks) here is working code:

public long skip(long byteCount) throws IOException {
            skipOffset = (int) (byteCount % mCipher.getBlockSize());
            long skipped = mUpstream.skip(byteCount - skipOffset);
                        try {
                     AlgorithmParameterSpec paramSpec = new PBEParameterSpec(iv, iterationCount);
                     mCipher.init(Cipher.DECRYPT_MODE, *your_key*, paramSpec);
                } catch (Exception e) {
                       return 0;
                }
            return skipped + super.read(new byte[skipOffset], 0, skipOffset);
}