Core: Manual decryption tool

[tflidd]

On the forum, the question about how to manually decrypt files comes up regularly. There has been a blog from @schiesbn : http://blog.schiessle.org/2013/05/28/introduction-to-the-new-owncloud-encryption-app/comment-page-1/#comment-61180 and there has been some modification that you use an additional header: https://forum.owncloud.org/viewtopic.php?f=23&t=14110&p=73608&hilit=decrypt#p73983

Unfortunately, I wasn't able to restore a file. First I used the script as it is, it worked without problem except that the output was binary. I used the hint to remove the header (I did it manually) but it returns:

Warning:  openssl_open(): unable to coerce parameter 4 into a private key in /var/www/owncloud/apps/files_encryption/lib/crypt.php on line 472
PHP Fatal error:  Class 'OCA\Encryption\Exceptions\MultiKeyDecryptException' not found in /var/www/owncloud/apps/files_encryption/lib/crypt2.php on line 477

I did define the cipher:

$decryptedContent = OCA\Encryption\Crypt::symmetricDecryptFileContent($encryptedContent, $decryptedKeyfile,'AES-256-CFB');

The forum post mentions a first block to remove (contains "--------") which I did but it didn't change the error. Probably it's some stupid error.

Did you think about a manual encryption/decryption function with the occ-tool? There are a lot of users with old encrypted data they'd like to decrypt, people with a big amount of data but when encrypting/decrypting it via the interface they run into timeouts.

The script I used:

<?PHP
require_once '../apps/files_encryption/lib/crypt.php';
// first get users private key and decrypt it
$encryptedUserKey = file_get_contents("user.private.key");
$decryptedUserKey = OCA\Encryption\Crypt::decryptPrivateKey($encryptedUserKey, "myuserpassword");

// now we need to decrypt the file-key, therefore we use our private key and the share key
$shareKey = file_get_contents("file.txt.user.shareKey");
$encryptedKeyfile = file_get_contents("file.txt.key");
$decryptedKeyfile = OCA\Encryption\Crypt::multiKeyDecrypt($encryptedKeyfile, $shareKey, $decryptedUserKey);

// finally we can use the decrypted file-key to decrypt the file
$encryptedContent = file_get_contents("file.txt");
$decryptedContent = OCA\Encryption\Crypt::symmetricDecryptFileContent($encryptedContent, $decryptedKeyfile,'AES-256-CFB');

file_put_contents('output.txt', $decryptedContent);
?>

[karlitschek]

@schiesbn

[smu]

+1 for an offline encryption tool.

I had the same problem some time ago and never found a solution. In the end I decided to disable the encryption app.

[seguri]

+1

This is my version of the now famous decryption code. I'm using owncloud 8.0
I'm testing it with a jpeg that I can't successfully decrypt :(

<?php

require_once 'crypt.php';
use OCA\Files_Encryption\Crypt;

function die_($msg) {
    fwrite(STDERR, $msg);
    exit(1);
}

if ($argc !== 6) die_("Usage: php decrypt.php <privateKey> <privateKeyPassword> <shareKey> <fileKey> <ecnryptedFile>\n");

// first get users private key and decrypt it
$encryptedUserKey = file_get_contents($argv[1]);
$decryptedUserKey = Crypt::decryptPrivateKey($encryptedUserKey, $argv[2]);
if ($decryptedUserKey === false) die_("ERROR: could not decrypt private key.\n");

// now we need to decrypt the file-key, therefore we use our private key and the share key
$shareKey = file_get_contents($argv[3]);
$encryptedKeyfile = file_get_contents($argv[4]);
$decryptedKeyfile = Crypt::multiKeyDecrypt($encryptedKeyfile, $shareKey, $decryptedUserKey);

// finally we can use the decrypted file-key to decrypt the file
$encryptedContent = file_get_contents($argv[5]);
// is this necessary? copied from crypt.php:354
// if (Crypt::isHeader($encryptedContent) === true) {
//     $encryptedContent = substr($encryptedContent, strpos($encryptedContent, Crypt::HEADEREND) + strlen(Crypt::HEADEREND));
// }
$decryptedContent = Crypt::symmetricDecryptFileContent($encryptedContent, $decryptedKeyfile);

// save decrypted file
if ($decryptedContent === false) {
    die_("Error decrypting file");
} else {
    file_put_contents("/tmp/out.jpg", $decryptedContent);
}

?>

[DeepDiver1975]

I'm honestly questioning the usability of such a command - we would need to ask for the user's password to get access to the private key.

Well - it might make sense for the admin in case recovery is enabled.

@schiesbn we might want to consider this architecturally for 8.1 with the new encryption architecture.

[smu]

I think some external tools I needed to recover encrypted files from external backups.

• I perform regular backups of my owncloud data directory (containing the encrypted files)
• at some point in time I might need to access (recover) a file from the backup, which no longer exists in data/
• so far, I have no possibility to read this (encrypted) file.

[seguri]

(EDIT) +1 smu

Consider someone using owncloud recommended backup policies through rsync. Or a user leaving owncloud.

How do they recover encrypted files from the backup?

We should start at least with a script working from console, then eventually extend it as a new web interface function.

[tflidd]

I found a way to recover files:
Install a new owncloud, create a user/password, they should be the same as the old username and his password. Login to create keys and initialize encryption, logout again.

Then replace all the user's files by the backup, including all keyfiles (shared-keys, keyfiles, private key).

Rescan the user's file

sudo -u www-data php /path/to/owncloud/occ files:scan username

Login and you should find all files.

[seguri]

I think you forgot to mention that you need to restore also the old SALT or this will be useless.

We still need a way to decrypt encrypted files; what should we do with your method, download all pictures one by one through the web interface?

(Don't get me wrong, I appreciate your solution cause it gives us some relief)

[tflidd]

Is the salt used as well for the keyfiles? I have no passwordsalt in my config.php (only a secret). As far as I remember, it worked without any salt (https://forum.owncloud.org/viewtopic.php?f=23&t=14110&start=10#p78459). In your provided script https://github.com/owncloud/core/issues/13591#issuecomment-77160718 you didn't set a salt either!

[mpellegrin]

Hi, I don't think the salt is used in keyfiles. Here is the code I managed to use to decrypt my files :

<?php

// Replace these with your custom values
$datadir = '_PATH_TO_DATADIR_INCLUDING_TRAILING_SLASH_';
$filepath = 'some/directory/relative/to/datadir/myfile.txt';
$username = '_OWNCLOUD_LOGIN_';
$password = '_OWNCLOUD_PASSWORD_';

require_once '_PATH_TO_OWNCLOUD_/apps/files_encryption/lib/crypt.php';

// first get users private key and decrypt it
$encryptedUserKey = file_get_contents($datadir . $username . '/files_encryption/' . $username . '.privateKey');
$decryptedUserKey = OCA\Files_Encryption\Crypt::decryptPrivateKey($encryptedUserKey, $password);

// now we need to decrypt the file-key, therefore we use our private key and the share key
$shareKey = file_get_contents($datadir . $username . '/files_encryption/keys/' . $filepath . '/' . $username . '.shareKey');
$encryptedKeyfile = file_get_contents($datadir . $username . '/files_encryption/keys/' . $filepath . '/fileKey');
$decryptedKeyfile = OCA\Files_Encryption\Crypt::multiKeyDecrypt($encryptedKeyfile, $shareKey, $decryptedUserKey);

// finally we can use the decrypted file-key to decrypt the file
// but first, strip header block
$handle = fopen($datadir . $username . '/files/' . $filepath, 'r');
$encryptedContent = '';
while ($data = fread($handle, OCA\Files_Encryption\Crypt::BLOCKSIZE)) {
    // if this block contained the header we move on to the next block
    if (OCA\Files_Encryption\Crypt::isHeader($data)) {
            $header = OCA\Files_Encryption\Crypt::parseHeader($data);
            $cipher = OCA\Files_Encryption\Crypt::getCipher($header);
    } else {
        $encryptedContent .= $data;
    }
}
fclose($handle);

// Actually decrypt file
$decryptedContent .= OCA\Files_Encryption\Crypt::symmetricDecryptFileContent($encryptedContent, $decryptedKeyfile, $cipher);

// Save the decrypted file
file_put_contents(str_replace('/', '-', $filepath), $decryptedContent);

[riemerio]

@mpellegrin Thanks for your script! I manage to decrypt my text files successfully. If I try to decrypt my pictures or pdf files, the decrypted files are corrupted.

I compared the original and the decrypted file and I found little differences with a hex editor. The first line, the header of the file, seems not be decrypted or recovered correctly. Furthermore, little differences appear randomly in some lines. My key are correct because text files are decrypted correctly. Why does pdfs, images, etc... behave differently?

Do you have any advice? I desperately try to recover my encrypted files.

[mpellegrin]

@MarkusPoparkus Hello. Actually I ran into issues with binary files, but I finally recovered my files with a little change in the script :

<?php

// Replace these with your custom values
$datadir = '_PATH_TO_DATADIR_INCLUDING_TRAILING_SLASH_';
$filepath = 'some/directory/relative/to/datadir/myfile.txt';
$username = '_OWNCLOUD_LOGIN_';
$password = '_OWNCLOUD_PASSWORD_';

require_once '_PATH_TO_OWNCLOUD_/apps/files_encryption/lib/crypt.php';

// first get users private key and decrypt it
$encryptedUserKey = file_get_contents($datadir . $username . '/files_encryption/' . $username . '.privateKey');
$decryptedUserKey = OCA\Files_Encryption\Crypt::decryptPrivateKey($encryptedUserKey, $password);

// now we need to decrypt the file-key, therefore we use our private key and the share key
$shareKey = file_get_contents($datadir . $username . '/files_encryption/keys/' . $filepath . '/' . $username . '.shareKey');
$encryptedKeyfile = file_get_contents($datadir . $username . '/files_encryption/keys/' . $filepath . '/fileKey');
$decryptedKeyfile = OCA\Files_Encryption\Crypt::multiKeyDecrypt($encryptedKeyfile, $shareKey, $decryptedUserKey);

// finally we can use the decrypted file-key to decrypt the file
// but first, strip header block
$handle = fopen($datadir . $username . '/files/' . $filepath, 'r');
$encryptedContent = '';
$decryptedContent = '';

$data = fread($handle, OCA\Files_Encryption\Crypt::BLOCKSIZE);
// if this block contained the header, we can continue
if (OCA\Files_Encryption\Crypt::isHeader($data)) {
        $header = OCA\Files_Encryption\Crypt::parseHeader($data);
        $cipher = OCA\Files_Encryption\Crypt::getCipher($header);
} else {
    die('Cannot find header');
}

while ($data = fread($handle, OCA\Files_Encryption\Crypt::BLOCKSIZE)) {
    // Decrypt the block
    $decryptedBlock = OCA\Files_Encryption\Crypt::symmetricDecryptFileContent($data, $decryptedKeyfile, $cipher);
    // At that point you can write the decrypted block to the decrypted file
    // instead of appending it to $decryptedContent. But I don't fear memory
    // limits and I choose to append the block to my string and write the
    //  file at the end
    $decryptedContent .= $decryptedBlock;
}
fclose($handle);

// Write the decrypted file
file_put_contents(str_replace('/', '-', $filepath), $decryptedContent);

The trick is that I don't decrypt the entire file with the function symmetricDecryptFileContent, I instead use it to decrypt the file block by block. I don't know why this works and the other doesn't.

[riemerio]

@mpellegrin Thanks a lot!!! All my files are successfully decrypted with the help of your script. I am really really happy that it worked :+1:

In order to decrypt all my files automatically, I modified your script by wrapping it into a filewalker. In case some people might want to do this as well, here is the script: (owncloud version 7.0.7)

<?php

// Replace these with your custom values
$datadir = 'e.g.: /home/ubuntu/owncloud/';
$pathToFiles = $datadir.'<OWNCLOUD_USERNAME_HERE>/files';
$username = '<OWNCLOUD_USERNAME_HERE>';
$password = '<OWNCLOUD_PASSWORD_HERE>';
$folderForRecoveredFiles = 'e.g.: /home/ubuntu/recovered/';

require_once '/var/www/owncloud707/apps/files_encryption/lib/crypt.php';

// first get users private key and decrypt it
$encryptedUserKey = file_get_contents($datadir . $username . '/files_encryption/' . $username . '.privateKey');
$decryptedUserKey = OCA\Encryption\Crypt::decryptPrivateKey($encryptedUserKey, $password);
$i=1;
$objects = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($pathToFiles), RecursiveIteratorIterator::SELF_FIRST);
foreach($objects as $name => $object){
        echo "$i $name\n";

        $filepath = str_replace($pathToFiles."/","",$name);

    // now we need to decrypt the file-key, therefore we use our private key and the share key
    if(!file_exists($datadir . $username . '/files_encryption/keys/' . $filepath . '/fileKey') || !file_exists($datadir . $username . '/files_encryption/keys/' . $filepath . '/' . $username . '.shareKey')){
        echo "\tskipped: ".$filepath."\n";
    } 
    else {
        $shareKey = file_get_contents($datadir . $username . '/files_encryption/keys/' . $filepath . '/' . $username . '.shareKey');
        $encryptedKeyfile = file_get_contents($datadir . $username . '/files_encryption/keys/' . $filepath . '/fileKey');
        $decryptedKeyfile = OCA\Encryption\Crypt::multiKeyDecrypt($encryptedKeyfile, $shareKey, $decryptedUserKey);

        // finally we can use the decrypted file-key to decrypt the file
        // but first, strip header block
        $handle = fopen($datadir . $username . '/files/' . $filepath, 'r');
        $encryptedContent = '';
        $decryptedContent = '';

        $data = fread($handle, OCA\Encryption\Crypt::BLOCKSIZE);
        // if this block contained the header, we can continue
        if (OCA\Encryption\Crypt::isHeader($data)) {
            $header = OCA\Encryption\Crypt::parseHeader($data);
            $cipher = OCA\Encryption\Crypt::getCipher($header);
        } else {
            die('Cannot find header');
        }

        while ($data = fread($handle, OCA\Encryption\Crypt::BLOCKSIZE)) {
            // Decrypt the block
            $decryptedBlock = OCA\Encryption\Crypt::symmetricDecryptFileContent($data, $decryptedKeyfile, $cipher);
            // A that point you can write the decrypted block to the decrypted file
            // instead of appending it to $decryptedContent. But I don't fear memory
            // limits and I choose to append the block to my string and write the
            //  file at the end
            $decryptedContent .= $decryptedBlock;
        }
        fclose($handle);

        $newDir = dirname($folderForRecoveredFiles.$filepath);  

        if (!is_dir($newDir)) {
          // dir doesn't exist, make it
          mkdir($newDir,0777,true);
        }
        // Write the decrypted file
        file_put_contents($folderForRecoveredFiles.$filepath, $decryptedContent);

        // limit iterations for debugging
        //if ($i>10)break;
        $i++;
    }
}
?>

It's a little bit hacky but it works :) I hope it will help someone :)

Cheers Markus

[schiessle]

With ownCloud 8.2 we will have a occ command line tool to disable encryption and to decrypt all files again:

./occ encryption:decrypt-all --help

Usage:
 encryption:decrypt-all [user]

Arguments:
 user                  user for which you want to decrypt all files (optional)

Options:
 --help (-h)           Display this help message
 --quiet (-q)          Do not output any message
 --verbose (-v|vv|vvv) Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
 --version (-V)        Display this application version
 --ansi                Force ANSI output
 --no-ansi             Disable ANSI output
 --no-interaction (-n) Do not ask any interactive question

Help:
This will disable server-side encryption and decrypt all files for all users if it is supported by your
encryption module. Please make sure that no user access his files during this process!

[schiessle]

I think this should solve the problem. If not feel free to re-open.

[smu]

Will this tool allow to decrypt files in locations different from the users data directory, for instance, from a backup?

[schiessle]

@smu no, it re-uses the already existing code in core to avoid that we have to maintain a second code path which does the same and always have to catch up with core, different encryption modules, etc.

[smu]

That makes sense. However, is there a way to recover an encrypted file from a backup? Can I simply copy it back into the users data folder and owncloud takes care of it (assuming the encryption keys are the same)?

[schiessle]

@smu it should work if you copy the files back, the encryption keys and make sure that encryption and the encryption module is enabled.

[arno01]

@smu or you can use my script that I've just published, tested with ownCloud v8.1.3 https://github.com/arno01/ocdec

[victorbw]

@schiesbn ive tried to put the files back from a backup (passwords never changed, lab-environment), and it didnt work on OC 9, and even better, OC disallows me to delete the files via web-browser. and yes, i did the occ scan command. any ideas?

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.