Core: "Encryption module defined in ... not loaded" & "Error while running background job: Module with id: OC_DEFAULT_MODULE does not exist"

[bcutter]

Steps to reproduce

1. Enable encryption on OC before v8
2. Update to OC >= 8
3. Disable encryption and decrypt all files
   

   Expected behaviour

OC log is free of errors

Actual behaviour

OC log contains several encryption related errors

Server configuration

Operating system:
Raspbian (Debian 8)
Web server:
Nginx
Database:
MySQL
PHP version:
5.6.14
ownCloud version: (see ownCloud admin page)
8.2.1
Updated from an older ownCloud or fresh install:
Updated
List of activated apps:

  - activity: 2.1.3
  - calendar: 0.8.1
  - contacts: 0.5.0.0
  - documents: 0.11.0
  - files: 1.2.0
  - files_external: 0.3.0
  - files_locking:
  - files_pdfviewer: 0.7
  - files_share_qr: 0.1
  - files_sharing: 0.7.0
  - files_texteditor: 2.0
  - files_trashbin: 0.7.0
  - files_versions: 1.1.0
  - files_videoviewer: 0.1.3
  - firstrunwizard: 1.1
  - gallery: 14.2.0
  - logreader: 1.1.2
  - notifications: 0.1.0
  - ocusagecharts: 3.4.5
  - ownbackup: 0.3.7
  - storagecharts2: 2.6.1
  - templateeditor: 0.1
  - updater: 0.6

The content of config/config.php:
Note:
"sudo -u www-data php occ config:list system --public" produces:

  [RuntimeException]
  The "--public" option does not exist.

I therefore manually removed some sensitive lines from the output of "...config:list system":

    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "datadirectory": "\/var\/www\/owncloud\/data",
    "dbtype": "mysql",
    "version": "8.2.1.4",
    "installed": true,
    "forcessl": true,
    "forceSSLforSubdomains": true,
    "trashbin_retention_obligation": "90, auto",
    "overwritewebroot": "\/owncloud",
    "overwrite.cli.url": "\/owncloud",
    "theme": "",
    "maintenance": false,
    "secret": "***REMOVED SENSITIVE VALUE***",
    "dbname": "owncloud",
    "dbhost": "127.0.0.1:3306",
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "memcache.local": "\\OC\\Memcache\\Redis",
    "memcache.locking": "\\OC\\Memcache\\Redis",
    "redis": {
        "host": "localhost",
        "port": 6379,
        "timeout": 0
    },
    "filelocking.enabled": "true",
    "activity_expire_days": 365,
    "default_language": "de",
    "default_app": "files",
    "knowledgebaseenabled": true,
    "enable_avatars": true,
    "allow_user_to_change_display_name": true,
    "remember_login_cookie_lifetime": 2592000,
    "session_lifetime": 86400,
    "session_keepalive": true,
    "updatechecker": true,
    "log_type": "owncloud",
    "loglevel": 3,
    "logtimezone": "Europe\/Berlin",
    "cron_log": true,
    "appstore.experimental.enabled": true,
    "preview_libreoffice_path": "\/usr\/bin\/libreoffice",
    "filesystem_check_changes": 0,
    "asset-pipeline.enabled": true

Are you using external storage, if yes which one: local/smb/sftp/...
smb, sftp, Dropbox
Are you using encryption: yes/no
No, not anymore
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
No

Logs

ownCloud log (data/owncloud.log)

{"reqId":"brbLuZO4dBWkKMweRKe1","remoteAddr":"","app":"no app in context","message":"Encryption module defined in \"files_versions\/*PATHandSomeFILENAME**PATHandSomeFILENAME*.ctv5.v1437123218\" not loaded!","level":3,"time":"2015-11-29T22:15:03+01:00"}
{"reqId":"brbLuZO4dBWkKMweRKe1","remoteAddr":"","app":"no app in context","message":"Error while running background job: Module with id: OC_DEFAULT_MODULE does not exist.","level":3,"time":"2015-11-29T22:15:03+01:00"}
{"reqId":"brbLuZO4dBWkKMweRKe1","remoteAddr":"","app":"no app in context","message":"Encryption module defined in \"files_versions\/*PATHandSomeFILENAME*.cus.v1437053228\" not loaded!","level":3,"time":"2015-11-29T22:15:03+01:00"}
{"reqId":"brbLuZO4dBWkKMweRKe1","remoteAddr":"","app":"no app in context","message":"Error while running background job: Module with id: OC_DEFAULT_MODULE does not exist.","level":3,"time":"2015-11-29T22:15:03+01:00"}
{"reqId":"wshk4wlqCaPB4TTZ4\/tz","remoteAddr":"","app":"no app in context","message":"Encryption module defined in \"files_versions\/*PATHandSomeFILENAME*.onetoc2.v1434453610\" not loaded!","level":3,"time":"2015-11-30T21:30:06+01:00"}
{"reqId":"wshk4wlqCaPB4TTZ4\/tz","remoteAddr":"","app":"no app in context","message":"Error while running background job: Module with id: OC_DEFAULT_MODULE does not exist.","level":3,"time":"2015-11-30T21:30:06+01:00"}
{"reqId":"wshk4wlqCaPB4TTZ4\/tz","remoteAddr":"","app":"no app in context","message":"Encryption module defined in \"files_versions\/*PATHandSomeFILENAME*.xlsx.v1436465371\" not loaded!","level":3,"time":"2015-11-30T21:30:06+01:00"}
{"reqId":"wshk4wlqCaPB4TTZ4\/tz","remoteAddr":"","app":"no app in context","message":"Error while running background job: Module with id: OC_DEFAULT_MODULE does not exist.","level":3,"time":"2015-11-30T21:30:06+01:00"}
{"reqId":"XhwRuYfgwhQI0zUC+b18","remoteAddr":"","app":"no app in context","message":"Encryption module defined in \"files_versions\/*PATHandSomeFILENAME*.txt.v1437059231\" not loaded!","level":3,"time":"2015-12-01T10:30:03+01:00"}
{"reqId":"XhwRuYfgwhQI0zUC+b18","remoteAddr":"","app":"no app in context","message":"Error while running background job: Module with id: OC_DEFAULT_MODULE does not exist.","level":3,"time":"2015-12-01T10:30:03+01:00"}
{"reqId":"XhwRuYfgwhQI0zUC+b18","remoteAddr":"","app":"no app in context","message":"Encryption module defined in \"files_versions\/*PATHandSomeFILENAME*.txt.v1436863916\" not loaded!","level":3,"time":"2015-12-01T10:30:04+01:00"}
{"reqId":"XhwRuYfgwhQI0zUC+b18","remoteAddr":"","app":"no app in context","message":"Error while running background job: Module with id: OC_DEFAULT_MODULE does not exist.","level":3,"time":"2015-12-01T10:30:04+01:00"}
{"reqId":"JbUFvDo5fq9BqYKt73uV","remoteAddr":"","app":"no app in context","message":"Encryption module defined in \"files_versions\/*PATHandSomeFILENAME*.onetoc2.v1434453610\" not loaded!","level":3,"time":"2015-12-03T18:30:04+01:00"}
{"reqId":"JbUFvDo5fq9BqYKt73uV","remoteAddr":"","app":"no app in context","message":"Error while running background job: Module with id: OC_DEFAULT_MODULE does not exist.","level":3,"time":"2015-12-03T18:30:04+01:00"}

• I removed encryption on 2015-07-20 on the at this time up2date OC version (8.1.?).
• There is no encryption since this time. Everything worked fine, but the log is full of those encryption-related errors.
• Are those from the database? Can I perform a "encryption rest cleanup"?
• Did the migration of encryption-keys not fully work?
• It feels like something is still thinking, encryption should be enabled.
• Note: I´m pretty sure all the related files were existing BEFORE the encryption has been disabled.
• Further information:

sudo -u www-data php /var/www/owncloud/occ encryption:list-modules
=> Output: EMPTY (nothing)

sudo -u www-data php /var/www/owncloud/occ encryption:status
=> Output:
  - enabled: false
  - defaultModule:

Any ideas?

[karlitschek]

@schiesbn

[PVince81]

@schiesbn does the encryption module need to still be enabled when running the decrypt code ?

[schiessle]

@bcutter you shouldn't disable the encryption manually before you run the decrypt all command. Of course the encryption module is needed to decrypt your files

[bcutter]

As mentioned the decryption was done in July. I don't know if there already was an encryption module; but anyway: I didn't disable it.

So what next?

Am 08.12.2015 um 17:00 schrieb Björn Schießle [email protected]:

@bcutter you shouldn't disable the encryption module manually before you run the decrypt all command. Of course the encryption module is needed to decrypt your files

—
Reply to this email directly or view it on GitHub.

[schiessle]

As mentioned the decryption was done in July.

Which ownCloud version did you used back then? Probably ownCloud 8.0 or earlier because ownCloud 8.1 didn't had a decrypt all function. But if you decrypted your files in July I don't understand why you want to run the occ decrypt all tool now again if encryption is disabled since then?

[schiessle]

Sorry, ignore my last comment. Probably you used ownCloud 8.0 or earlier because ownCloud 8.1 didn't had a decrypt all function.
The error message probably appear because you have still some encrypted versions. You could enable the encryption module but not server side encryption in the admin settings. This way ownCloud could still read the encrypted files. But for this you would also need to run the migration because we changed the key location between 8.0 and 8.1. Or you just clean up your old versions.

[bcutter]

Because encryption is disabled for the physical files but NOT for the recent versions of those files. At least that's what those error messages are telling me.

More specific: The logs are full of the posted errors, I guess when the cron is running. They are usually related to files in Module/App "files_versions", example:

01.07.2015 FileA.txt (encrypted)
20.07.2015 DECRYPTION
10.12.2015 FileA.txt (unencrypted)
10.12.2015 FileA.txt.v1437059231

My problem is NOT FileA.txt today (line 3, which is unencrypted), it's one or more recent version of FileA.txt (line 4). I guess the "v14..." at the end is a UNIX time stamp indicating the snapshot time of the file.

So what I probably could do:
A) Delete all recent versions of all files (physically and in database).
B) Disable app "files_versions" which maybe would end in the same result like A.

BUT what I want to know is: How can I fix this problem and get my error logs cleared of those flooding messages WITHOUT doing A) or B)?

For you developers the much more interesting point probably is: What happens if I try to restore one of those affected "still encrypted" versions of FileA.txt? I guess it would run into the same issue like the cron does.

Maybe a workaround would be to simply adjust all affected entries in the database, so basically telling the database THERE IS no encryption module for those files (anymore). Something like a cleanup.

Do you need any other information which could help you to help me?

Am 10.12.2015 um 17:25 schrieb Björn Schießle [email protected]:

As mentioned the decryption was done in July.

You mentioned July before, but anyway. Which ownCloud version did you used back then? Probably ownCloud 8.0 or earlier because ownCloud 8.1 didn't had a decrypt all function. But if you decrypted your files in July I don't understand why you want to run the occ decrypt all tool now again if encryption is disabled since then?

—
Reply to this email directly or view it on GitHub.

[bcutter]

Hi, saw your msg. after my heavy detailed post. We're thinking to the same direction.

HOW can I cleanup my old versions? <<

Would be the easiest; additionally I don't understand the other encryption actions you mentioned and to be honest: with all my bad experiences with OC and encryption (which was back in 8.0, probably now it's more handsome) I don't even want to think about enabling something encryption related.

Am 10.12.2015 um 17:32 schrieb Björn Schießle [email protected]:

Sorry, ignore my last comment. Probably you used ownCloud 8.0 or earlier because ownCloud 8.1 didn't had a decrypt all function.
The error message probably appear because you have still some encrypted versions. You could enable the encryption module but not server side encryption in the admin settings. This was ownCloud could still read the encrypted files. But for this you would also need to run the migration because we changed the key location between 8.0 and 8.2. Or you just clean up your old versions.

—
Reply to this email directly or view it on GitHub.

[schiessle]

For you developers the much more interesting point probably is: What happens if I try to restore one of those affected "still encrypted" versions of FileA.txt? I guess it would run into the same issue like the cron does.

Yes, that's why with encryption 2.0 and beginning with 8.2 (where we introduced the decrypt all tool) what you typically do if you don't want to use encryption any longer:

• run the decrypt all script
• this will decrypt all "real files" but no versions or files in the trash bin
• disable server side encryption in the admin settings but keep the default encryption module app enabled
• this way ownCloud can still read the old encrypted versions because the encryption module is still there but it will not encrypt any new/modified files

(But even in this case it is really something you should only do if you know exactly what you do, generally encryption is a fundamental decision you take at the beginning and shouldn't change back and forth on a existing system)

In your situation you have now two options.

1. Delete all old versions. I would do this if you don't really need them because that's the easiest solution.
2. Alternatively you could restore the backup of your encryption keys (during decrypt all ownCloud should have created a backup of your keys). Enable server side encryption and the default encryption module, run "occ encryption:migrate" because the key location changed between 8.0 and 8.1. Now call "occ encryption:decrypt-all [username]". But as said, this is are quite complex steps because we don't know exactly which version you used back then, if the backup of the keys from July is still complete, etc. So if you don't really need the versions I would go with option 1.

[bcutter]

I chose option #1 (neither the blue, nor the red pill ^^) and used https://github.com/owncloud/core/pull/16964 (and additionally https://github.com/owncloud/core/pull/17140, not related to this issue but just for regaining some disk space). Worked perfectly. I will monitor the logs the next days, but I´m pretty sure the errors should be eliminated. Issue can be closed.

[bcutter]

...and finally it seems there´s still a piece of encryption remaining.

What I did today (without any relation to this issue):
I changed logging-setting in config.php from 3 (error) to 2 (warning, default).

What I got immediately after that (maybe the cron ran or every time the admin page is loaded) in oc-log:

Warning no app in context   Installation is in transit between the old Encryption (ownCloud <= 8.0) and the new encryption. Please enable the "Default encryption module" and run 'occ encryption:migrate'  8 minutes ago

Any ideas how to fix this (finally)? Maybe related to https://github.com/owncloud/core/issues/17450 ? Check in table "oc_appconfig" gives:

    files_encryption    enabled                 no
    files_encryption    installed_version       0.7.1
    files_encryption    ocsid                   166047
    files_encryption    publicShareKeyId        pubShare_XXXXXXXX
    files_encryption    recoveryAdminEnabled    1
    files_encryption    recoveryKeyId           recovery_XXXXXXXX
    files_encryption    types                   filesystem

Would be great to get the final "encryption killer information" here. Because I´m planning to re-enable encryption in the near future, I want to start on a fresh ground.

[bcutter]

Fixed it with https://github.com/owncloud/core/issues/17450#issuecomment-119694971

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.