Server: CRC error with zip files downloaded from web UI

[kiv57]

Steps to reproduce

1. login to web UI
2. select a folder or multiple files, clic "Download"
3. unzip the downloaded zip with your desktop zip handler

Expected behaviour

Unzipped files should have correct CRC and be valid

Actual behaviour

• Errors about CRC and headers (message details depending on the unzip program used)
• Some files are invalid and can't be opened, depending on the file type

Server configuration

Operating system: Debian 8.6

Web server: Nginx 1.6.2

Database: MySQL 5.5.50-0+deb8u1

PHP version: 5.6.24

Nextcloud version: 10.0.1

Updated from an older Nextcloud/ownCloud or fresh install: OC 8.2.7 (debian repos) -> manual upgrade to NC 9.0.54 -> upgrade to NC 10.0.1

Where did you install Nextcloud from: https://download.nextcloud.com/server/releases/

Signing status: No errors have been found.

List of activated apps:

App list

Enabled:
  - activity: 2.3.2
  - admin_audit: 1.0.0
  - calendar: 1.4.1
  - comments: 1.0.0
  - contacts: 1.4.0.0
  - dav: 1.0.1
  - federatedfilesharing: 1.0.1
  - federation: 1.0.1
  - files: 1.5.2
  - files_mv: 0.8.2
  - files_pdfviewer: 0.8.1
  - files_sharing: 1.0.0
  - files_texteditor: 2.1
  - files_trashbin: 1.0.0
  - files_versions: 1.3.0
  - files_videoplayer: 0.9.8
  - firstrunwizard: 1.1
  - gallery: 15.0.0
  - notes: true
  - notifications: 0.3.0
  - password_policy: 1.0.0
  - passwords: 20.1
  - provisioning_api: 1.0.0
  - qownnotesapi: true
  - serverinfo: 1.1.1
  - survey_client: 0.1.5
  - systemtags: 1.0.2
  - tasks: true
  - templateeditor: 0.1
  - theming: 1.0.1
  - updatenotification: 1.0.1
  - workflowengine: 1.0.1
Disabled:
  - encryption
  - external
  - files_accesscontrol
  - files_automatedtagging
  - files_external
  - files_retention
  - user_external
  - user_ldap
  - user_saml

The content of config/config.php:

Config report

{
    "system": {
        "instanceid": "xxxxxxxxxxxx",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud.domain.tld",
            "local.ip.behind.reverse.proxy"
        ],
        "datadirectory": "\/home\/www-data\/data",
        "overwrite.cli.url": "https:\/\/cloud.domain.tld",
        "default_language": "fr",
        "dbtype": "mysql",
        "version": "9.1.1.5",
        "dbname": "owncloud",
        "dbhost": "localhost",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "mail_from_address": "me",
        "mail_smtpmode": "smtp",
        "mail_domain": "domain.tld",
        "loglevel": 2,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "theme": "",
        "maintenance": false,
        "remember_login_cookie_lifetime": 1296000,
        "session_lifetime": 86400,
        "session_keepalive": true,
        "updatechecker": false,
        "trashbin_retention_obligation": "auto",
        "mail_smtpsecure": "ssl",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtphost": "mail.xxx.yyy",
        "mail_smtpport": "465",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "appstore.experimental.enabled": true
    }
}

Are you using external storage, if yes which one: no

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: firefox 50.0

Operating system: Windows 7

Logs

Web server error log

Web server error log (dates don't fit with zip download dates)

2016/11/24 07:37:31 [error] 853#0: *7379 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Uncaught exception 'Exception' with message 'Session has been closed - no further changes to the session are allowed' in /var/www/owncloud/lib/private/Session/Internal.php:154
Stack trace:
#0 /var/www/owncloud/lib/private/Session/Internal.php(64): OC\Session\Internal->validateSession()
#1 /var/www/owncloud/lib/private/Session/CryptoSessionData.php(164): OC\Session\Internal->set('encrypted_sessi...', '43893e78d74e0a2...')
#2 /var/www/owncloud/lib/private/Session/CryptoSessionData.php(67): OC\Session\CryptoSessionData->close()
#3 [internal function]: OC\Session\CryptoSessionData->__destruct()
#4 {main}
  thrown in /var/www/owncloud/lib/private/Session/Internal.php on line 154" while reading upstream, client: 10.0.3.1, server: cloud.domain.tld, request: "GET /index.php/login HTTP/1.0", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "cloud.domain.tld"

2016/11/27 13:26:43 [error] 850#0: *46841 open() "/var/www/owncloud/favicon.ico" failed (2: No such file or directory), client: 10.0.3.1, server: cloud.domain.tld, request: "GET /favicon.ico HTTP/1.0", host: "cloud.domain.tld"

Nextcloud log (data/nextcloud.log)

Nextcloud log (just one repeated error I didn't investigate yet)

Error   core    Could not get application: cURL error 60: SSL certificate problem: unable to get local issuer certificate   2016-11-23T19:00:08+00:00

Browser log

Not sure it's useful in this case, if so please ask.

[nickvergessen]

Any information which files are affected? Because it seems to work just fine here with images and documents.

[kiv57]

The error messages always appear, with any file type :

• "Erreur en-têtes" with 7zip ("headers error" should be the translation ?)
• "There is a CRC mistake on the file (...) Continue ?" with power archiver (translated from french to).I have to clic "Yes" for each file in the zip.
  (test with the same .zip of course)

That said, some files are corrupted and can't be opened, some others seem valid and I can open them. The fun part is that it depends on the program which did the unzip :

• pdf and docx : valid with 7zip, corrupted with power archiver
• xlsx : valid with both
• CREO files (CAD) : always corrupted
  I must admit I didn't test every possible file type...
  Moreover, users of the download feature don't have always the choice of the unzip program.

Please feel free to ask for more tests if necessary.

[nickvergessen]

Do zips that only have english characters work?

[kiv57]

No : I already tested files without accents in file name, the problem persists.

[nerdissimo-de]

I can confirm the error. When trying to upload to Gmail it doesn't allow the upload because it thinks there is a virus (there definitely isn't). When trying to open the file with 7zip or Nemo Fileroller it's also not possible. Funny thing is: When I use the commandline unzip it works fine.

Edit: doesn't matter which filetypes are in the archives. Happens for images and documents here.

[JarbyDev]

I have the same problem Here using the microsoft default unzip software

and using 7zip, i was able to extracted the content, but after i tried to open the file i get the file is currupted. happen with pdf, excel, image files

[qwertygc]

I have the same problem with NC 11.

[jospoortvliet]

Same issue here, zip files are often not working in some unzip software but OK in others. On my desktop - Dolphin can't open them as folders but Ark is OK with 'em.

[Idlebawb]

I am having a similar Issue. I did not get any CRC Errors or completely unreadable files, but Headers Error on 7-zip. The Main issue with that is that there is a gateway / reverse proxy between the server and the clients and that doesn't let any of those zips through. So nobody can download folders or multiple files, which is a big problem.

See Errors in attached screenshots.

Server: SUSE Enterprise 12
NextCloud Version: tried 10.0.3 & 11.0.1
Client: Only tried on Windows 10 Pro x64 (1511) but I guess its not a Client OS issue.
Browser: tried FF 50.1, Chrome 56 & IE11 - no difference

What I did:
Select a Folder & Download it
Select multiple Files at once & Download

The Files were: a single line txt file and a new .xlsx with just random numbers in it.
No special characters in either folder or filenames.

I don't know how I would look at the headers in 7zip, the most information I can get out the zip file is from zipinfo -v, but the Version on SUSE is from 2009, so I don't know how helpful that will be. But here it is.

Output of zipinfo -v

Archive: /home/it-department/scripts/New folder.zip
There is no zipfile comment.

End-of-central-directory record:

Zip archive file size: 8451 (0000000000002103h)
Actual end-cent-dir record offset: 8353 (00000000000020A1h)
Expected end-cent-dir record offset: 8353 (00000000000020A1h)
(based on the length of the central directory and its expected offset)

This zipfile constitutes the sole disk of a single-part archive; its
central directory contains 3 entries.
The central directory is 328 (0000000000000148h) bytes long,
and its (expected) offset in bytes from the beginning of the zipfile
is 8025 (0000000000001F59h).

Central directory entry #1:

New folder/

offset of local header from start of archive: 0
(0000000000000000h) bytes
file system or operating system of origin: Unix
version of encoding software: 4.5
minimum file system compatibility required: MS-DOS, OS/2 or NT FAT
minimum software version required to extract: 4.5
compression method: none (stored)
file security status: not encrypted
extended local header: no
file last modified on (DOS date/time): 2017 Feb 15 08:50:18
32-bit CRC value (hex): 00000000
compressed size: 0 bytes
uncompressed size: 0 bytes
length of filename: 11 characters
length of extra field: 32 bytes
length of file comment: 0 characters
disk number on which file begins: disk 1
apparent file type: binary
Unix file attributes (040755 octal): drwxr-xr-x
MS-DOS file attributes (10 hex): dir

The central-directory extra field contains:

• A subfield with ID 0x0001 (PKWARE 64-bit sizes) and 28 data bytes. The first
  20 are: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00.

There is no file comment.

Central directory entry #2:

New folder/Microsoft Excel-Arbeitsblatt (neu).xlsx

offset of local header from start of archive: 73
(0000000000000049h) bytes
file system or operating system of origin: Unix
version of encoding software: 4.5
minimum file system compatibility required: MS-DOS, OS/2 or NT FAT
minimum software version required to extract: 4.5
compression method: none (stored)
file security status: not encrypted
extended local header: yes
file last modified on (DOS date/time): 2017 Feb 15 08:50:18
32-bit CRC value (hex): 163d861c
compressed size: 7694 bytes
uncompressed size: 7694 bytes
length of filename: 50 characters
length of extra field: 32 bytes
length of file comment: 0 characters
disk number on which file begins: disk 1
apparent file type: binary
Unix file attributes (100644 octal): -rw-r--r--
MS-DOS file attributes (00 hex): none

The central-directory extra field contains:

• A subfield with ID 0x0001 (PKWARE 64-bit sizes) and 28 data bytes. The first
  20 are: 0e 1e 00 00 00 00 00 00 0e 1e 00 00 00 00 00 00 49 00 00 00.

There is no file comment.

Central directory entry #3:

There are an extra 20 bytes preceding this file.

New folder/Neues Textdokument.txt

offset of local header from start of archive: 7899
(0000000000001EDBh) bytes
file system or operating system of origin: Unix
version of encoding software: 4.5
minimum file system compatibility required: MS-DOS, OS/2 or NT FAT
minimum software version required to extract: 4.5
compression method: none (stored)
file security status: not encrypted
extended local header: yes
file last modified on (DOS date/time): 2017 Feb 15 08:50:18
32-bit CRC value (hex): 4a17b156
compressed size: 11 bytes
uncompressed size: 11 bytes
length of filename: 33 characters
length of extra field: 32 bytes
length of file comment: 0 characters
disk number on which file begins: disk 1
apparent file type: binary
Unix file attributes (100644 octal): -rw-r--r--
MS-DOS file attributes (00 hex): none

The central-directory extra field contains:

• A subfield with ID 0x0001 (PKWARE 64-bit sizes) and 28 data bytes. The first
  20 are: 0b 00 00 00 00 00 00 00 0b 00 00 00 00 00 00 00 db 1e 00 00.

There is no file comment.

Edit: Just some additional info.
I tried to create some 64bit Zips of the same file with 7zip and dotnetzip.
7zip wont let me create any 64bit zips unless the source files are bigger than 4GB
dotnetzip with -64 option creates the same "Header Error", but always useses deflate, couldnt get it to "Store"
32Bit Zips were fine.

EDIT2:
Disabled zip64 through a change in Streamer.php line 45

$this->streamerInstance = new ZipStreamer(['zip64' => PHP_INT_SIZE !== 4]);

I just set straight up false for zip64. I guess I will get errors on bigger files, but I am willing to trade not working at all (for me) for possible occasional errors. Hope this will get a real fix in a future release.

[andyboeh]

Same issue for me (NC 11.0.2). I can also confirm that disabling Zip64 fixes the problem (I do not have any files larger than 4GB anyway) by setting
$this->streamerInstance = new ZipStreamer(['zip64' => false]);
in lib/private/Streamer.php

[McNetic]

Hi there. As some may already know, I'm the author of the ZipStreamer library responsible for creating zip files in OC and NC.
That said, I'm still using OC in production and can't seem to reproduce the problem in my installation. So, a few questions to those who can:
1) Is the problem reproducible with any file, or only with specific files? (if the latter, could you provide some of the problematic files?)
2) Are you running nextcloud on a 32 bit or a 64 bit machine?
3) Although the responsible code seems to still be the same: has anyone been able to reproduce the problem in Owncloud?

Thank you for your help.

[andyboeh]

Sure, thanks for the reply:

1. It seems to be easy to reproduce as it happens nearly every time I share something. I'll send you a problematic file directly.
2. I'm on a 64-bit Server, Apache 2.4, PHP 7.1.1, NextCloud 11.0.2

[kiv57]

Thank you for the reply.

1. Problem occurs on any file type, and any size, even without special characters
2. I run Nextcloud 10.0.3 on a 64 bit server under Debian 8.7, with PHP 5.6.24 and MySQL 5.5.50
3. I had this problem on every upgrade versions since my first installation, which was an ownCloud 8.*

Feel free to ask for more tests or info.

[nerdissimo-de]

Hey, thanks for the reply:

1. Happens every time with all filetypes I tested. Some apps are able to extract it, for example a simple "unzip" on the commandline in Archlinux (UnZip 6.00) works.
2. I run Nextcloud 11.0.2 on Ubuntu 16.04 LTS (64bit) with PHP 7.0.15 and MariaDB 10.0.29
3. I had the problem since installing Nextcloud (10.0.1 I think it was)

Best

[andyboeh]

For me, the issue seems to be a bit more complicated than I initially thought. I found the following cases:

• 64 bit ZIP that works on Windows (7z and Windows integrated ZIP), but fails on Linux (7z)
• 32 bit ZIP that fails on Windows (integrated ZIP, 7z reports it as correct), works on Linux (7z)

Unfortunately, I cannot share the failing 32 bit ZIP.

[Idlebawb]

I've been analyzing the 64bit Zips created by Nextcloud / Zipstreamer. And according to the Zip specification ( https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT ) they seem to be just fine.

The headers error I got with 7-Zip are likely incorrectly detected by 7-zip - see the post by the developer here:

https://sourceforge.net/p/sevenzip/discussion/45797/thread/cc8912f1/

The error i got referred to the file being a multi-part archive.
This was most likely caused by an incomplete or defect detection of zip64 in the gateway appliance we use.

I manually edited a test "broken" zip file created by nextcloud in a hex editor and replaced all the values in the "End of Central Directory Record" which are 0xFFFF with the corresponding values from the "Zip64 End of Central Record" and the zip went trough the gateway just fine.

So we opened a case with the manufaturer of our Web gateway appliance.

[KopfKrieg]

I'm running Nextcloud v11.0.3 via the official docker image behind an nginx reverse proxy (running on Ubuntu 16.04 LTS, all updates installed) and stumbled across the same issue.

I can't open any of the downloaded zip files (with and without special characters), but I'm able to unzip them via commandline. Disabling zip64 as suggested works.

Would be nice to have a solution for this :)

[qwertygc]

Yes :

$this->streamerInstance = new ZipStreamer(['zip64' => false]); in lib/private/Streamer.php

It's work

[McNetic]

It would be very nice to know if the problem persists when the reverse proxy is removed from the setup...

[qwertygc]

I write a PR https://github.com/nextcloud/server/pull/5112

[McNetic]

We should first try to identify what is causing the problems in the first place, before deliberately disabling features that appear to work for the vast majority of users (Zip64 is required for zip files and file entries bigger than 4 GB, and is standardized and documented since 2001).

I would love to fix any bug in ZipStreamer when it is found, but given that we already fixed a lot of the issues with the initial zip64 implementation, and given that we know various zip implementations out there do not correctly implement the standard (at least two are referenced in this thread alone), I suspect the problem(s) may lie in some of the other components involved.

So to narrow down the problem(s) and find it's cause, everyone having an issue should (1) make sure no third party software is involved. That means, no proxy, no deep packet inspecting and/or manipulating firewall, no antivirus or similar software. If the problem persists, you should exactly document which zip program in which version on which operating system exposed the problem.

[KopfKrieg]

I just created a sample folder with two sample files and downloaded it the following way:

• via proxy → gives an error if I try to open the zip file
• without proxy → still gives an error
• with modified Streamer.php via proxy → works fine

Nextcloud seems to generate the zip-files on-the-fly, everytime someone's downloading the files, using a non-deterministic algorithm (every zip file has another hashsum).

If you want to try it/inspect the generated zips, you can download them here (1,8kB all three files combined): [Link removed]

System is, as already said, Nextcloud 11.0.3 from nextcloud/docker, running on an up-to-date Ubuntu 16.04 LTS host with nginx 1.10.0 as proxy and let's encrypt for my certificates.

But maybe the zip files are fine (I can unzip them without problems via commandline [unzip-command]) and my archive manager (Nautilus archive manager, but I don't know what underlying library is used to unzip files) has a bug.

Does anybody know if the zip files can be verified? E.g. if they are compliant to the specification?

Update: $ 7z t Sample_via_Proxy.zip says:

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=de_DE.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz (306A9),ASM,AES-NI)

Scanning the drive for archives:
1 file, 709 bytes (1 KiB)

Testing archive: Sample_via_Proxy.zip

ERRORS:
Headers Error

--
Path = Sample_via_Proxy.zip
Type = zip
ERRORS:
Headers Error
Physical Size = 709
64-bit = +



Archives with Errors: 1

Open Errors: 1

Update: For me the issue persists with Nextcloud 12.0.0

[Bubu]

I can confirm the observations from @KopfKrieg. unzip works fine and tests the zip okay. p7zip gives the error posted above and can't extract the archive:

Type = zip
ERRORS:
Headers Error
Physical Size = 102989366
64-bit = +

Winrar (5,20, 2014) apparently also doesn't work. (Someone notified me that the downloads aren't working.)

Here are the program details:

UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;
see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.

Compiled with gcc 5.2.1 20151119 for Unix (Linux ELF).

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Core(TM) i5-6440HQ CPU @ 2.60GHz (506E3),ASM,AES-NI)

Using #5112 works around the problem.

[midluk]

I don't see a good reason (except to feel superior when showing that you support the latest specification and everybody else does not) to enable ZIP64 when it is not needed. It's a fact that there is software that does not work properly with those files and most people will assume nextcloud is broken and not that their unpacker is at fault. This just makes nextcloud look bad.
So I think ZIP64 should not be used unless there are too many files or too big files to fit in a normal zip file. Sum of uncompressed file sizes should be a good enough upper bound for resulting file size.

[McNetic]

My PR #5116 was created because I did not know the reasons for disabling Zip64 on 32Bit machines (meanwhile, I have to admit there are good reasons, though they are still not documented anywhere I could find).
Supporting Zip64 is not about feeling superior, but about being able to support including files bigger than 4GB and producing archives bigger than 4GB. It is absolutely up to the nextcloud maintainers to decide wether they want to enable support for this or not (or do it selectively, which might be a lot more complicated than you think - it could already take a lot of time to determine uncompressed file size if directories with many files are to be zipped).

As to supporting the latest specification, I can only reiterate: The Zip64 extension is documented and publicly available since at least 2001. That is 16 years! It is shame there are still major vendors of zip tools (and operating systems) who are not able to support this. This is not rocket science. It is correct parsing of a few file headers.

[nickvergessen]

Supporting Zip64 is not about feeling superior, but about being able to support including files bigger than 4GB and producing archives bigger than 4GB.

But you can't have them on 32bit OS anyway, right?

[McNetic]

Yes, you can. With LFS enabled, 32bit linux supports files up to 2 TB. X86 Windows also supports files bigger than 4GB (I suppose MacOS does, too). PHP file_size() would fail/produce wrong values on those, but is not used in ZipStreamer - the files are streamed, and the size is calculated internally (which is currently buggy for 32 bit systems).

Additionally, the file storage can be a different system than the nextcloud web server. And output is technically not a file, but a stream - it is up to the client system to handle those, so it would be possible to download a zip from a 32bit machine without LFS enabled that gets bigger than 4 GB.

[cpt1gl0]

Here is a temporary fix disabling zip-file creation and using tar only:

--- lib/private/Streamer.php.orig   2017-08-29 16:34:01.235111362 +0200
+++ lib/private/Streamer.php    2017-08-29 16:35:36.005122748 +0200
@@ -35,14 +35,7 @@
    private $streamerInstance;

    public function __construct(){
-       /** @var \OCP\IRequest */
-       $request = \OC::$server->getRequest();
-       
-       if ($request->isUserAgent($this->preferTarFor)) {
-           $this->streamerInstance = new TarStreamer();
-       } else {
-           $this->streamerInstance = new ZipStreamer(['zip64' => PHP_INT_SIZE !== 4]);
-       }
+       $this->streamerInstance = new TarStreamer();
    }

    /**

[tazinblack]

Same issue here with Nextcloud 12.0.0.

I think replacing zip with tar doesn't really help since a lot of windows users can't handle tar files.
I would prefer disabling the download via zip but unfortunately it looks like this option isn't available in the config.php any more.

I need to go productive with an installation next days and I think most users will name nextcloud to be broken since they never had issues with their zipper.
So would be nice if you could provide a solution even if this would mean to give your users the option to disable the zip download.

This would at least avoid the image damage and help our support team.
Also there would be enough time to find a solution in the meanwhile

[ITwrx]

"I think replacing zip with tar doesn't really help since a lot of windows users can't handle tar files."

[please imagine a wot/rant here]

eventually, i think the downloader should have the choice of the archive format instead of a config option that affects all users, but until this issue is fixed and that feature implemented, i would definitely vote for being able to choose tar (along with a "disable" option, i guess) in the config.

[es20490446e]

I can reproduce it using the KDE Ark archiver.

[smoebody]

i think this is related to https://github.com/owncloud/core/issues/25580

Frankly i think a file sharing application should produce a zip-file readable by most client-applications rather than using bleeding-edge features only recent client applications might be able to read.
It is only prevents from being used. Nobody will try to upgrade one's unzipper to open a zip-file somebody shared with one (at least this is what i experienced for maybe a year now since the issue exists).

[es20490446e]

I agree with the above statement, specially because I'm using a cutting edge distro (Manjaro) and even Ark in its latest version doesn't support it.

[nickvergessen]

So for now try to tempary copy the change from https://github.com/McNetic/PHPZipStreamer/pull/39/commits/3925705cc9632b217b6bf1d1a819716ec96bd258
to 3rdparty/mcnetic/zipstreamer/src/ZipStreamer.php (near line 528)
Find:

$cdRecCount = -1;

Replace with:

$cdRecCount = min(sizeof($this->cdRec), 0xffff);

[Samunosuke]

@nickvergessen Your change did solve the problem for me. It was reproduceable for any ZIP download I tried on my server.
At first I have tried the workaround to disable Zip64, but then I saw your fix.

[ghost]

Hi,

i also have the same problem with Nextcloud Version 12.0.4. Every ZIP Archive is damaged and you you are not able to extract it.

Will also try the temporary solution, but would be nice to have this in one of the next Versions of Nextcloud.

Many thanks to the developers......

/danyo

[Keridos]

Also reproducing this on 12.0.4.

[mathiasconradt]

When I mark multiple files for download via web UI:

• on Mac, I get a .tar file (why not a zip file?) that I can open without problems
• on Ubuntu, I get a .zip file instead, which can’t be opened / corrupt

both times using Opera, NC13 beta

[nickvergessen]

on Mac, I get a .tar file (why not a zip file?) that I can open without problems

Because it is defined to prefer tar: https://github.com/nextcloud/server/blob/7d806bdfbd54ecd4f8986b8b3f371f6074112a92/lib/private/Streamer.php#L31-L32
The reason is: https://github.com/owncloud/core/issues/17930
Maybe it needs to be adjusted, because mac evolved in the last two years.

on Ubuntu, I get a .zip file instead, which can’t be opened / corrupt

I guess that could be because of https://github.com/nextcloud/server/issues/2352#issuecomment-335150443

[mathiasconradt]

Using tar doesn't work well in the business world, though. tar files (downloaded by a Mac user) cannot be shared with Windows users easily/usefully. Just entirely looking at it from an end-user / UX perspective: the very same file set can be downloaded as zip on other EFSS without problems. Can't the zipstream library be replaced with another one? From my understanding it's not a problem with Mac or any other operating system (btw: even the zip on Ubuntu couldn't be opened) but with the php zip library used. Otherwise the issue would exist in any other EFSS as well.

[haraldkrischner]

For me this currently is the major issue when using nextcloud. Because this also affects people whom I want to share with not caring about the correct support of zip standards. If @nickvergessen 's fix fixes it release a fix? Perhaps?

[nickvergessen]

I will prepare a custom patch for our repository

[MorrisJobke]

Maybe it needs to be adjusted, because mac evolved in the last two years.

I just tested it (together with nextcloud/3rdparty#74) but I still get this error:

[es20490446e]

I have checked that gzip is supported by modern Android Phones and iPhones, but it's problematic with not so old models.

So for the time being I would simply use zip universally, and change to gzip once 90% of mobile phones support it.

Also I believe Windows cannot be taken into account in this regard. Windows is a clunky OS, and when you get it you assume you have to install other applications to make it work.

A Windows system without compression support is just very incapable. The same as not having office suite or multimedia codecs, they are just the basics.

[Keridos]

Windows can open and create zip files...

[es20490446e]

But that's the only compressed format it can open. Even when rar, 7zip, and gzip are widely used.

[zguithues]

many people are restricted from installing outside software on their computers, so they are left with only zip support.

I'm an insurance agent, i commonly have to send archives to my underwriter. He is not able/allowed to open .rar files, or any other alternative archive. I am required to send .zip files.

I don't like it, but we 100% DO have to take windows into account. it's a fact of life right now.

[es20490446e]

And the drawbacks of using ZIP are only superfluous.

[KopfKrieg]

Question: Why don't we let the user choose between zip and tar instead of assuming the user's preference based on his/her platform? (Maybe create a feature request?)

[es20490446e]

Because it doesn't matter:
(https://signalvnoise.com/archives2/it_just_doesnt_matter.php)

[KopfKrieg]

Are you serious? Of course it matters, that's why I'm asking.

[es20490446e]

We have said that it's somehow unpredictable if the target platform, the receiver of your files, will have support for gzip. So in practise having the option does nothing.

[nickvergessen]

While I see the point in "It just doesnt matter" and think its a valid argument in many cases,
this topic here is none of them.

Downloading files/archives is (apart from syncing files) the number two use case of Nextcloud. So that must work in as many situations as possible (by default). An option to download tar instead of zip (or vice versa) sounds like an option to me.

But fixing the broken zip download, which happens on some systems with some unzippers, should be the priority in first place.

[rullzer]

I just tested https://github.com/nextcloud/3rdparty/pull/74 and now at least p7zip can extract the archive. It seems that OSX has a different bug. Lets get that in and we can see how to fix OSX in the next step.

[arnowelzel]

Looks like there are still some issues, see #8798

[AntoineMazuyer]

I have the same issue with nexctloud 13.0.1 :(

[pjoscha]

still the same isse in nextcoud 15.0.4

[alogoc]

Looks like this was never fixed?

[es20490446e]

Do you need me to test?

[arnowelzel]

Would it help to change to another Zip-Library? Maybe https://github.com/maennchen/ZipStream-PHP works better.

[osscombat]

So this is a known problem and it's still open for almost 3 years in production versions?

$this->streamerInstance = new ZipStreamer(['zip64' => false]); is already merged in lib/private/Streamer.php in the latest 15 and 16 versions, but zip's are still being corrupt.

[kesselb]

~Mind to open a new issue if you still see this?~ https://github.com/nextcloud/server/issues/15871