Runtime: ZipArchive permissions on linux

[chris-mackubin]

ZipArchive doesn't currently allow manipulation permissions within. I ran into this when creating file in memory, dropping it into a zip and handing off to another system. With this code deployed on Linux, I'm not seeing a way to give the receiving system read access to contents of zip.

Example:

``` c#
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var lambdaFile = archive.CreateEntry("index.js");

        using (var entryStream = lambdaFile.Open())
        using (var streamWriter = new StreamWriter(entryStream))
        {
            streamWriter.Write(functionContent);
        }
    }

    memoryStream.Seek(0, SeekOrigin.Begin);
    svcFile.UploadFile(bucketName, s3Key, memoryStream);
}

```

[ianhays]

I'm not sure I follow. In your example you don't actually have a zip file, you have a stream that contains zipped data. Are you sending that data somewhere and constructing a new ZipArchive object from it?

Or are you saying that when unzipping a zip you want the files within to have different Unix permissions? We don't have support in CoreFX to modify traditional unix RWX permissions, though I added an issue to track that here.

It's not clear where the failure point in your example is.

[chris-mackubin]

Sorry, should have added more detail. In my example above, the upload method is sending the stream into another system (in this case AWS S3) which is saving the stream to disk in their system. By default thou, there is no read access on the ArchiveEntry, leading to permission denied when the other system attempts to access the contents of zip.

I should note: The service that is running the code above to create the ZipArchive and stream is running on Linux in a docker container.

• If I manually download the same zip file back out of S3 and access onto a windows machine I have access. But, if i download same said zip file down onto a linux box, the zip file has permissions, the entries inside have none.

-rw-rw-r-- 1 ec2-user ec2-user 1507 Jul 21 07:16 test.zip
---------- 1 ec2-user ec2-user 3788 Jul 20 15:32 index.js

It's not that I want to change the permissions at unzip. I want to set the permissions at creation or better yet within the ZipArchive. I shouldn't have to create a physical file to set them. Same as if I manually created a file, gave it permission and then added it to a Zip. The permissions of the file within would retain those permissions when unzipped. The permissions are part of the data being compressed.

[ianhays]

How are you unzipping on the target AWS machine? Are you using ZipArchive, ZipFile, or `the``unzip``` command?

The Zip format has a bit of a shaky relationship with unix file permissions. Technically Unix RWX isn't formally supported as a part of the spec, but some programs represent it through the extra flags in the file headers. Others will assume permissions based on the permissions of the zip itself when unzipping.

[chris-mackubin]

I'm not unzipping it, it's handed off to the external system (aws). Zip contains a lambda function, they are unzipping and pulling out contents. So can't say with certainty. Given they are Linux, I would assume unzip.

Interesting point, if the above example were run on a windows machine, the zip file and contents handed off is readable. If I pull the zip file down and opened it on a Linux box, I can see the zip file and content both have -rw-rw-r-- rights.

[ianhays]

Interesting point, if the above example were run on a windows machine, the zip file and contents handed off is readable

The only real difference in our ZipArchive/Entrys between Unix and Windows is that a ZipArchive created on Unix has the ZipVersionMadeByPlatform bit set to indicate it came from a Unix machine. I did some manual testing to repro your issue, and it seems the Unix unzip command defaults permissions for zips created on Windows but not for those created on Unix. I'm guessing that it expects extra flags to be set for zips created on Unix, and since our zips do not have those flags, permissions are set to 0 since that's what's in the extra flag field.

My suggested workaround would be to chmod when you unzip, but it sounds like you don't have control over that. The long term solution here would be to default the extra flag permission values when creating a zip on Unix and possibly provide some sort of control over them.

[ianhays]

The long term solution here would be to default the extra flag permission values

On second thought, this doesn't make a ton of sense. The values being excluded should already be treated as "default". Using the extra bits to retain unix permissions is a fairly new part of Zip, so unzip _should_ accommodate the case where they're left out in the same way that they accommodate zips created on a non-Unix platform.

and possibly provide some sort of control over them.

Personally I'd like to see everything in the header exposed eventually, but that's a big API pill to swallow particularly when you account for the huge number of undocumented 3rd party extra fields.

For now though, I don't think we should mess with adding permissions values for new entries unless they are specifically set by the programmer. There are too many instances where assuming permissions for new entries could very well be the wrong thing to do. It would be nice to retain existing file permissions for ZipFile.CreateFrom*, but as long as ZipFile is its own assembly that's going to require adding the API to ZipArchiveEntry first.

For some background on the formatting, this StackOverflow post does a good job describing the format of the external attr field that Unix uses to store its permissions.

[chris-mackubin]

I get you, just sucks. Basically in it's current form the functionality is limited when run on a non-windows server. For a work around, I wound up adding a condition for OSPlatform and on non-windows having to not utilize ZipArchive at all, but rather run zip command via system.diagnostics.process :disappointed:

I was however able to avoid also having to run chmod is similar fashion. Only because i found by looking through code base, that if File.Attributes is set to ReadOnly (which luckily was enough in my case :smiley: ) it was handled in corefx. Only reference i saw doing such...

[auspex]

I have a program pulling two archive files from a server (GeoServer, running in Linux), combining the two by writing the contents of each to a new stream, and downloading the new file to the user's browser:

        public IActionResult GetShapeFile(
            string filename,
            string cql_filter,
            string viewparams
        )
        {
            return new FileCallbackResult(new MediaTypeHeaderValue("application/zip"), async (outputStream, _) =>
            {
                using (var zipArchive = new ZipArchive(new WriteOnlyStreamWrapper(outputStream), ZipArchiveMode.Create))
                {
                    await copyZip(zipArchive, $"{_settings.EffortLayer}_Detail", viewparams);

                    if (cql_filter != null){
                        await copyZip(zipArchive, $"{_settings.SpeciesLayer}_Detail", viewparams, $"{cql_filter}");
                    }
                }
            })
            {
                FileDownloadName = $"{filename}"
            };
        }
        public async Task<int> copyZip (ZipArchive outZip, string layer, string viewparams, string taxon=""){
            // get the layer as a zip stream
            using (var stream = await _wfsService.GetLayer(layer, viewparams, taxon))
            {
                // open the incoming stream read-only
                using(var inZip = new ZipArchive(stream, ZipArchiveMode.Read))
                {
                    // copy each entry in the stream to 'outZip'
                    foreach (ZipArchiveEntry entry in inZip.Entries)
                    {
                        var name = entry.FullName;
                        var zipEntry = outZip.CreateEntry(name);
                        using (var outStream = zipEntry.Open())
                            using (var inStream = entry.Open())
                                await inStream.CopyToAsync(outStream);
                    }
                }
            }
            return 1;
        }

If the dotnet app is running on Linux, I get no read permissions. If it's running on Windows, I do.

Since it's a web app, I can hardly expect the user to know that the only way to use his downloaded Zip is to unzip it and chmod the resulting files.

Now, it's not a huge problem for me, as the dotnet server is going to be on Windows for the foreseeable future, but I shouldn't be tied there just because ZipArchive doesn't provide decent defaults. Especially since the _input_ archives are being created with the Unix permissions set correctly, so it _should_ be possible to create an archive entry that exactly matches one that's being read.

[daves0]

This is still an issue. I'm creating a .zip archive in a MemoryStream using ZipArchive that is emailed to someone, and when the recipient opens the .zip and saves the contained file to disk, that file has NO read/write access permissions and cannot be used unless they change the file permissions using their OS. Unlike Auspex above, I am not using a Windows server (I'm running on AWS Lambda) and this is a show-stopping bug for me.

From @ianhays comment above on Jul 22, 2016, I have to wonder if the ZipVersionMadeByPlatform bit should NOT be set to Unix, since in fact these .zip files do not conform to ones created on Unix .. these archives appear like Windows ones in that they should be handled like them by Unix unzip, per the comment "and it seems the Unix unzip command defaults permissions for zips created on Windows but not for those created on Unix."

Would that be an appropriate way to go forward with these, and get this issue resolved? Without some resolution the ZipArchive functionality just isn't an acceptable option at this point unfortunately.

[skaman]

I confirm that ZipArchive is not usable for create zip files on a linux server (for this problem with permissions). I had to use a third party library. I think it should generate readable zip files also on linux servers.

[marrrcin]

This is still a valid issue, I don't understand why it's closed.

[danmosemsft]

Reopening in case IO area triage would like to see this feedback. (I have no views)

[jgimness]

I just ran into this problem today.

There's also issue #1548 which looks related

[marcwittke]

entry.ExternalAttributes = entry.ExternalAttributes | (Convert.ToInt32("664", 8) << 16);

...assuming 664 permissions

[ptsneves]

I confirm @marcwittke solution works. Thanks a lot man. This was quite a deal breaker.
Before the ExternalAtributes change:

# unzip -Z /tmp/videos.zip 
Archive:  /tmp/videos.zip
Zip file size: 8860217 bytes, number of entries: 3
?---------  2.0 unx  1655297 b- stor 20-Jun-26 15:07 0-video.webm
?---------  2.0 unx  2738625 b- stor 20-Jun-16 18:42 4-video.webm
?---------  2.0 unx  4465751 b- stor 20-Jun-18 01:16 7-video.webm

After

# unzip -Z /tmp/videos.zip 
Archive:  /tmp/videos.zip
Zip file size: 8860217 bytes, number of entries: 3
?rw-rw-r--  2.0 unx  1655297 b- stor 20-Jun-26 15:07 0-video.webm
?rw-rw-r--  2.0 unx  2738625 b- stor 20-Jun-16 18:42 4-video.webm
?rw-rw-r--  2.0 unx  4465751 b- stor 20-Jun-18 01:16 7-video.webm

[v-yaremenko]

@marcwittke , @ptsneves , thanks for the provided solution.
However, I've faced that executable bit for user can't be set in a such way.
rwXrwxrwx <-- I mean the marked bit

My code:
fileEntry.ExternalAttributes |= (Convert.ToInt32("100777", 8) << 16);

Result (it is set to some default value):
-rw------- 2.0 fat 24 b defN 20-Sep-18 17:35 fileA.txt

But if I try to set 677, or some value without X bit for user, then it is okay:
fileEntry.ExternalAttributes |= (Convert.ToInt32("100677", 8) << 16);
-rw-rwxrwx 2.0 fat 19 b defN 20-Sep-18 17:35 fileB.txt

Would you please confirm that you are able to set the X bit for user?
If you have same issue, then it is a bug, as I understand.