Choco: Setting default for checksums to on with HTTP has caused regression for choco users who are relying on existing packages

[adcreare]

What You Are Seeing:

Setting default for checksums to on with HTTP has caused regression for choco users who are relying on existing packages

The original feature request where this was turned on: https://github.com/chocolatey/choco/issues/112

Issue:

This isn't a code level issue, obviously the choco client is functioning as expected.

However, for those of us using choco for automated build of servers this has caused every single build I have to break.
Unfortunately we have a dependency on vcredist2010 and that package currently its http without a checksum.

Ideally I think it might have been better if we contacted the affected packages so they had a chance to update their configuration to comply with the new requirement.

---

I do understand the need for turning this on and I completely support it. That said it would be nice if this was rolled out in a way that didn't break existing high use packages.

[marcinbojko]

At least allow us to change this option on puppet-chocolatey class level.

[ferventcoder]

@adcreare absolutely that was the original plan, a smooth migration. Unfortunately there was recently an incident with FossHub where we had packages downloading binaries from that site over HTTP with no checksum protection (we had several more packages also using downloading from FossHub, but with checksum protection). We needed to move up our plan to set that requirement for the protection of the community at large.

Instead of smooth migration as originally planned, it breaks some packages by default.
We talk about this in more detail in the CHANGELOG - https://github.com/chocolatey/choco/blob/master/CHANGELOG.md#0100-august-11-2016

[ferventcoder]

Typically you have workarounds here and choices. Chocolatey is going to protect you by default, but you have the choice to override that. You can do three things:

• Do not upgrade to 0.10.0 yet. Stay on an older version of choco while the community works to adjust their packages.
• pass the option --allow-empty-checksums to the install/upgrade
• set the feature switch to allow empty checksums.

If you are using Chocolatey for production purposes and using the community feed that is run by volunteers (and is subject to copyright law/distribution rights), you should consider the trust and control issues with that scenario - you've likely seen the disclaimer stating as much (it's linked right from the packages page) - https://chocolatey.org/docs/community-packages-disclaimer. Organizations should look to their own internal repos if they have a low tolerance for breakages, plus they are not subject to copyright law as they are not public (so you can embed binaries in the packages and/or point to internal shares, etc).

[ferventcoder]

@marcinbojko I'll follow up elsewhere - in short it's already available.

[adcreare]

@ferventcoder thanks for the reply.
Totally understand the need for this, its a good idea. That said, until you turn it on for https its not going to completely fix issues like the FossHub problem.

I've logged a pull request for the package that was breaking, however that's only for https support not checksums, I might need to modify it to include that too.

Would there by anyway we can get a list of the packages that don't have this support? Might be able to get a few more pull requests done to speed up transition.

Also aware of the workarounds, but thanks for that :) . In my situation using an old client isn't an option. We use choco as part of our system build process (also aware of the community disclaimer) and part of that process installs a new choco agent. For now I've added the --allow-empty-checksums but that's short term and kinda defeats the purpose.

Thanks again for the reply.

[ferventcoder]

You can set the choco version in an environment variable before you use the install scripts. See https://chocolatey.org/install#before-you-install

[ferventcoder]

You can turn it on for HTTPS as well in 0.10.0. We defaulted it for non-secure also because HTTP is much easier to hack. #895 is for the secure checksum requirement (which you've likely seen).

[ferventcoder]

For now I've added the --allow-empty-checksums but that's short term and kinda defeats the purpose.

Agreed, it is short term. I'm not sure it defeats the purpose - you are now aware this package is using HTTP and could have security issues and you are making the choice to allow that, where before you did not have that awareness.

[anuriq]

such events are the reason why we won't have nice automation in Windows.

[ferventcoder]

such events are the reason why we won't have nice automation in Windows.

Apologies. I did read what you wrote in your commit message and I wanted to clear up one misunderstanding. Chocolatey runs its own internal host (and only prompts when it detects that host is running). If you read the message that is written out, you will also notice that the prompt times out after 30 seconds. In other words, the packages are failing, Chocolatey still works in automated deployment scenarios without blocking where normal PowerShell hosts would fail.

[ferventcoder]

Here's a visual -

Note the 30 seconds message.

[anuriq]

@ferventcoder no offence was intended.
chef fails because of requested user input. and fails fast.

       ---- Begin output of "C:\ProgramData\chocolatey/bin/choco" install -y  vim ----
       STDOUT: Chocolatey v0.10.0
       Installing the following packages:
       vim
       By installing you accept licenses for the packages.

       vim v7.4.2066 [Approved]
       Downloading vim
         from 'http://sourceforge.net/projects/cream/files/Vim/7.4.2066/gvim-7-4-2066.exe/download'
Progress: 100% - Completed download of C:\Users\vagrant\AppData\Local\Temp\chocolatey\vim\7.4.2066\gvim-7-4-2066.exe (8.68 MB).
       Download of gvim-7-4-2066.exe (8.68 MB) completed.
       WARNING: Missing package checksums are not allowed (by default for HTTP/FTP,
        HTTPS when feature 'allowEmptyChecksumsSecure' is disabled) for
        safety and security reasons. Although we strongly advise against it,
        if you need this functionality, please set the feature
        'allowEmptyChecksums' ('choco feature enable -n
        allowEmptyChecksums')
        or pass in the option '--allow-empty-checksums'.
       The integrity of the file 'gvim-7-4-2066.exe' from 'http://sourceforge.net/projects/cream/files/Vim/7.4.2066/gvim-7-4-2066.exe/download' has not been verified by a checksum in the package scripts.
       Do you wish to allow the install to continue (not recommended)?
       [Y] Yes [N] No (default is "N")
         Confirmation (`-y`) is set.
         Respond within 30 seconds or the default selection will be chosen.
       WARNING: Write-ChocolateyFailure is deprecated and will be removed in v2. If you are the package maintainer, please use 'throw $_.Exception' instead.
       ERROR: Empty checksums are no longer allowed by default for non-secure sources. Please ask the maintainer to add checksums to this package. In the meantime if you need this package to work correctly, please enable the feature allowEmptyChecksums or provide the runtime switch '--allowEmptyChecksums'. We strongly advise against allowing empty checksums for HTTP/FTP sources.
       The install of vim was NOT successful.
       Error while running 'C:\ProgramData\chocolatey\lib\vim\tools\chocolateyInstall.ps1'.
        See log for details.

       Chocolatey installed 0/1 packages. 1 packages failed.
        See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

       Failures
        - vim (exited -1) - Error while running 'C:\ProgramData\chocolatey\lib\vim\tools\chocolateyInstall.ps1'.
        See log for details.

       Check out Pro / Business for more features! https://chocolatey.org/compare
       STDERR:
       ---- End output of "C:\ProgramData\chocolatey/bin/choco" install -y  vim ----
       Ran "C:\ProgramData\chocolatey/bin/choco" install -y  vim returned -1

[ferventcoder]

@anuriq No worries. I laughed when I read the "can't have nice things" comment. :D

It's a frustrating situation for everyone. Thank goodness it is temporary.

[ferventcoder]

I added all of the different options you have when you run into this issue here: https://github.com/chocolatey/choco/issues/112#issue-58511395

The first option is consumer-based. Did you know you could pass the checksums yourself?