Powershell: Version of Windows10 SDK downloaded from Chocolatey does not install on microsoft/windowsservercore:latest docker image

[raghushantha]

I am attempting to build PowerShell in microsoft/windowsservercore:latest docker image. The version of Win10SDK downloaded from chocolatey does not install. We need to ensure that Start-PSBootstrap works across all images.

@brycem has already reported this on Chocolatey

I am opening this issue to to make sure we update the package name (if required), when a new version is available on chocolatey.

Steps to reproduce

docker pull microsoft/windowsservercore
docker run -it microsoft/windowsservercore:latest

#git clone PowerShell/PowerShell to c:\PowerShell
PS C:\powershell> ipmo .\build.psm1
PS C:\powershell> Start-PSBootStrap -Package -Force

Expected behavior

bootstrapping succeeds

Actual behavior

Installing windows-sdk-10.0...
ERROR: Running ["C:\Users\ContainerAdministrator\AppData\Local\Temp\chocolatey\windows-sdk-10.0\10.0.26624\sdksetup.exe" /Quiet /NoRestart /Log "C:\Users\ContainerAdministrator\AppData\Local\Temp\chocolatey\windows-sdk-10.0.log" ] was not successful. Exit code was '-2147023293'. See log for possible error messages.
The install of windows-sdk-10.0 was NOT successful.
Error while running 'C:\ProgramData\chocolatey\lib\windows-sdk-10.0\tools\chocolateyInstall.ps1'.
 See log for details.

Chocolatey installed 0/1 packages. 1 packages failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
Failures
 - windows-sdk-10.0 (exited -2147023293) - Error while running 'C:\ProgramData\chocolatey\lib\windows-sdk-10.0\tools\chocolateyInstall.ps1'.(exited -2147023293) - Error while running 'C:\ProgramData\chocolatey\lib\windows-sdk-10.1\tools\ch
 See log for details.                                                                                                  o
Execution of { choco install windows-sdk-10.0 -y } failed with exit code -2147023293
At C:\powershell\build.psm1:2074 char:13
+             throw "Execution of {$sb} failed with exit code $LASTEXIT ...
+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Execution of { ...ode -2147023293:String) [], RuntimeException
    + FullyQualifiedErrorId : Execution of { choco install windows-sdk-10.0 -y } failed with exit code -2147023293

Environment data

PS C:\powershell> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.14393.206
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.206
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

[brycem]

I think we'd be well-served to switch to verifiable links to authoritative sources... Chocolatey is a pretty soft target to upload malicious scripts that pass virus scans to... Chocolatey is also not Nano-friendly.

[TSEO1]

Hi, raghushantha.
Are you slove problem?
I have a same problem. But I downloaded standalone Windows 10 SDK from Microsoft.com.

Here my Dockerfile:

FROM microsoft/windowsservercore:10.0.14393.321
LABEL Description="Windows Server Core WinSDK" Vendor="My" Version="0.1.0"
ADD distrib/StandaloneSDK.10.0.10240.0 /distrib
RUN C:\distrib\sdksetup.exe /features OptionId.WindowsSoftwareDevelopmentKit /q /norestart
RUN pushd "C:\ProgramData\Package Cache" && (rd /s /q "C:\ProgramData\Package Cache" 2>nul & popd)
RUN pushd "C:\distrib" && (rd /s /q "C:\distrib" 2>nul & popd)

P.S.
StandaloneSDK.10.0.10240.0 installed correctly on Windows Server Core 2016
StandaloneSDK.10.0.14393.33 not installed even on Windows Server Core 2016

[brycem]

Looks like the reason 10.0.10240.0 installs is because it doesn't contain the 'Windows IP Over USB-x86_en-us.msi' feature which is what is failing and rolling everything back... Where are you downloading it from?

[TSEO1]

Hello, brycem.
I previously downloaded SDKSETUP.EXE from Microsoft.com and saved SDK locally. But now I can't find a link for version 10.240.0.
Yes you are right about 'Windows IP Over USB-x86_en-us.msi' for version 10.0.14393.33.
And for version 10240.0 in docker container package "WindowsDesktopExtensionSDK_x86_en_us" fails with error:

MSI (s) (6C:EC) [14:38:41:770]: Executing op: ActionStart(Name=GenUnionWinMD,,)
MSI (s) (6C:EC) [14:38:41:770]: Executing op: CustomActionSchedule(Action=GenUnionWinMD,ActionType=3073,Source=BinaryData,Target=CAQuietExec,CustomActionData="C:\Windows\SysWOW64\cmd.exe" /C "C:\Program Files (x86)\Windows Kits\10\bin\x86\GenerateUnionWinMD.cmd")
MSI (s) (6C:24) [14:38:41:800]: Invoking remote custom action. DLL: C:\Windows\Installer\MSI2C5D.tmp, Entrypoint: CAQuietExec
MSI (s) (6C:E8) [14:38:41:800]: Generating random cookie.
MSI (s) (6C:E8) [14:38:41:808]: Created Custom Action Server with PID 880 (0x370).
MSI (s) (6C:F4) [14:38:41:874]: Running as a service.
MSI (s) (6C:F4) [14:38:41:879]: Hello, I'm your 32bit Elevated Non-remapped custom action server.
CAQuietExec:  Microsoft (R) Generate UnionWinMD Tool version 10.0.2
CAQuietExec:  Copyright (c) Microsoft Corporation
CAQuietExec:  All rights reserved.
CAQuietExec:  Error: You must run this script from an elevated command prompt.
CAQuietExec:  Error 0x80070002: Command line returned an error.
CAQuietExec:  Error 0x80070002: CAQuietExec Failed
CustomAction GenUnionWinMD returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)
MSI (s) (6C:EC) [14:38:42:385]: Note: 1: 2265 2:  3: -2147287035 

I deleted "package_WindowsDesktopExtensionSDK_x86_en_us" entry in "UserExperienceManifest.xml" and SDK was successfully installed without this package.

Also I read page https://social.msdn.microsoft.com/Forums/sqlserver/en-US/bfe645b4-2cec-4550-9b7d-c61bc9562bfd/docker-container-users-other-than-containeradministrator-in-tp5. But command "runas" has not worked in docker container.

[brycem]

I found the download (though my workaround for the desktop extension failure was less elegant :-) - https://raw.githubusercontent.com/brycem/PowerShell/docker/docker/nightly/windowsservercore/Dockerfile

All prerequisites are getting met for build, but I'm getting a build failure still, but we're getting close:

C:Program Files (x86)Microsoft Visual Studio 14.0VCbinamd64CL.exe /c /I"C:PowerShellsrcpowershell-nativenati
vemshpwrshcommon" /Zi /nologo /W3 /WX- /O1 /Ob2 /Oi /Oy- /GL /D WIN32 /D _WINDOWS /D NDEBUG /D CORECLR /D _AMD64_ /D _W
IN64 /D AMD64 /D BIT64=1 /D _TARGET_AMD64_=1 /D WIN32 /D _WIN32 /D WINVER=0x0602 /D _WIN32_WINNT=0x0602 /D WIN32_LEAN_AN
D_MEAN=1 /D _CRT_SECURE_NO_WARNINGS /D EnC_SUPPORTED /D UNICODE /D _UNICODE /D "CMAKE_INTDIR="Release"" /D _UNICODE /D
UNICODE /U_MT /GF /Gm- /EHa /MT /Zp8 /GS /Gy /fp:precise /Zc:wchar_t- /Zc:forScope /Zc:inline /GR- /Fo"pwrshcommon.dir
Release" /Fd"pwrshcommon.dirReleasepwrshcommon.pdb" /Gd /TP /FC /Zl /errorReport:queue /d2Zi+ /Zm200 /ZH:SHA_256 /g
uard:cf "C:PowerShellsrcpowershell-nativenativemshpwrshcommonpwrshcommon.cpp" "C:PowerShellsrcpowershell-native
nativemshpwrshcommonWinSystemCallFacade.cpp" "C:PowerShellsrcpowershell-nativenativemshpwrshcommonConfigFileRea
der.cpp"
pwrshcommon.cpp
c:powershellsrcpowershell-nativenativemshpwrshcommonnativemsh.h(14): fatal error C1083: Cannot open include file:
'stdio.h': No such file or directory [C:PowerShellsrcpowershell-nativenativemshpwrshcommonpwrshcommon.vcxproj]
WinSystemCallFacade.cpp
c:powershellsrcpowershell-nativenativemshpwrshcommonsystemcallfacade.h(14): fatal error C1083: Cannot open include
file: 'stdio.h': No such file or directory [C:PowerShellsrcpowershell-nativenativemshpwrshcommonpwrshcommon.vcxpr
oj]
ConfigFileReader.cpp
c:powershellsrcpowershell-nativenativemshpwrshcommonconfigfilereader.cpp(1): fatal error C1083: Cannot open includ
e file: 'iostream': No such file or directory [C:PowerShellsrcpowershell-nativenativemshpwrshcommonpwrshcommon.vcx
proj]
Done Building Project "C:PowerShellsrcpowershell-nativenativemshpwrshcommonpwrshcommon.vcxproj" (default targets)
-- FAILED.
Done Building Project "C:PowerShellsrcpowershell-nativenativemshpwrshexepowershell.vcxproj" (default targets) -- F
AILED.
Done Building Project "C:PowerShellsrcpowershell-nativeALL_BUILD.vcxproj" (default targets) -- FAILED.

Build FAILED.

"C:PowerShellsrcpowershell-nativeALL_BUILD.vcxproj" (default target) (1) ->
"C:PowerShellsrcpowershell-nativenativemshpwrshexepowershell.vcxproj" (default target) (3) ->
"C:PowerShellsrcpowershell-nativenativemshpwrshcommonpwrshcommon.vcxproj" (default target) (4) ->
(ClCompile target) ->
c:powershellsrcpowershell-nativenativemshpwrshcommonnativemsh.h(14): fatal error C1083: Cannot open include file
: 'stdio.h': No such file or directory [C:PowerShellsrcpowershell-nativenativemshpwrshcommonpwrshcommon.vcxproj]
c:powershellsrcpowershell-nativenativemshpwrshcommonsystemcallfacade.h(14): fatal error C1083: Cannot open inclu
de file: 'stdio.h': No such file or directory [C:PowerShellsrcpowershell-nativenativemshpwrshcommonpwrshcommon.vcx
proj]
c:powershellsrcpowershell-nativenativemshpwrshcommonconfigfilereader.cpp(1): fatal error C1083: Cannot open incl
ude file: 'iostream': No such file or directory [C:PowerShellsrcpowershell-nativenativemshpwrshcommonpwrshcommon.v
cxproj]

0 Warning(s)
3 Error(s)

Time Elapsed 00:00:05.93
Execution of { Invoke-Expression -Command:$command } failed with exit code 1
At C:PowerShellbuild.psm1:2091 char:13

• throw "Execution of {$sb} failed with exit code $LASTEXIT ...
• ~~~~~~~~~~~~~
  
  
  
  • CategoryInfo : OperationStopped: (Execution of { ...ith exit co
    
    de 1:String) [], RuntimeException
  
  
  • FullyQualifiedErrorId : Execution of { Invoke-Expression -Command:$comma
    
    nd } failed with exit code 1
  
  

[TSEO1]

Hello, brycem.
I think than enviroment variable "Include" not set.

To build project you can try:

1) Add path to "cl.exe" to enviroment variable "path".
May use ".ps1" script like this:

echo "Set enviroment vars for current session and current User ..."
$path = $env:path
If ($path.EndsWith(";")  -eq $False) {$path = [System.String]::Concat($path,";")} 
$path =  [System.String]::Concat($path,"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin")
$env:path = $path
[System.Environment]::SetEnvironmentVariable('path',$path,'User')
echo "Done."

2) Go to the root folder with your .cpp files
3) Create the ".bat" file like this:

echo "save current directory ..."
set curpath=%cd%
call "C:\Program Files (x86)\Microsoft Visual C++ Build Tools\vcbuildtools.bat" amd64
echo "change directory ..."
cd "%curpath%"
cl.exe simple_test.cpp

4) Run ".bat" file

[StefanScherer]

@brycem Thank you for your Dockerfile to install the Windows 10 SDK. I just solved building a NanoServerApiScan Docker image with it.

[brycem]

I've verified that I'm now getting all of the needed includes installed and that my build environment variables are getting properly populated, but I'm still seeing issues in my VC++ Build Tools environment... Here's the docker command to install the image:

docker build https://raw.githubusercontent.com/brycem/PowerShell/master/docker/nightly/windowsservercore/Dockerfile

It takes a few minutes to pull the baseimage (it's 16gb) the first time, but a lot less time than installing that whole mess from scratch :-) Here's the current output I'm getting from Start-PSBuild:

Build FAILED. "C:\PowerShell\src\powershell-native\ALL_BUILD.vcxproj" (default target) (1) -> "C:\PowerShell\src\powershell-native\nativemsh\pwrshexe\powershell.vcxproj" (default target) (3) -> "C:\PowerShell\src\powershell-native\nativemsh\pwrshcommon\pwrshcommon.vcxproj" (default target) (4) -> (ClCompile target) -> c:\powershell\src\powershell-native\nativemsh\pwrshcommon\pwrshcommon.cpp(12): fatal error C1083: Cannot open include file: 'corerror.h': No such file or directory [C:\PowerShell\src\powershell-native\nativemsh\pwrshcommon\pwrshcommon.vcx proj] 0 Warning(s) 1 Error(s) Time Elapsed 00:00:12.61 Execution of { Invoke-Expression -Command:$command } failed with exit code 1 At C:\PowerShell\build.psm1:2225 char:13 throw "Execution of {$sb} failed with exit code $LASTEXIT ... ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : OperationStopped: (Execution of { ...ith exit co de 1:String) [], RuntimeException + FullyQualifiedErrorId : Execution of { Invoke-Expression -Command:$command } failed with exit code 1 [buildlog.txt](https://github.com/PowerShell/PowerShell/files/691351/buildlog.txt)

[brycem]

The missing 'corerror.h' header can be found under 'C:Program Files (x86)Windows KitsNETFXSDK4.6.1Includeum' and that path is in the INCLUDE path:

INCLUDE=C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE ;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE ;C:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\ucrt ;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um ;c:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\shared ;c:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\um ;c:\Program Files (x86)\Windows Kits\10\include\10.0.14393.0\winrt

C:\>dir "Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\Include\um" /b alink.h asmparse.grammar clrdata.h clrdata.idl cor.h cordebug.h cordebug.idl CorError.h CorHdr.h corhlpr.cpp corhlpr.h corprof.h corprof.idl corpub.h corpub.idl corsym.h CorSym.idl fusion.h gchost.h gchost.idl ICeeFileGen.h isolation.h ivalidator.h ivalidator.idl ivehandler.h IVEHandler.idl metahost.h mscoree.h opcode.def openum.h StrongName.h tlbref.h tlbref.idl VerError.h

[StefanScherer]

@brycem Interesting image. I had a look at the layers and for some reason it uses not the official (foreign) Windows base images. That's why someone has to download the first two layers of 4.8 GByte. This could be saved if the image is directly derived FROM microsoft/windowsservercore from the Docker Hub. So the download size would go down from 8 GByte to 3.2 GByte.

PS C:\> docker run stefanscherer/winspector brycem/msbuild
Image name: brycem/msbuild
Tag: latest
Number of layers: 27
Sizes of layers:
  sha256:c480435b7cbad5a6ade1a89fa1e133618b0ad4d5f3ef542318330e6aa051a60b - 4175157401 byte
  sha256:34236d34e62437a094777c475360c01a0e35bce00eb110f5c1d23efad369d729 - 861744788 byte
  sha256:8a593722c24de028da15492b0bdd009d8918b50bd0b23fb58c82c213644be26b - 1214 byte
  sha256:1e02b4158ec298f03b35db322b6fa2a5755d88decfd439e512cece060cfc7601 - 1216 byte
  sha256:eb667104ecb0e9df9c323f1827b8ab1a351edc52679efb8bff8e95ba635ad69d - 1216 byte
  sha256:208c611b9463c3c80a2ba36369a55a941426d9f21a8a1e70f62cc330b603d7f3 - 1223 byte
  sha256:5907f800aa21afb72770997364ea1ed327d4c35247f5861edff3993a58648d59 - 1220 byte
  sha256:9240eb95468ef9da8f20e13b70632837d683f059b3a1baf05eabea1d1b7e12b3 - 1219 byte
  sha256:fcaabc7943aa80cda572f613f7bc237944646d64b27226f88c062a21a9747aaa - 1221 byte
  sha256:eb6ff87d96cdd75a11d336401a04ec29d6a29113dbd0385769592f27ce8b9bd9 - 1217 byte
  sha256:7fe014b5aa7bc7d104674207618de2657b4551518e2e04f033ff3e5c99f3cea1 - 1218 byte
  sha256:7478165d821ccd24b718cbb6976284f103132106fce69b6cb9669303ca322f1c - 740973 byte
  sha256:016fab8099ecbed4bb98db39225aa5dfe7b479449b447ad6fa2a92399c33e97e - 357653819 byte
  sha256:04a51b13d6086a36d6be67278e594a522de5377c8e92f0b76aa0b9547e89144b - 2954146 byte
  sha256:2c5f3ea6208fe824652ecda0076464ba3e5b4eeb3cc3c1ae344c2ea82919a82a - 3005 byte
  sha256:57ef71a5e22f6bceb5653e82670457135bbb688d875b16f7586d9eded7dd9a46 - 2321023663 byte
  sha256:34f8316c35d4ae311f7b850b423aae0507d20b42b642a90f5399311d6267fc11 - 1218 byte
  sha256:2f3a7a06893b4d9acb9d82ad4913eb0ce5d22a6b1e08459d171a1735c7e26900 - 343102825 byte
  sha256:105e67733055d25a8adca1ade281c55fd6845d979314577de86e863119ca3aa8 - 1214 byte
  sha256:16a55606cea5e4f263e6722b490b34dd5f1e38b9cf3d5c742680da4d85b2ded9 - 1211 byte
  sha256:d507949f2561c2e31626a0db6e83ce76d54e75d1461d1177ccf615717027070b - 1212 byte
  sha256:5061ca3c7ae9b60f509b86922c11cf801981557bbd4369d83b3c493fb595f2e6 - 1213 byte
  sha256:5d3a0ae2ab84201fa6699dae61dc2814cfc53eff138f342d4aa731bdd860dcb7 - 1209 byte
  sha256:307b1914bb025f6fa0ea0e90305361c78b1af6613cf173611fa4812984c4f591 - 1215 byte
  sha256:b7217039fbfe360d21ad0ec6190672ccdd8490d8337c9d465d5b57d9b0ae65cc - 286601 byte
  sha256:eff306e2094c7b0f79477dc5814ff92e5c9ab8bfbf93695457aa55d0819f5e37 - 1214 byte
  sha256:58d43e184362cfcdb42ee2d292eb7793e1e67dbccf19e15a5c7d4d9ee4ef45a9 - 1213 byte
Total size (including Windows base layers): 8062689104 byte
Application size (w/o Windows base layers): 8062689104 byte
Windows base image used:

[brycem]

Interesting - It was built on a Win10 desktop rather than a server OS, so that would likely explain it. I'll rebuild it on server and see if that improves it. Thanks @StefanScherer!

[brycem]

Rebuilt on matching build of server with dockerd 1.12.2-cs2-ws-beta, and @TSEO1 's trick to delete the 'C:ProgramDataPackage Cache' - A wee bit slimmer :-)

latest: digest: sha256:c1e3def332f9a8e56678717a82e9531220c2c8031045f64efb33e65ce87fba44 size: 3668

Powershell is now building, but I'm now getting an error when using the pscore exe:

PS C:>
Oops, something went wrong. Please report this bug with the details below.

Report on GitHub: https://github.com/lzybkr/PSReadLine/issues/new

Last 1 Keys:
p

Exception:

System.ComponentModel.Win32Exception: Incorrect function

[lzybkr]

I've seen other reports of that exception - but I don't know what's going on.

I think our ideal solution for building in a container will rely on the VS 2017 Build Tools as they have a silent install option. I experimented some with it already and it didn't feel quite ready, hopefully it will be by RTM.

[StefanScherer]

@brycem About the problem pushing the two microsoft/windowsservercore base images to Docker Hub: I also managed to do this by rebuilding winspector today. Scratching my head why this could happend. But it is logged in AppVeyor https://ci.appveyor.com/project/StefanScherer/winspector/build/1.0.23#L194
I'll have a deeper look and probably file an issue in docker/docker repo.

[brycem]

@lzybkr , I've bugged the PSReadLine in https://github.com/lzybkr/PSReadLine/issues/492

[brycem]

Closing this issue as we have workarounds - Hopefully the next version of the Windows 10 StandaloneSDK will have this addressed. They have an internal bug tracking this as well (OS:9651133)