Navcontainerhelper: The timestamp signature and/or certificate could not be verified or is malformed.

[sirhc101]

Describe the issue
From this night all of our pipelines crashed in Publish App-step due to the following error message:

[error]The timestamp signature and/or certificate could not be verified or is malformed. (Exception from HRESULT: 0x80096005)

The app has been compiled and signed successfully, but while publishing it crashes.

Scripts used to create container and cause the issue

Param(
    [ValidateSet('AzureDevOps','Local','AzureVM')]
    [Parameter(Mandatory=$false)]
    [string] $buildEnv = "AzureDevOps",

    [Parameter(Mandatory=$false)]
    [string] $containerName = $ENV:CONTAINERNAME,

    [Parameter(Mandatory=$false)]
    [string] $buildProjectFolder = $ENV:BUILD_REPOSITORY_LOCALPATH,

    [Parameter(Mandatory=$false)]
    [string] $buildArtifactFolder = $ENV:BUILD_ARTIFACTSTAGINGDIRECTORY,

    [Parameter(Mandatory=$true)]
    [string] $appFolders,

    [switch] $skipVerification
)

Sort-AppFoldersByDependencies -appFolders $appFolders.Split(',') -baseFolder $buildProjectFolder -WarningAction SilentlyContinue | ForEach-Object {
    Write-Host "Publishing $_"
    Get-ChildItem -Path (Join-Path $buildArtifactFolder $_) -Filter "*.app" | ForEach-Object {

        # get app name
        if ((Test-Path -Path (Join-Path -Path $_.DirectoryName -ChildPath "app.json"))) {
            $appName = (Get-Content -Raw -Path (Join-Path -Path $_.DirectoryName -ChildPath "app.json") | ConvertFrom-Json).name;
            $appVersion = (Get-Content -Raw -Path (Join-Path -Path $_.DirectoryName -ChildPath "app.json") | ConvertFrom-Json).version;

            Write-Host $("App Name: " + $appName)

            if (((Get-BcContainerAppInfo -containerName $containerName) | Select-Object "Name") -match $appName) {
                Write-Host $("Previously installed version of " + $appName + " has been found in container " + $containerName + ".")
                Write-Host $("Uninstalling " + $appName + " in " + $containerName + ". . .")
                UnInstall-BCContainerApp -containerName $containerName -appName $appName
            }
        } else {
            Write-Warning $("Unable to check for previously installed app version, because manifest (app.json) could not be found in " + $_.DirectoryName + ".")
        }

        if (!$appName) {
            Publish-BCContainerApp -containerName $containerName -appFile $_.FullName -skipVerification:$skipVerification -sync -install
        } else {
            Publish-BCContainerApp -containerName $containerName -appFile $_.FullName -skipVerification:$skipVerification -sync

            $ErrorActionPreference = "Stop"
            try {
                Write-Host $("Installing " + $appName + " to " + $containerName)
                Install-BCContainerApp -containerName $containerName -appName $appName -appVersion $appVersion
            } catch {
                Write-Host $("Installation for " + $appName + " in " + $containerName + " failed. Try to apply data upgrade to install.")
                Write-Host $("Starting Data Upgrade for " + $appName + " to " + $containerName)
                Start-BCContainerAppDataUpgrade -containerName $containerName -appName $appName -appVersion $appVersion
            }
            $ErrorActionPreference = "Continue"
        }
    }
}

Full output of scripts

Generating script.
Formatted command: . 'C:\agent\_work\407\s\businessdev.BC.Pipeline\ps\Publish-App.ps1' -appFolders "businessdev.Api.Connector\app" -skipVerification:("***" -eq "")
========================== Starting Command Output ===========================
"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'C:\agent\_work\_temp\3ae9f897-1d6f-4a7b-9575-3fe95003c381.ps1'"
BcContainerHelper version 1.0.17
Publishing businessdev.Api.Connector\app
App Name: 365 business API
Publishing C:\Agent\_work\407\a\businessdev.Api.Connector\app\365 business development_365 business API_15.4.2.1846.app
The timestamp signature and/or certificate could not be verified or is malformed. (Exception from HRESULT: 0x80096005)
at <ScriptBlock>, <No file>: line 15
##[error]The timestamp signature and/or certificate could not be verified or is malformed. (Exception from HRESULT: 0x80096005)
At C:\Program 
Files\WindowsPowerShell\Modules\BcContainerHelper\1.0.17\ContainerHandling\Invoke-ScriptInNavContainer.ps1:39 char:13
+             Invoke-Command -Session $session -ScriptBlock $scriptbloc ...
+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Publish-NAVApp], InvalidOperationException
    + FullyQualifiedErrorId : MicrosoftDynamicsNavServer$BC/nav-systemapplication,Microsoft.Dynamics.Nav.Apps.Manageme 
   nt.Cmdlets.PublishNavApp
    + PSComputerName        : 5b635ec734fbb06d113b1d9c9a41a0518cfb8ba136c0217b804a2632bce66afe
##[error]PowerShell exited with code '1'.
Finishing: Publish App

[pmatsconsulting]

Hi, we have the same issue. All pipelines with different BC versions, different navcontainerhelper and bccontainerhelper versions, on different servers are in error on publishing the new signed App.

[sirhc101]

An addition to this issue: It apparently "only" has something to do with signing. With the -skipVerification parameter everything goes through and the app works fine too.
I have now tested the app in different container versions and all lead to the same problem.

Here is how we signing the app and the console output:
Script

if (-not ($CodeSignPfxFile)) {
    $CodeSignPfxFile = try { $ENV:CODESIGNPFXFILE | ConvertTo-SecureString } catch { ConvertTo-SecureString -String $ENV:CODESIGNPFXFILE -AsPlainText -Force }
}

if (-not ($CodeSignPfxPassword)) {
    $CodeSignPfxPassword = try { $ENV:CODESIGNPFXPASSWORD | ConvertTo-SecureString } catch { ConvertTo-SecureString -String $ENV:CODESIGNPFXPASSWORD -AsPlainText -Force }
}

$unsecurepfxFile = ([System.Runtime.InteropServices.Marshal]::PtrToStringAuto([System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($codeSignPfxFile)))
$appFolders.Split(',') | ForEach-Object {
    Write-Host "Signing $_"
    Get-ChildItem -Path (Join-Path $buildArtifactFolder $_) -Filter "*.app" | ForEach-Object {
        Sign-BCContainerApp -containerName $containerName -appFile $_.FullName -pfxFile $unsecurePfxFile -pfxPassword $codeSignPfxPassword
    }
}

Output

Signing businessdev.Api.Connector\app
BcContainerHelper version 1.0.17
Downloading Signing Tools
Installing Signing Tools
Signing C:\Agent\_work\407\a\businessdev.Api.Connector\app\365 business development_365 business API_15.4.2.1846.app
Done Adding Additional Store
Successfully signed: C:\Agent\_work\407\a\businessdev.Api.Connector\app\365 business development_365 business API_15.4.2.1846.app

[pmatsconsulting]

Yes, but the signed App will never be able to be published on a tenant without -skipverification (and so AppSource, ...)

[sirhc101]

Yes, but the signed App will never be able to be published on a tenant without -skipverification (and so AppSource, ...)

Yes, you are right. I didn't say it wasn't critical. It is critical!

By the way: I checked our certificate and the password. It's all fine. No idea what's wrong...

[staedter]

We are experiencing the same thing with all our Pipelines

[freddydk]

Did anybody try to specify a different timestampserver?
Default is http://timestamp.verisign.com/scripts/timestamp.dll
But I think you can use http://timestamp.geotrust.com/tsa or http://timestamp.digicert.com or other RFC 3161 timestamp servers

[sirhc101]

I used http://timestamp.digicert.com as timestamp server. Now it works! Thank you very much.

But it's a little strange, due to the fact Sign-App worked well, but publishing failed.

[pmatsconsulting]

It's a problem with http://timestamp.verisign.com/scripts/timestamp.dll timestamp server?
This will be fixed or I need to update my builds to use http://timestamp.digicert.com timestamp server?

[staedter]

Yes, switching to http://timestamp.digicert.com worked for us as well. Thanks for the quick support :)

I just ran this command on all our Build Servers:
$bcContainerHelperConfig.timeStampServer = "http://timestamp.digicert.com";$bcContainerHelperConfig | ConvertTo-Json | Set-Content C:\ProgramData\BcContainerHelper\BcContainerHelper.config.json

And now all our pipelines are using the modified BcContainerHelper.config.json, I just wonder if a future BcContainerHelper Update might be incompatible with this version of the Json.

@freddydk Is that something I need to worry about?

[freddydk]

I will investigate and probably change the default.
I will also make sure that sign-bccontainerapp fails if this happens again (instead of signalling success when it is actually not).

BTW - you can safely use a bccontainerhelper.config.json. I might add new settings, but the code reading the config file (here: https://github.com/microsoft/navcontainerhelper/blob/61632814afc0d60118fc8fc056930497197b2d62/BcContainerHelper.psm1#L66) will only read the keys from the settings class. I will never change the meaning of a key - meaning that keys might be obsolete, but it is ok if they are present.

[staedter]

BTW - you can safely use a bccontainerhelper.config.json. I might add new settings, but the code reading the config file [...] will only read the keys from the settings class. I will never change the meaning of a key - meaning that keys might be obsolete, but it is ok if they are present.

Awesome that is great to hear. Really nice design btw :) I guess I will be using it more often from now on.

Let's hope that's the only thing breaking today and I hope you all will have a very happy new year :)

[jwikman]

Hi @freddydk,

These different timestamp servers seems to fail now and then. Not often, but it happens.
What do you think about having a list of timestamp servers as parameter instead? And then if the first fails, you try the second, third and so on...
I've had that in my backlog for years (from signing ClickOnce packages) , but it wasn't as big annoyance back then since that was not fully automated anyway... But in an automated pipeline it would be nice if this would never fail.

[freddydk]

I had the same idea today:-)
I will probably build into the sign function that the timestampserver can be a comma-separated list of timestamp servers and then set the default to 3 of the known servers.
However... - on the error today - the signing did not fail - it succeeded - with a wrong timestamp - I would never be able to catch that unless you publish the app.

[jwikman]

wow, that issue sounds no good...
That must be a very rare issue - We've been signing with a timestamp server for 6+ years, all issues I had would have failed the signing, I had never seen this before.
Solutions is, of course, to test the signing with a publish.

Your suggested solution with default 3 servers sounds great. I can't imagine all three would have issues at the same time!

[freddydk]

Fixed in BcContainerHelper 1.0.18 (not the 3 servers yet, only the defaulting)