Powershell: Import-PowerShellDataFile Errors on files over 3230 lines

[pulkjr]

Steps to reproduce

1. Create a .psd1 file with 3231 lines in it.
2. Attempt to import with Import-PowerShellDataFile. It should fail stating that there is a dynamic ScriptBlock.
3. Delete one line from the file. It should import successfully.

Expected behavior

> Import-PowerShellDataFile -Path largeConfiguration.psd1
No Error just results

Actual behavior

> Import-PowerShellDataFile -Path largeConfiguration.psd1
Exception calling "SafeGetValue" with "0" argument(s): "Cannot generate a Windows PowerShell object for a ScriptBlock evaluating dynamic
expressions. Dynamic expression:

Environment data

Attempted from multiple different environments without success

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.1.0
PSEdition                      Core
GitCommitId                    6.1.0
OS                             Darwin 17.4.0 Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

> $PSVersionTable                                                      

Name                           Value
----                           -----
PSVersion                      6.1.0-rc.1
PSEdition                      Core
GitCommitId                    6.1.0-rc.1
OS                             Darwin 17.4.0 Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      5.1.17134.228
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17134.228
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

[SteveL-MSFT]

@pulkjr is this a real scenario or a test scenario? Trying to understand how important this is.

[39Delta]

records.txt

I just ran into this myself as I am also seeing this issue with storing record objects in a psd1 file. At about 3000 lines it returns the error complaining about the presence of a dynamic expression. My example psd1 attached fails when passed to 'Import-PowerShellDataFile' command. If you remove one table from the array it imports fine in my environment. Running powershell version 5.1.14409.1005.

This is related to a real scenario in which we are trying to store domain record data in a psd1 file but are unable to import the file after creation unless we decrease the number of records stored. In total after all records are stored the psd1 is 12k+ lines long.

[pulkjr]

@SteveL-MSFT this is a real scenario where I am building configuration data for DSC dynamically and archiving them for use in testing later. When I attempt to import them for pester testing I am unable to with this command. Import-LocalizedData has been working for me as a work around.

[SteveL-MSFT]

The problem seems to be that PowerShell has some hardcoded limits in the AST and for hashtables here. I did a private build increasing the MaxVisitCount to 10000 (from 5000) and was able to import @39Delta records.txt sample.

@lzybkr what is the impact if we remove these limits altogether? Seems like it would just be local memory/cpu consumption for a large AST?

[lzybkr]

Check with @LeeHolmes and/or @JamesWTruher - I think that limit exists to help avoid a DOS style attack.

I suppose it makes sense to add a parameter to the cmdlet to increase the limits.

[SteveL-MSFT]

I couldn't get a definitive conclusion it's safe to completely remove the limits, so we should go with a parameter to set the limits, however, I don't think the user can guess what value to put as it's not the number of lines in the file. So my proposal is to have a -NoLimit switch that removes the limit check altogether but it's opt-in.

[SteveL-MSFT]

@PowerShell/powershell-committee reviewed this and recommends -SkipLimitCheck