Powershell: Reloading module does not reload submodules.

[powercode]

Steps to reproduce

#Sub.psm1:
  class Sub {
    [int] $Value
  }

#modrepo.psm1:
  using module .\Sub.psm1

  function Get-Sub { 
     [Sub]@{
         Value = 42
     }
  }

Import-Module modrepo
Get-Sub

Value : 1

Change the definition of Sub:

#Sub.psm1:
  class Sub {  
    [string] $Name # added field
    [int] $Value
  }

#modrepo.psm1:
  using module .\Sub.psm1

  function Get-Sub{ 
     [Sub]@{
         Name = ‘Staffan’ # added field
         Value = 42
     }
  }

#Then

Remove-Module -force modrepo
Import-Module -force modrepo
Get-Sub 

Expected behavior

output of object of class Sub with

Name : 'Staffan'
Value : 42

Actual behavior

Cannot create object of type "Sub". The Name property was not found for the Sub object. The available property is: [Value <System.Int32>]
At C:\Users\<user>\Documents\WindowsPowerShell\Modules\modrepo\modrepo.psm1:9 char:3
+   [Sub] @{
+   ~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [], RuntimeException
    + FullyQualifiedErrorId : ObjectCreationError

Environment data

Name                           Value                                                                                                                                                       
----                           -----                                                                                                                                                       
PSVersion                      5.1.14393.206                                                                                                                                               
PSEdition                      Desktop                                                                                                                                                     
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                                                                     
BuildVersion                   10.0.14393.206                                                                                                                                              
CLRVersion                     4.0.30319.42000                                                                                                                                             
WSManStackVersion              3.0                                                                                                                                                         
PSRemotingProtocolVersion      2.3                                                                                                                                                         
SerializationVersion           1.1.0.1                                                                                                                                                     

[daxian-dbw]

Hi @powercode, I'm unable to reproduce this issue in both powershell 5.1 and latest powershell core:

powershell 5.1

PS:3> dir

    Directory: C:\arena\tmp

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----       10/19/2016   9:16 PM            127 modrepo.psm1
-a----       10/19/2016   9:16 PM             74 Sub.psm1

PS:4> v .\Sub.psm1
PS:5> v .\modrepo.psm1
PS:6> Import-Module .\modrepo.psm1
PS:7> Get-Sub

Value
-----
   42

PS:8> v .\Sub.psm1
PS:9> v .\modrepo.psm1
PS:10> Remove-Module modrepo -Force
PS:11> Import-Module .\modrepo.psm1 -Force
PS:12>
PS:12> Get-Sub

Name    Value
----    -----
Staffan    42

PS:13> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.14393.206
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.206
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

powershell core

PS C:\arena\tmp> dir

    Directory: C:\arena\tmp

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----       10/19/2016   9:15 PM            128 modrepo.psm1
-a----       10/19/2016   9:15 PM             75 Sub.psm1

PS C:\arena\tmp> Import-Module .\modrepo.psm1
PS C:\arena\tmp>
PS C:\arena\tmp> Get-Sub

Value
-----
   42

PS C:\arena\tmp>
PS C:\arena\tmp> Remove-Module modrepo -Force
PS C:\arena\tmp> Import-Module .\modrepo.psm1 -Force
PS C:\arena\tmp>
PS C:\arena\tmp> Get-Sub

Name    Value
----    -----
Staffan    42

Am I missing anything? Are you able to reproduce the issue in the latest alpha.11 release of powershell core?

[powercode]

I was actually a bit surprised when I created this repro that it failed the first time. I usually get this issue intermittently but it consistently is a pain when using classes in modules. I often end up restarting powershell.exe or the ise to get back to working modules.
I'll try installing the alpha and see if I can repro it there too.

[LaurentDardenne]

I can reproduce this issue on Windows Seven x64 & x86 with PS v5.0.

remove-item c:\temp\Sub.psm1 -ea Silently
remove-item c:\temp\modrepo.psm1 -ea Silently 
@'
  class Sub {
    [int] $Value
  }
'@ >c:\temp\Sub.psm1

@'
  using module c:\temp\Sub.psm1

  [sub]::new()|gm > c:\temp\c1.txt

  function Get-Sub { 
     [Sub]@{
         Value = 42
     }
  }
'@ > C:\temp\modrepo.psm1

Set-location c:\temp
Import-Module c:\temp\modrepo.psm1
Get-Sub

@'
  class Sub {  
    [string] $Name # added field
    [int] $Value
  }
'@ >c:\temp\Sub.psm1

@'
  using module c:\temp\Sub.psm1

  [sub]::new()|gm > c:\temp\c2.txt

  function Get-Sub{ 
     [Sub]@{
         Name = ‘Staffan’ # added field
         Value = 42
     }
  }
'@ > C:\temp\modrepo.psm1

Set-location c:\temp
Remove-Module -force modrepo
Import-Module -force c:\temp\modrepo.psm1
Get-Sub 
#-> Exception

fc.exe c:\temp\c2.txt c:\temp\c1.txt
# Equal
# Comparaison des fichiers C:\TEMP\c2.txt et C:\TEMP\C1.TXT
# FC : aucune différence trouvée

$PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.0.10586.117
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0, 5.0.10586.117}
BuildVersion                   10.0.10586.117
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

$StackTrace

   à System.Management.Automation.LanguagePrimitives.CreateMemberNotFoundError(PSObject pso, DictionaryEntry property, T
ype resultType)
   à System.Management.Automation.LanguagePrimitives.SetObjectProperties(Object o, IDictionary properties, Type resultTy
pe, MemberNotFoundError memberNotFoundErrorAction, MemberSetValueError memberSetValueErrorAction, Boolean enableMethodCa
ll, IFormatProvider formatProvider, Boolean recursion, Boolean ignoreUnknownMembers)
   à System.Management.Automation.LanguagePrimitives.ConvertViaNoArgumentConstructor.Convert(Object valueToConvert, Type
 resultType, Boolean recursion, PSObject originalValueToConvert, IFormatProvider formatProvider, TypeTable backupTable,
Boolean ignoreUnknownMembers)

[powercode]

I did the following with trace-command to see module logging (still on the desktop):
Anything else you can do to trace the caching of modules?

[datetime]::Now
$modPath = '~/documents/windowspowershell/modules/modrepo/'
$null = mkdir $modPath -ea:0


Set-Content $modPath/modrepo.psm1 -Value @'
using module .\Sub.psm1
function Get-Sub{  
  $s = [Sub]::new(); 
  $s | get-Member | Format-Table 
  $s.ToString();
  $s.GetType().Assembly.Modules.ModuleVersionId 
}
'@

$sub1 = @'
class Sub {
  [int] $Value 
  #[string] $Name
  Sub(){
    $this.Value = 42
    #$this.Name='Answer'
  }     
  [string] ToString() {return 'I am the first version'}
}   
'@

$sub2 = @'
class Sub {
  [int] $Value 
  [string] $Name
  Sub(){
    $this.Value = 42
    $this.Name='Answer'
  }     
  [string] ToString() { return 'I am new and improved' }
}   
'@


Set-Content $modPath/sub.psm1 -Value $sub1
Trace-Command -Expression {Import-Module -Force modrepo} -PSHost Modules
Get-Sub

# now change the definition of sub.psm1
Set-Content $modPath/sub.psm1 -Value $sub2
Trace-Command -Expression {Import-Module -Force modrepo} -PSHost Modules
Get-Sub

The output is as follows. Note that the definitions of Sub has not been been updated and the same dynamic assembly is used.

`````` powershell

den 20 oktober 2016 11:55:25
DEBUG: Modules Information: 0 : WriteLine C:\Users\powercode\Documents\WindowsPowerShell\ModulesmodrepoSub.psm1: cache entry out of date, cached on 2016-10-20 11:29:20, last updated on 2016-10-20 11:55:25
DEBUG: Modules Information: 0 : WriteLine Returning NULL for exported commands.
DEBUG: Modules Information: 0 : WriteLine Analyzing path: C:\Users\powercode\Documents\WindowsPowerShell\ModulesmodrepoSub.psm1
DEBUG: Modules Information: 0 : WriteLine Requested caching for Sub
DEBUG: Modules Information: 0 : WriteLine Caching command: Sub
DEBUG: Modules Information: 0 : WriteLine Requested caching for Sub
DEBUG: Modules Information: 0 : WriteLine Existing cached info up-to-date. Skipping.
DEBUG: Modules Information: 0 : WriteLine Requested caching for modrepo
DEBUG: Modules Information: 0 : WriteLine Caching command: Get-Sub
DEBUG: Modules Information: 0 : WriteLine Caching command: Sub

TypeName: Sub

Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
Value Property int Value {get;set;}

I am the first version

DEBUG: Modules Information: 0 : WriteLine Requested caching for Sub
DEBUG: Modules Information: 0 : WriteLine C:\Users\powercode\Documents\WindowsPowerShell\ModulesmodrepoSub.psm1: cache entry out of date, cached on 2016-10-20 11:55:25, last updated on 2016-10-20 11:55:31
DEBUG: Modules Information: 0 : WriteLine Caching command: Sub
DEBUG: Modules Information: 0 : WriteLine Requested caching for modrepo
DEBUG: Modules Information: 0 : WriteLine Existing cached info up-to-date. Skipping.

Guid

fa9abef1-c3b5-4c5f-88f2-aa11f6a8d427

TypeName: Sub

Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
Value Property int Value {get;set;}

I am the first version

Guid

fa9abef1-c3b5-4c5f-88f2-aa11f6a8d427```

``````

[iSazonov]

I confirm @LaurentDardenne's test on:

$PSVersionTable

Name                           Value
----                           -----
WSManStackVersion              3.0
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   3.0.0.0
PSEdition                      Core
SerializationVersion           1.1.0.1
PSVersion                      6.0.0-alpha
GitCommitId                    v6.0.0-alpha.11-39-g0104d205a46846b9df0ef4940eedb226d21324dc
PSRemotingProtocolVersion      2.3
CLRVersion

[powercode]

Modrepo.ps1.txt

I can reproduce the problems on WSL

Name                           Value
----                           -----
PSVersion                      6.0.0-alpha
PSEdition                      Core
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   3.0.0.0
GitCommitId                    v6.0.0-alpha.11
CLRVersion
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

[iSazonov]

@daxian-dbw

Am I missing anything?

Do you modify submodule before second test?

[powercode]

@iSazonov
Yes, he got

Value
-----
   42

the first run and

Name    Value
----    -----
Staffan    42

the second.

So there are some other factors involved here too.

[powercode]

Modrepo.Tests.ps1.txt

Attaching a pester test. It is failing on

PSVersion     BuildVersion   PSEdition GitCommitId            
---------     ------------   --------- -----------            
6.0.0-alpha   3.0.0.0        Core      v6.0.0-alpha.11 wsl (Ubuntu 14)  
6.0.0-alpha   3.0.0.0        Core      v6.0.0-alpha.11 windows
5.1.14393.206 10.0.14393.206 Desktop                          

[joeyaiello]

@vors: Do you know if this is a known problem or not? I vaguely remember a conversation about classes getting cached in perpetuity for a given runspace?

[powercode]

Are you saying that you can't repro this?
I have no machine where it works.

[pcgeek86]

Does this have to do with the inability to unload an assembly from an AppDomain, perhaps? What does the output of [System.AppDomain]::CurrentDomain.GetAssemblies() look like, after each execution? If I recall correctly PowerShell generates dynamic, in-memory assemblies that only last for the lifetime of the process (or maybe runspace, as Joey mentioned).

[pcgeek86]

Further reading on .NET: http://stackoverflow.com/questions/123391/how-to-unload-an-assembly-from-the-primary-appdomain

[daxian-dbw]

Thank you @powercode! I'm able to reproduce the issue with your script Modrepo.Tests.ps1.txt. I will investigate further to find the root cause.

[powercode]

@pcgeek86 Not really. New assemblies are generated for each edited version of a module. So lots of assemblies with the same name but with a unique guid. And PowerShell needs to map the name of a class to the correct version of the generated assembly.

[pcgeek86]

@powercode Good info! Appreciate the clarification.

[lzybkr]

There is a special kind of dynamic assembly that can be unloaded and garbage collected - PowerShell uses that kind of assembly for our implementation of classes.

And yes, @powercode is correct, we do maintain a mapping of name to assembly for PowerShell classes - that mapping is checked for all typenames before using reflection to resolve a typename.

[powercode]

@lzybkr Thinking about that: is there a side-by-side story in PowerShell? If multiple versions of an assembly is loaded, can we indicate which one to use? I.E. the fullname of the type?

[lzybkr]

An ugly solution is to use the AssemblyQualifiedName, e.g.

[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]

[iSazonov]

Is it impossible 'Import-Module -Force' first unload the old version of the assembly?

[powercode]

@daxian-dbw Found anything?

[daxian-dbw]

@powercode Sorry that I was distracted by other things and didn't get back to this issue. I will post my update by end of this week.

[daxian-dbw]

@powercode There are 2 problems in the module loading:

1. Import-Module modrepo -Force doesn't reload the nested module of modrepo (Sub.psm1 in this case) when modrepo was already loaded. So when running Import-Module modrepo -Force, it would actually reuse the same Sub module loaded previous, even though that module file had been changed.
2. The module analysis result is stored in a cache with the module file path as the key and the PSModuleInfo object as the value. The cache entries are not properly invalidated based on the LastWriteTime of the module file, and thus same cached value got reused.

More details for #1:
When running Import-Module modrepo -Force, if modrepo was already loaded, powershell would remove the module first. However, it doesn't handle the nested modules in this case and thus the nested module stays in the module table. Then when reloading modrepo, the old nested module would be reused.
Concrete repro:

PS C:\temp>   $sub1 = @'
>> class Sub {
>>   [int] $Value
>>   #[string] $Name
>>   Sub(){
>>     $this.Value = 42
>>     #$this.Name='Answer'
>>   }
>>   [string] ToString() {return 'I am the first version'}
>> }
>> '@
PS C:\temp>
PS C:\temp>   $sub2 = @'
>> class Sub {
>>   [int] $Value
>>   [string] $Name
>>   Sub(){
>>     $this.Value = 4711
>>     $this.Name='Answer'
>>   }
>>   [string] ToString() { return 'I am new and improved' }
>> }
>> '@
PS C:\temp> Set-Content .\modrepo\Sub.psm1 -Value $sub1
PS C:\temp> $s = Import-Module .\modrepo -PassThru
PS C:\temp> Set-Content .\modrepo\Sub.psm1 -Value $sub2
PS C:\temp> $v = Import-Module -Force .\modrepo -PassThru
PS C:\temp> $v.NestedModules[0].Name
Sub
PS C:\temp> $s.NestedModules[0].Equals($v.NestedModules[0])
True

More details for #2:
When you have using module .\Sub.psm1, Get-Module <path-to-Sub.psm1> -ListAvailable would be used to analyze the module. However, since the analysis result is cached and not properly invalidated, the second call to Get-Module after changing Sub.psm1 would return the same result as before the change (clones of the cached value). Therefore, the class definition being used is stale.
Concrete repro:

PS C:\temp> Set-Content .\modrepo\Sub.psm1 -Value $sub1
PS C:\temp> $first = Get-Module C:\temp\modrepo\Sub.psm1 -ListAvailable
PS C:\temp> Set-Content .\modrepo\Sub.psm1 -Value $sub2
PS C:\temp> $second = Get-Module C:\temp\modrepo\Sub.psm1 -ListAvailable
PS C:\temp> $first.Name
Sub
PS C:\temp>
PS C:\temp> $first.GetExportedTypeDefinitions()

Key Value
--- -----
Sub class Sub {...

PS C:\temp> $firstType = $first.GetExportedTypeDefinitions()
PS C:\temp> $secondType = $second.GetExportedTypeDefinitions()
PS C:\temp> $firstType.Sub.Equals($secondType.Sub)
True

I had troubles reproducing the issues interactively in powershell console, and it turned out that was because the analysis cache from #2 would be cleared by tab completion when completing module name for cmdlets like Import-Module mod<tab>. That's probably by design, but it did make it harder to get to the root causes.

[alx9r]

Thank you for sharing your analysis @daxian-dbw. I've been trying to understand this behavior myself, and your last post helps.

Can you confirm that the correct behavior for Import-Module -Force is to reload the module and its nested modules regardless of whether any module file has changed? This wasn't clear from your last post. Modules can keep all manner of internal state. Intuitively I'd expect Import-Module -Force to result in the same state as Import-Module would for that module in a fresh session. That behavior would also be consistent with the help which reads as follows:

-Force
Indicates that this cmdlet re-imports a module and its members, even if the module or its members have an access mode of read-only.

As best I can tell neither PowerShell 5.0, 5.1, nor 6.0.0-alpha.13 behave in this manner consistently. Rather, those versions all seem to skip reloading modules in some circumstances despite using -Force.

There's a fairly significant test matrix to consider. Here's a test I have written that reloads modules after changing their module arguments, module files, and submodule files then tests the results. Here is the summary of the test for PowerShell 5.0, 5.1, and 6.0.0-alpha.13:

• submodules never seem to get reloaded
• unless the module file is changed, a module containing a class never gets reloaded whether it is a submodule or the module named in the Import-Module -Force command

These are pretty painful results because it means that you have to start a new PowerShell session to have any certainty over the internal state of any module. During development I am launching a new PowerShell process to run each test. powershell.exe takes about 15 seconds to load itself, Pester, and the module-under-test, so a lot of developer time is spent just waiting to clear state to get a meaningful test result on each iteration. And if you want to do any meaningful interactive debugging you have to open a fresh instance of ISE for each test which also takes time.

I'm really hoping that module reloading can be cleaned up.

Thanks for your help with this,

Alex

[iSazonov]

Can you confirm that the correct behavior for Import-Module -Force is to reload the module and its nested modules regardless of whether any module file has changed?

It would be good to have two possibilities:

1. Reload only changed module(s) and submodules based on LastWriteTime
2. Force reload module(s) and submodules regardless of the LastWriteTime

[alx9r]

@iSazonov

1. Reload only changed module(s) and submodules based on LastWriteTime

Can you share an example use case this behavior would be useful for?

[daxian-dbw]

@alx9r Thanks for the thorough pester tests!

Can you confirm that the correct behavior for Import-Module -Force is to reload the module and its nested modules regardless of whether any module file has changed?

I believe Import-Module -Force Foo should work the same as Remove-Module Foo -Force; Import-Module Foo, and that means it should remove the Foo module and its nested modules, and then import it again.

unless the module file is changed, a module containing a class never gets reloaded whether it is a submodule or the module named in the Import-Module -Force command

I played with your pester tests and found that the root module with a class definition does get reloaded with Import-Module -Force, however, the class definition from it doesn't get updated unless the module file has been changed since the last loading.

Steps to reproduce:

PS F:\tmp> $content = @'
>> $passedArgs = $Args
>> class Root { $passedArgs = $passedArgs }
>> function Get-PassedArgsRoot { [Root]::new().passedArgs }
>> function Get-PassedArgsNoRoot { $passedArgs }
>> '@
PS F:\tmp>
PS F:\tmp> Set-Content .\testmodule\testmodule.psm1 -Value $content
PS F:\tmp> Import-Module .\testmodule -ArgumentList 'value1'
PS F:\tmp> Get-PassedArgsNoRoot
value1
PS F:\tmp> Get-PassedArgsRoot
value1
PS F:\tmp>
PS F:\tmp> Import-Module .\testmodule -ArgumentList 'value2' -Force
PS F:\tmp> Get-PassedArgsNoRoot  ## indicate the module did get reloaded
value2
PS F:\tmp> Get-PassedArgsRoot
value1
PS F:\tmp>
PS F:\tmp> Add-Content .\testmodule\testmodule.psm1 -Value "`n" ## add a newline char to the file
PS F:\tmp> Import-Module .\testmodule -ArgumentList 'value3' -Force
PS F:\tmp> Get-PassedArgsNoRoot
value3
PS F:\tmp> Get-PassedArgsRoot
value3

Actually, the class definition doesn't get updated even you run Remove-Module and then Import-Module:

PS F:\tmp> Remove-Module testmodule
PS F:\tmp> Import-Module .\testmodule -ArgumentList 'value4' -Force
PS F:\tmp> Get-PassedArgsNoRoot
value4
PS F:\tmp> Get-PassedArgsRoot
value3

It looks like there is some caching for powershell class definition, which I need to investigate further to find out.

[alx9r]

@daxian-dbw

Thanks for the thorough pester tests!

You're welcome. :)

I believe Import-Module -Force Foo should works the same as Remove-Module Foo -Force; Import-Module Foo, and that means it should remove the Foo module and its nested modules, and then import it again.

That sounds like good behavior to me.

I played with your pester tests and found that the root module with a class definition does get reloaded with Import-Module -Force, however, the class definition from it doesn't get updated unless the module file has been changed since the last loading.

Actually, the class definition doesn't get updated even you run Remove-Module and then Import-Module:

Interesting. I've expanded the test matrix to cover the different reloading methods. Import-Module -Force definitely behaves different from Remove-Module -Force; Import-Module.

It looks like there is some caching for powershell class definition, which I need to investigate further to find out.

That explains a good chunk of the problems I've seen related to module reloading. The results of the automated test DifferentArgs_Classes_ForceRemoveThenImport is consistent with caching of the class definition.

I'll try to summarize the findings:

1. With current versions of PowerShell use Remove-Module; Import-Module instead of Import-Module -Force. Remove-Module; Import-Module reloads nested modules (but not class definitions contained in them) whereas Import-Module -Force does not.
2. With current versions of PowerShell, the only known way to reload a class definition is to modify its module file.

There is another scenario that isn't part of the automated test: Dot-sourcing files in the .psm1. Dot-sourcing files in .psm1 files is commonplace in bigger modules. Pester is one example. ZeroDSC is another. I wonder whether the cache validation (for both modules and class definitions) is handled correctly when .ps1 files are dot-sourced in .psm1 files.

[iSazonov]

Above I meant that perhaps we need some optimization of module reload that can be significantly for _large_ modules. For example, to reload only those modules or/and submodules that actually changed. This can significantly reduce the time needed to reload.
Maybe @daxian-dbw comment this.

[daxian-dbw]

@iSazonov I prefer to keep the logic simple so that it's easy to reason and maintain. Here are my arguments:

1. There are other cases where we want to reload a module even if the module file is not changed since the last loading, for example, Import-Module -Force -ArgumentList uses different arguments which might affect the module behavior.
2. IMHO, the performance of Import-Module -Force is not a problem unless there is data to prove I'm wrong. Besides, Import-Module -Force is more of a developer scenario, where I think the easy-to-reason advantage would outweigh the slight performance gain.

[lzybkr]

The right way to think about performance here is - make Import-Module fast, then Import-Module -Force will be fast too.

[iSazonov]

@daxian-dbw @lzybkr Thanks for comments! "make Import-Module fast" is great. My concern about the performance of a module reload comes from the fact that some modules are heavy enough and if the custom module depends on such module, the user may be dissatisfied with the fact that reload of his little module leads to the heavy module is reloaded too. But I agree that is more of a developer scenario and a full reload is acceptable.

[daxian-dbw]

This is an update to my post about class definition reloading with module reloading.

The root cause is that powershell doesn't correctly refresh the SessionState associated with the class definition when a module gets reloaded. So the PS class is still referencing to the SessionState from the old module instance, and thus doesn't get the module state changes that happen during the reloading.
The PR #2837 was submitted to fix this particular issue.

There are still 2 more issues, and the detailed information can be found at this comment.

[powercode]

@alx9r

During development I am launching a new PowerShell process to run each test. powershell.exe takes about 15 seconds to load itself, Pester, and the module-under-test, so a lot of developer time is spent just waiting to clear state to get a meaningful test result on each iteration

When the loading times are that bad, it may be caused by your organization having ExecutionPolicy set by a group policy, triggering a very unfortunate code path in PowerShell.

I personally have a scheduled task to remove the group policy key just to get PowerShell usable. I had load times of 30+ seconds just loading my profile.

This is what I run (vbscript to not get a window popping up when the scheduled task is executed).

on error resume next
Set objShell = Wscript.CreateObject("Wscript.Shell")
objShell.RegDelete "HKLM\SOFTWARE\Policies\Microsoft\Windows\PowerShell\"

See #2578. Some fixes has been made that are expected in the January release if I understand correctly.

[AlFeersum]

WRT Import-/Remove-Module caching subs in 5.0/5.1 (ref: https://github.com/PowerShell/PowerShell/issues/2505#issuecomment-263369811) - this also happens with 'using module xxx.psm1'

xxx.psm1 contains a class definition, which gets loaded into the core script. If a method or property in xxx.psm1 changes, the module isn't reloaded as part of an execution, despite the module being listed with Get-Module, and successfully being removed with Remove-Module.

If addressing this for Import-/Remove-Module, consideration needs to be made for 'using module ...' - presumably the same underlying code is used, but just in case...

[joeyaiello]

This is super-annoying, obviously, but given our desire to postpone classes work until 6.0.0 is stable, I'm moving this out to 6.1.0.

[garfbradaz]

Is this fixed yet? Happens to me as well when using "using"

using module 'C:\Program Files\WindowsPowerShell\Modules\GitHubClient\GitHubClient.psm1'

If i make changes to my classes within 'GitHubClient.psm1' i have to close and re-open PowerShell ISE.

[mallochine]

yeah........I hit this while on pwsh in Linux.

$ pwsh --version
PowerShell v6.0.1

[BrucePay]

Some historical design notes i.e. what we were thinking when we designed this:

Import-Module -Force was explicitly the equivalent of Remove-Module ... then Import-Module .... The core scenario was for module developers who needed to force a reload (reset) of their module.

Doing an Import-Module -Force does not reload sub-modules by design. (Similarly Remove-Module does not remove dependent modules.) The reason for this is that the dependent modules may have been imported by more than one parent module. If you forced the reload of the dependent modules you might break something else. The solution is to require the developer to explicitly do a reload (or remove) of each module. (FWIW I now think we got this wrong. Since the scenario is developers, having a possibly unstable session is just a fact of life and so Import-Module -Force should reload everything in NestedModules recursively.)

Getting Remove-Module to work even tolerably well was ludicrously difficult. Removing modules is tricky, because there can be references to module elements in a variety of places. So even if the module is removed from the module table, it may still be in memory because of these references. Loading that module again will result in two copies of the module. (Note: we did consider reference counting for unloads but didn't ever get to it.) Ick.

Removing a module that exports types is particularly tricky since there may be references to the Type object in various places.

using module doesn't have a -Force flag and shouldn't. Perhaps we need a developer mode in PowerShell Set-DeveloperMode -on to turn on module auto-reloading...

(I've probably forgotten some possibly significant stuff. If I remember anything, I will add it later.)

@iSazonov

Above I meant that perhaps we need some optimization of module reload that can be significantly for large modules.

Given that this is a developer scenario not a production one (and assuming it isn't ridiculously slow) do we really need this? And even if a file hasn't changed, if one of it's dependent modules is updated, the depending module still has to be reloaded. As @lzybkr said, making Import-Module fast is probably where the work should go. When I wrote the module code in V2, performance was not one of my goals. As far as I know, we've never done any performance work since then.

[iSazonov]

It is clear and simple design. Sometimes PowerShell do magic things. Perhaps here it could do it too.
We could inform the user that there are submodules which are not reloaded or to request their forced reload or do something more magic.

[rjmholt]

I think, on the face of it, the intent seems simple. But after spending a fair amount of time reading and stepping through the Import-Module code, my feeling is that magic is what's gotten us into trouble here, rather than knowing what our promised functionality is and at what point the onus is back on the scripter.

My concern is that Import-Module's behaviour is already so complicated as to make it hard to maintain, and it gets called a lot (e.g. several times per completion). So really I think we should make a clear decision on what its contract with the invoker is, and ensure that the code reflects that (both tasks being still quite tricky).

[SCLDGit]

After skimming through this thread, are we to understand that currently there is no way to reliably auto-update modules with classes/methods? Having both Remove-Module ModuleName -Force and Import-Module .\Module\Location\ModuleName.psm1 -Force fixes nothing here, and necessitating a Visual Studio/ISE restart every time a code change is made makes development a non-starter.

[alx9r]

@SCLDGit You might be interested in this summary of the issue, its implications, and workarounds.

[SCLDGit]

@alx9r That was useful for confirmation, but I can't believe that anyone would accept that kind of workaround for anything other than the simplest unit test and development cases. It's kind of mind-boggling that this issue was never addressed in the design phase.

[alx9r]

...I can't believe that anyone would accept that kind of workaround for anything other than the simplest unit test and development cases.

@SCLDGit FWIW, the workaround is somewhat less painful using VS Code compared with ISE. VS Code (v 1.22.2, at least) allows you to reload the PowerShell environment without restarting the editor. So your workspace survives changes to class definitions.

[SCLDGit]

@alx9r That's good to know. I couldn't manage to pull it off with the Visual Studio extension, but if VS code can do it I may just use that for PS development. Thanks.

[rjmholt]

Another possible workaround is to manually reset the offending cache:

function Clear-ScriptAnalysisCache
{
    $fieldInfo = ([Microsoft.PowerShell.Commands.ModuleCmdletBase].GetField("s_scriptAnalysisCache", [System.Reflection.BindingFlags]::NonPublic -bor [System.Reflection.BindingFlags]::Static))[0]
    $scriptAnalysisCache = $fieldInfo.GetValue($null)
    $scriptAnalysisCache.Clear()
}

The problem is that s_scriptAnalysisCache is not thread-safe and currently there's a lock object that protects access. So to do this properly (without corrupting the cache) you'd need to find Microsoft.PowerShell.Commands.ModuleCmdletBase.s_lockObject using the same reflection technique and do some locking in PowerShell on that object.

Which is really not a great workaround (but unlike restarting things, is simpler to encapsulate in a function).

I'm currently working on a fix for this.

[rjmholt]

Going to tag this for review by the @PowerShell/powershell-committee , with the specific question of:

Should Import-Module -Force recursively reload nested modules, or should it only reload the top-level module.

@daxian-dbw's comment and @BrucePay's comment seem to weigh the issue well.

[iSazonov]

Should we think about long sessions (login shell), runspaces and concurrency? What if I run a script in one runspace and then I start reload a module in other runspace? Should we use runspace/global cache and/or something like "copy-on-write"?

[SteveL-MSFT]

@PowerShell/powershell-committee reviewed this and rather than breaking -Force, we prefer to add -Recurse to Import-Module and also Remove-Module for this desired behavior used with -Force and would not be a breaking change.

[rjmholt]

It looks like Remove-Module removes nested modules recursively even without the -Force flag.

Is the @PowerShell/powershell-committee's new desired behaviour with the -Recurse flag on Remove-Module that we only recursively unload with -Force and -Recurse specified? Or just -Recurse? And does that constitute a breaking change we should be worried about?

[SteveL-MSFT]

@rjmholt the intent is that -Recurse is used with -Force (part of same parameterset). Because this is a new switch, it avoids the breaking change. Note that -Recurse isn't just about nested modules, but dependent modules as well.

[rjmholt]

It looks like -Force isn't part of any parameter set. Import-Module's parameter sets are things like Name, ModuleInfo, Assembly, PSSession and you can use -Force with all of them -- do we have a mechanism available to say "-Recurse is only allowed when -Force is present"?

By dependent modules above, do you mean ("nested modules" ∪ "required modules") (to use the language on the module manifest documentation page)?

I think the remaining (non-cache-related) complaint in the issue is described by this Pester test:

Describe "Nested module refreshing in script modules" {
    BeforeAll {
        $testRoot = Join-Path $TestDrive "nestedModuleTest"
        $mod1Name = "mod1"
        $subModName = "sub"
        $mod1Path = Join-Path $testRoot $mod1Name
        $subModPath = Join-Path $testRoot $subModName
        $mod1PsmPath = Join-Path $mod1Path "$mod1Name.psm1"
        $subModPsmPath = Join-Path $subModPath "$subModName.psm1"

        foreach ($modPath in $mod1Path,$subModPath)
        {
            New-Item -Path $modPath -ItemType Directory
        }
    }

    BeforeEach {
        Get-Module $mod1Name,$subModName | Remove-Module -Force

        New-Item -Path $mod1PsmPath -Force -Value @"
Import-Module $subModPath

function Test-TopModuleFunction
{
    Test-SubModuleFunction
}
"@


        New-Item -Path $subModPsmPath -Force -Value @"
function Test-SubModuleFunction
{
    "Hello"
}
"@
    }

    It "Does not refresh the submodule definition when Import-Module -Force is used" {
        Import-Module $mod1Path

        Test-TopModuleFunction | Should -BeExactly "Hello"

        Set-Content -Path $subModPsmPath -Force -Value "function Test-SubModuleFunction { 'Aloha' } "

        Test-TopModuleFunction | Should -BeExactly "Hello"

        Import-Module -Force $mod1Path

        # NOTE: We imported with force here, but the submodule function is not refreshed
        Test-TopModuleFunction | Should -BeExactly "Hello"
    }

    It "Refreshes the submodule defintion when Remove-Module is used without -Force" {
        Import-Module $mod1Path

        Test-TopModuleFunction | Should -BeExactly "Hello"

        Set-Content -Path $subModPsmPath -Force -Value "function Test-SubModuleFunction { 'Howdy' }"

        Test-TopModuleFunction | Should -BeExactly "Hello"

        Remove-Module $mod1Name

        Import-Module $mod1Path

        # NOTE: We never used -Force with Remove-Module, but the submodule is refreshed
        Test-TopModuleFunction | Should -BeExactly "Howdy"
    }
}

Like @BrucePay has described, we don't refresh submodules with -Force by design (although the presence of this issue indicates that that's not what everyone expects).

The -Recurse flag seems like the right way to go, but my concerns for it are:

• Import-Module probably needs a way to refresh nested modules without refreshing required modules. This seems like what -Recurse should do to me.
• Remove-Module already recursively unloads nested modules, even without -Force. There's not much else we can do there, because those modules would be orphaned otherwise. (The difference with Remove-Module -Force is that it will remove ReadOnly modules and modules that other modules have as RequiredModules).
• The module cmdlets currently do no manipulation of required modules anywhere. They only check for their existence and raise errors.

So my suggestion is:

• Import-Module should have a -Recurse flag that can only be used with -Force (and errors otherwise -- probably has to be runtime error because I don't think we have a parameter binder mechanism for this), and -Recurse should recursively refresh nested modules only.
• Remove-Module already does the nested module stuff, so no need for a -Recurse flag there.
• If there's demand for a way to unload/refresh required modules, we can add that later (with a more dangerous looking flag like -RemoveRequired or something). But so far in this issue, there has been no mention of required modules.

[iSazonov]

We could implement -Recurse as dynamic parameter.

[aaroncalderon]

@rjmholt Your suggestion is grate.

However, I want to point out one detail, consistency.

Consistency

Is there really a need to add more flags rather than fix the behaviour of the current and documented flag?

AS @alx9r pointed out, the documentation explains that the flag -Force should reload modules and submodules.

-Force
Indicates that this cmdlet re-imports a module and its members, even if the module or its members have an access mode of read-only.

When running Import-Module -Force, one would expect it to behave as documented. -Force Must be used if the module change and one wants to reload the changes, but it will not reload submodules. This contradicts the documentation.

Conclusion

It seems to me that the behavior of the flag -Force should be fixed, rather than add another flag and complicate things further.

[rjmholt]

@aaroncalderon I agree with your preference here. The problem is that the other behaviour is the current one (and has been deemed to be by design). So a recursive reload on -Force would be a breaking change. It can be changed, but needs to go through the PowerShell RFC process to do so.

[eedwards-sk]

A recent project I started, I decided rather than dot sourcing all my library code, I would encapsulate them in basic nested modules and then import them. This would hopefully make the code more re-usable and easier to test.

However, with this current issue, it sounds like it would be better to go back to dot sourcing the library files instead.

So, 👍 to -Recurse or anything that lets me leverage native powershell features for library loading rather than having to manage it myself with dot sourcing

[rjmholt]

So, 👍 to -Recurse or anything that lets me leverage native powershell features for library loading rather than having to manage it myself with dot sourcing

Libraries will load perfectly well; this issue is about the development scenario about reloading (changing the code and then wanting to load the new version to kick out the old version).

If you're talking about .NET DLL libraries, then even solving this issue wouldn't help you; .NET Core doesn't allow assembly unloading, so you just need to run things in a new process each time.

If it's nested PowerShell modules you're trying to reload, you can currently achieve that with Remove-Module $TopLevelModule; Import-Module $TopLevelModule.

[rjmholt]

In fact, dot-sourcing would make the unloading of old versions much harder I'd imagine

[eedwards-sk]

Libraries will load perfectly well; this issue is about the development scenario about reloading (changing the code and then wanting to load the new version to kick out the old version).

Yeah I'm referring to development scenarios (quickly loading/unloading and reloading to test changes).

[alx9r]

@rjmholt @eedwards-sk You might be interested in this comment. My interpretation is that (from the perpective of at least one language author at one time) the _only_ "supported" way of implementing multi-file modules was by way of dot-sourcing from .psm1.